How neutral is Alaska News?
We measure our own newsroom the way we wish someone would measure every newsroom: a panel of three AI models from three different companies reads articles with every outlet name and byline stripped out, scores them against a published rubric, and the results ship here, including the weeks we lose. This page is the instrument's first public output and is labeled accordingly: an experiment, with sample sizes shown next to every score.
Latest run: 2026-09-17. Rubric v0.2.0. Panel: anthropic/claude-sonnet-5, openai/gpt-5.6-terra, google/gemini-3.8-flash.
Alaska's top outlets, side by side
Lean is reported three ways because one average can hide two opposite leans: the signed number gives net direction, lean strength ignores direction (it is computed per article before averaging, so left and right cannot cancel), and the count shows how consistently articles register no lean at all. Lean is direction only, not quality: a lean-free article can still be badly sourced, which is what the remaining dimensions (0 to 4, higher is better) measure. Evidence fairness rewards scrutiny proportional to evidence rather than equal time, and source independence asks whether official framing was tested or relayed: a dimension this newsroom carries deliberately, because its articles are drafted from public meetings and records, which is exactly where institutional framing capture would hide. Judges may mark a dimension not applicable for genres where it has no meaning (a weather brief is not penalized for lacking opposing sources).
| Dimension | Alaska News | Anchorage Daily News | Alaska Beacon |
|---|---|---|---|
| Directional lean (-3 left to +3 right) | 0 (no directional lean detected) | 0 (no directional lean detected) | -0.5 (leans left) |
| Lean strength (0-3, direction ignored) | 0 | 0 | 0.6 |
| Articles with no directional lean detected | 10 of 10 | 10 of 10 | 4 of 10 |
| Evidence fairness (0-4) | 3.1 | 3.5 | 3.1 |
| Sourcing (0-4) | 2.5 | 3.3 | 3.1 |
| Source independence (0-4) | 1.8 | 3 | 2.4 |
| Analysis labeling (0-4) | 3.9 | 3.7 | 3.4 |
| Language neutrality (0-4) | 3.9 | 3.7 | 3.3 |
| Articles sampled | 10 | 10 | 10 |
Small samples, honestly labeled: these are first-run numbers, not verdicts. The sample grows weekly, and every score above will move as it does. Alaska Public Media is absent because it cannot be collected fairly: it offers no feed, and its robots.txt disallows article pages for all crawlers.
Access and distribution practices
Deterministic facts, checkable by anyone with a terminal, describing each outlet's publishing choices: these are not neutrality or quality scores, and blocking AI crawlers is a legitimate choice, not a journalistic failing. Full-text RSS means the outlet's journalism arrives without a site visit; the AI column counts crawler user-agents robots.txt turns away entirely.
| Outlet | Paywall | RSS | Full-text RSS | AI crawlers blocked | Readable without JS |
|---|---|---|---|---|---|
| Alaska News | none | yes | no | 0 | yes |
| Anchorage Daily News | metered | yes | yes | 7 | yes |
| Alaska Beacon | none | yes | yes | 12 | no |
| Alaska Public Media | none | none we could find | no | 9 | yes |
| Must Read Alaska | none | yes | no | 0 | yes |
Our own row is not spotless: Alaska News's feed is excerpt-only where Anchorage Daily News and Alaska Beacon publish full text. Publishing that fact here is the point of the instrument. A performance dimension (Google PageSpeed) joins this table once its API access is set up.
Methodology
How the numbers are computed. Each judge scores each article (averaging that judge's repeat runs), the per-article score is the median across the three judges, and an outlet's number is the mean of its per-article scores, shown to one decimal. Per-article scores and judge disagreement are in the raw data download. One boundary, stated plainly: the panel assesses the text, not the world. It scores whether claims are attributed and whether the article shows its checking; it does not verify facts and produces no factuality rating, because a model reading one article about a brand-new local event has no independent knowledge to check it against, and pretending otherwise would be the exact overreach this page exists to avoid.
Partial blinding. Before any model sees an article, outlet names, bylines, self-references, and boilerplate are stripped by a deterministic pass, and a guard refuses to score any article where a known identity string survives. This removes explicit identity, not every clue: house style, subject matter, or distinctive passages could still hint at the source, so we call it partial blinding rather than claiming the judges cannot know.
The panel. Three models from three vendors (anthropic/claude-sonnet-5, openai/gpt-5.6-terra, google/gemini-3.8-flash), each scoring independently at temperature 0. The headline number is the panel median; disagreement between judges is data we keep, not noise we hide. No single human rater sits above the panel: one person's read is one set of unauditable priors, while the panel's disagreement is published.
Validation checks and limitations
Can the instrument detect lean at all? A judge that scores everything "center" is broken. Every run includes a control outlet with a widely documented political lean (Must Read Alaska, rated right-leaning by independent media-bias raters). This run, all three models independently read the control at bias +1 to +2 with the panel median at +1.3, while reading the outlets above near 0. If the control ever stops registering, publication stops until a person has looked. One known-lean outlet is a necessary check, not a sufficient one: the next validation step is article-level controls in both directions, including neutral articles from partisan outlets and slanted ones from mainstream outlets.
External cross-check. The panel never sees outlet names, yet its first-run reads match the established rater ecosystem: Ground News aggregates Anchorage Daily News as Center (our panel: -0.3) and Alaska Beacon as Lean Left (our panel: -1), and the raters underneath split on Beacon the same way our seats did. Convergence with independent human raters, reached blind, is the strongest external evidence this instrument measures something real.
Calibration baseline, and where it came from. The panel was first run against a frozen 14-article reference set. Full provenance, because it matters: the reference audit was produced by an AI assistant and reviewed by a human in conversation, and the rubric's scoring anchors were derived from that same audit, so agreement with the seed set shows the judges reproduce a documented read, not independently established truth. Every later week is scored on articles the rubric has never seen, which is the ongoing check that matters. Bias mean absolute error per seat on the seed set (2 runs per seat): anthropic 0, openai 0.2, google 0. Any change of models or rubric re-scores this set and the discontinuity is published.
Collection ethics. An honestly identified crawler, robots.txt respected, no paywall circumvention, no competitor text republished. Anchorage Daily News and Alaska Beacon are read from the full-text RSS feeds they publish. Scores describe the text, never anyone's intent.
Conflicts, stated plainly. This page grades our own newsroom and is operated by us; the mitigations are the blinding, the cross-vendor panel, and shipping our losses (see the openness table). One panel seat runs a model from Anthropic, whose models also draft Alaska News articles; the other two seats exist so that seat never rules alone.
Raw data. Every number on this page comes from one downloadable JSON (aggregates and sampling frames; no article text). Last updated 2026-09-17; the page and the download update together.