Edition of Wednesday 11 November 2026

Architecture Decisions for Marin 535B-A23B

Comments on “Architecture Decisions for Marin 535B-A23B”

Claim-checked. Scored by all three judges, and its checkable claims were compared with the sources it cites. How far stories are checked

Open Athena's Larry Dial explains the architecture choices behind Marin 535B-A23B, a mixture-of-experts model pretrained on 18T tokens on 768 GB200s. Scaling-ladder runs up to 9e21 FLOPs forecast a final loss near 2.04. Expert parallelism with token dropping was tested for stability: extending context from 4k to 65k raised dropping from 7% to 37%, with eval loss within 0.01 of a dropless baseline. Debugging led to logit z loss, plain Adam on the embedding and a learnable norm on the latent. Most figures come from the team's own ablations.

Read the original at openathena.ai. The judges saw only its text, with the site, the author and link domains removed; they are shown again after ranking.

Why it ranked

No. 13 of 143 with a score of 0.71. The panel rated it highest on technical depth (0.97) and substantiated (0.93), lowest on audience breadth (0.60). Placed No. 13, where the score alone would also have put it: the fourth AI & machine learning story of the day (−0.020). The placement terms move stories; they are not part of the score.

AI judgeJudge 2claude-haiku-5-5
Detailed, well-evidenced engineering write-up of a large MoE pretraining run with ablations and reproducible methods; high technical depth, strong relevance to ML builders, limited breadth beyond AI infrastructure.
AI judgeJudge 3gpt-5.6-luna
A highly detailed first-person engineering account of training a 535B MoE, with architecture specifics, ablations, scaling-law evidence, hardware constraints, stability investigations, and reproducible experimental results. It is broadly valuable to frontier-model practitioners.

Each judge wrote its line after answering the rubric, with the site and the author still hidden. Every answer behind this is in the table under Score.

Score

0.71total under rubric v2.3

Judges’ mean answer per question, on a scale from 0 to 1. Questions marked minus count against the score.
  • Significance0.69
  • Reader interest0.81
  • Audience breadth0.60
  • Practical relevance0.67
  • Substantiated0.93
  • Technical depth0.97
  • New information0.91
  • Counts against: Promotional0.05
  • Counts against: Overclaim0.16

Each judge’s answer per question (0–1), their mean and spread, and what the question adds to the score (weight × mean).

QuestionJudge 1Judge 2Judge 3MeanSpreadWeightContribution (weight times mean)
SignificanceHow much does this change what people in tech know or can do?0.640.550.880.690.33+0.24+0.166
Reader interestWould a well-informed senior engineer or tech founder want to read this today, and likely pass it on?0.670.800.950.810.28+0.26+0.210
Audience breadthHow many people who build technology does it concern?0.490.600.720.600.23+0.13+0.078
Practical relevanceHow useful is it to someone who builds software or tech products?0.520.700.780.670.26+0.22+0.146
SubstantiatedHow well is it backed? Consider whether it is the primary source, and whether its main claims come with data, code, a demo, a reproducible method or citations.0.990.850.950.930.14+0.05+0.046
Technical depthHow technically substantive is it?1.000.920.990.970.08+0.05+0.048
New informationDoes it contain information absent from the stories covered in the last 7 days (list provided)?0.890.850.980.910.13+0.05+0.045
PromotionalIs it mainly marketing with little substance (a press release, ad or sponsored post with no real information)? A launch post that carries real news is not promotional.0.050.080.030.050.05−0.19−0.010
OverclaimDo the headline or conclusions claim more than the body supports?0.190.200.080.160.12−0.11−0.017
Sum of contributions+0.713
Claim-check penalty0.000
Total, as published0.713

Spread is the highest minus the lowest judge answer. marks a spread above 0.40: the judges disagree, and the story is marked contested.

The badge for this ranking, for the author's page or anyone's: one line of HTML, and it links back here.

Ranked #13 on Rubric, 11 Nov 2026
<a href="https://rubric.hrih.si/editions/2026-11-11/13/"><img src="https://rubric.hrih.si/badge/2026-11-11/13.svg" alt="Ranked #13 on Rubric, 11 Nov 2026" width="245" height="28"></a>

Claim check

Claim check: 1 matches source · 2 no source given · 5 couldn’t check. Each claim is compared with the source the article itself cites; a flag is published only when both checkers agree.

  1. No source givenDate

    Marin 535B-A23B finished pretraining on 18T tokens on November 5, 2026.

    The article gives no link or citation for it.

    The article says

    “Marin 535B-A23B finished pretraining on 18T tokens on November 5, 2026”

    The source says

    No source to compare with.

  2. No source givenNumber

    Marin's two prior MoE runs (129B-A16B on 1T tokens and 67B-A2B Snowball on 10T tokens) both landed within 1 percent of their loss target with zero loss spikes.

    The article gives no link or citation for it.

    The article says

    “Both pre-training runs landed within 1 percent of their loss target with zero loss spikes”

    The source says

    No source to compare with.

  3. Matches sourceNumber

    Moving to expert parallelism, adding 50% more routed weights, and moving to normalized latent MoE together gave roughly 1.3× compute efficiency, measured from a scaling ladder from 4e19 to 8e21 FLOPs.

    The cited source says the same thing.

    The article says

    “Moving to EP, adding 50% more routed weights, and moving to normalized latent MoE together gave roughly 1.3× compute efficiency”

    The source says

    “Under an equal-throughput assumption, the loss differences corresponded to an estimated 18–35% compute-efficiency gain over the FSDP baselines”

    Source cited by the article: openathena.ai/blog/expert-parallelism

  4. Couldn’t checkNumber

    Extending context from 4k to 65k raised EP token dropping from 7% to 37%, yet EP's eval loss landed within 0.01 of switching to dropless FSDP.

    The source was unreachable, paywalled or too long, or the two checkers disagreed.

    The article says

    “Token dropping jumped from 7% to 37%, yet EP's eval loss landed within 0.01 of switching to dropless FSDP”

    The source says

    No passage could be compared.

    Source cited by the article: github.com/marin-community/marin/issues/8062

  5. Couldn’t checkNumber

    Adding a learnable RMSNorm to the latent gave 1.25× compute efficiency compared to hyperball with no intervention.

    The source was unreachable, paywalled or too long, or the two checkers disagreed.

    The article says

    “Compared to hyperball with no intervention, this gave 1.25× compute efficiency”

    The source says

    No passage could be compared.

    Source cited by the article: github.com/marin-community/marin/issues/6822

  6. Couldn’t checkBenchmark result

    The hero run finished at a Paloma loss of 1.982 against the ladder's forecast of 2.039.

    The source was unreachable, paywalled or too long, or the two checkers disagreed.

    The article says

    “finishing at 1.982 against a forecast of 2.039”

    The source says

    No passage could be compared.

    Source cited by the article: github.com/marin-community/marin/issues/8435

  7. Couldn’t checkCausal conclusion

    Applying the quantile-balancing bias to raw logits rather than after the sigmoid gave substantially faster balancing; the Kimi K3-style histogram ended at 3.4% token dropping and averaging at 4.9%, versus near 2% on raw logits.

    The source was unreachable, paywalled or too long, or the two checkers disagreed.

    The article says

    “we see substantially faster balancing from logits in our experiments”

    The source says

    No passage could be compared.

    Source cited by the article: github.com/marin-community/marin/issues/8033

  8. Couldn’t checkCausal conclusion

    Without logit z loss, loss blows up in the overtrained regime because the RMSNorm gain on the activation entering the lm_head keeps growing, causing large logit growth.

    The source was unreachable, paywalled or too long, or the two checkers disagreed.

    The article says

    “Running without logit z loss on the lm_head caused loss blow up in the overtrained regime, so we kept it in the recipe”

    The source says

    No passage could be compared.

    Source cited by the article: github.com/marin-community/marin/issues/8227

Also covered by

No other coverage was clustered with this story.

Elsewhere

Elsewhere: HN 3 pts, not on front page

Discussion on Hacker News

Recorded for context only, checked 11 Nov 2026, 05:47 UTC. It played no part in the ranking.

Panel for this edition

Four AI agents ranked this edition: three judges scored every story blind to its site and author, and a fourth compared the top stories’ claims with the sources they cite. Summaries are written by AI.

Model versions are pinned for every edition. How ranking works describes rubric v2.3.

Comments

Readers’ comments on this story have a page of their own, ranked by a public rubric rather than by votes.

Comments on “Architecture Decisions for Marin 535B-A23B”

More on Rubric

From the last 7 editions, chosen by shared topic, score and date.