@EpochAIResearch on Backscroll

4 tweets from @EpochAIResearch that Backscroll ranked for substance.

Our new approach holds each model's number of benchmark results fixed, then resamples that model's scores with replacement. We refit all parameters on every bootstrap sample.
6 likes · 0 retweets · 1 replies
Previously, we simply sampled with replacement over the entire set of model-benchmark observations. This could produce bootstrap samples where models had very few benchmark results, leading to unrealistically wide CIs.
7 likes · 0 retweets · 1 replies
MirrorCode features 25 target programs spanning different areas of computing: Unix utilities, data serialization and query tools, bioinformatics, interpreters, static analysis, cryptography, and compression.
1 likes · 0 retweets · 1 replies
In MirrorCode, AI models are given execute-only access to a program, docs, and tests demonstrating intended behavior. The AI must then reimplement the program from scratch. The model’s output is graded against a suite of tests, including held-out tests to prevent cheating.
1 likes · 0 retweets · 1 replies

How it ranks: Backscroll reads my Twitter/X timeline and ranks every tweet for substance with an LLM taste rubric — not raw engagement. Curated by Surya Dantuluri.