Drew's Comic NewsroomAbout

Comics on the day’s news

Claude spent the weekend fixing other models’ bad habitsCoyote vs. Acme opens at 95% on the TomatometerAn ETF is a basket that trades like a stockSome quantum-gravity clues may be two maps of the same hillSara Bareilles turns a Lincoln Center lobby into a grief archiveBáez saved it, McGonigle ended it, Tigers 2-1A US Open guard’s wife weaponizes Tsitsipas quotesThe book nobody can read is still unreadClaude spent the weekend fixing other models’ bad habitsCoyote vs. Acme opens at 95% on the TomatometerAn ETF is a basket that trades like a stockSome quantum-gravity clues may be two maps of the same hillSara Bareilles turns a Lincoln Center lobby into a grief archiveBáez saved it, McGonigle ended it, Tigers 2-1A US Open guard’s wife weaponizes Tsitsipas quotesThe book nobody can read is still unread
Four-panel comic: a student with an answer key and a 100-star score marked mush, two locked boxes labeled benchmark and model, two people who cannot see each other’s secrets, a small Gemini Flash Lite robot taking a hidden quiz.
Four panels: a 100-star score marked mush, locked benchmark and model boxes, two people who cannot see each other’s secrets, a small Gemini Flash Lite robot taking a hidden quiz. · Comic: Topics / Drew’s Comic Newsroom. Source: Google DeepMind, Aug. 27, 2026.

Technology and AI

If the model already saw the test, the score is mush

What happened

Google DeepMind’s new pitch is an old exam rule. If a student already saw the questions, a perfect score does not mean much. The lab says AI benchmarks have the same contamination problem: once a model has “peeked,” the number inflates. On Aug. 27 it said it is running what it calls the first double-blind evaluation of a proprietary frontier-class model. A Gemini Flash Lite model is being tested against confidential benchmarks inside Google Cloud Confidential Space, part of the company’s confidential computing tools.

Partners named on the post: Singapore AI Safety Institute, OpenMined, AVERI, and MLCommons. DeepMind says it already uses a wide mix of tests through development, and does not want to rely on internal testing alone. The new piece is cryptographic. The evaluator cannot see Gemini’s weights. Google cannot see the test prompts. Both stay in a box that can be verified without opening it. Authors on the post are William Isaac, Sol Messing, and Kristian Lum.

Why it matters

The old tradeoff was ugly. Either the tester handed over the prompts, risking the company seeing the exam, or the company handed over the weights, risking the model. Zero-logging and contracts have been the soft version of secrecy. DeepMind is arguing that a hardware-backed confidential environment is a harder version, and that scores from that box are easier to trust. As models get more capable, a contaminated leaderboard is not a quirky footnote. It is a fake report card, which is why the lab uses the student-and-answer-key picture in the first place.

The post is a pilot, not a new public scoreboard. It is a method claim: you can let an outside group stress-test a closed model without either side seeing the other’s crown jewels. If other labs copy the box, the useful output is not a viral number. It is a number that was not rehearsed. The strip’s last panel is a hidden quiz for a reason.

Conclusion

If the model already saw the test, the score is mush, and a 100 with an asterisk is not a measurement. DeepMind is trying to run an exam where the tester cannot see the weights and the company cannot see the questions. That is the whole story of the Aug. 27 post, and it is why the comic is a locked box instead of a trophy. The pilot either becomes a pattern other labs can copy or it stays a one-off method note. Either way, the recap should not pretend a new leaderboard dropped.

Source: Google DeepMind