DeepMind runs the first double-blind eval of a frontier model, on Gemini Flash Lite
Google DeepMind tested Gemini Flash Lite against confidential benchmarks inside a secure GPU enclave built on Google Cloud's Confidential Space, with the Singapore AI Safety Institute, OpenMined, AVERI, and MLCommons as partners — the evaluator never sees the weights and Google never sees the test prompts. That collapses the old tradeoff between handing over model weights and handing over benchmark questions, and keeps evaluation sets out of future training data, which matters most for cybersecurity and government-run tests. It is a pilot on a small model, not the frontier flagship, so the cost of running larger evaluations this way is still unproven.
Source: deepmind.google ↗
Either evaluators handed over their testing prompts (risking the model provider seeing the test questions in advance), or the model provider handed over their model weights (risking their intellectual property). Double-blind evaluations eliminate this compromise.
Why this matters
- → Evaluators can now test models without revealing test data to future training runs
- → Solves the old tradeoff between protecting model weights and protecting benchmark questions
- → Cryptographic verification raises the bar for cybersecurity and government AI assessments