Anandkumar's neural operators forecast weather on consumer GPUs, not supercomputers
Caltech's Anima Anandkumar argues physics has no foundation model because the data does not exist — industrial-scale grids imply context lengths in the hundreds of billions, so no transformer can reach them. Her neural operators encode physical structure instead, using spherical harmonics to match Earth's geometry: FourCastNet rivals physics-based weather simulation on consumer-grade GPUs, and a few thousand fusion samples predict plasma disruptions a million times faster than traditional simulation. The bet is that continuous physical systems — weather, fusion, fluid flow — advance through architectural priors rather than dataset scaling, a route that resists the bitter lesson driving the rest of AI.
Source: latent.space ↗
If each dimension is even a few hundred grid points, which is where industrial scale starts... we're talking hundreds of billions to even a trillion context length.
Why this matters
- → Physics problems solved on consumer GPUs without supercomputers, unlocking accessibility.
- → Architectural priors (not data scaling) advance continuous systems — a different route to foundation models.
- → Weather, fusion, and fluid flow modeling now tractable with limited data, resisting bitter-lesson scaling.