415.tech
AI & tech, from the frontlines of Silicon Valley
IBM open-sources CodeAlchemy, a near-1-trillion-token synthetic code dataset

IBM open-sources CodeAlchemy, a near-1-trillion-token synthetic code dataset

IBM Research released CodeAlchemy — nearly 1 trillion tokens of synthetic code across 15 languages, more than twice the size of Nvidia's Nemotron corpus — along with the pipeline and recipes used to build it. What's genuinely new is 1.3 million code files paired with real execution traces, teaching models runtime behavior instead of syntax; the full dataset ships on Hugging Face under a permissive license for anyone training a code model.

Source: research.ibm.com ↗

Post on XEmail

Downloading billions of lines of code from GitHub teaches a model what code looks like, but it doesn't teach what happens behind the scenes when you execute that code on different inputs.

Ankit Gupta, IBM AI researcher

Why this matters

  • → Teaches models runtime behavior, not just syntax—Claude Sonnet struggles to predict code execution.
  • → Smallest synthetic dataset (100B tokens) outperforms 6x larger real-code corpus on benchmarks.
  • → Open release under permissive license democratizes code-model training beyond tech giants.
Teaching models to think