IBM Research's ALTK-Evolve beats ACE on AppWorld at 40% of the inference cost
IBM Research's ALTK-Evolve retrieves only the lessons a given model and task need instead of injecting ACE's full playbook at every step, scoring 89.3% task-goal completion on AppWorld's 168 test_normal tasks with DeepSeek-V3.2 against ACE's 80.4%, at roughly 40% of the inference cost. The library and the full technical report are public, so a developer running a ReAct agent can swap whole-playbook injection for per-task retrieval — on gpt-oss-120b that change cut tokens from 777K to 116K per task for a statistically tied score.
Source: huggingface.co ↗
Retrieving a few guidelines per task instead of injecting the whole playbook on every step is where the tokens go.
IBM Research, ALTK-Evolve technical report
Why this matters
- → Agentic memory can cut inference costs 40-87% while matching or beating baseline accuracy.
- → Per-task retrieval outperforms full-context injection on harder problems.
- → Open-source library lets developers optimize agent prompting for their model's actual capacity.
Agentic memory, cheaper