DABstep
Hugging Face and Adyen · 450 multi-step tasks on real financial data
93.78%
Overall accuracy · 94.44% on the hard set
New paperActionRAG: graph memory for LLM agentsReasoning, evaluation
Reasoning, evaluation
and learning
Our research spans multi-step reasoning, long-horizon planning and reinforcement learning, with evaluation grounded in verified outcomes
Results on public benchmarks
KramaBench
78.8%
DA Bench
UC Berkeley and PromptQL
72%
Publications
ActionRAG: Evidence-Refining Graph Diffusion for Long-Term Memory in LLM Agents
A graph memory for long-running LLM agents. ActionRAG indexes cautiously, keeping provenance and reconciling conflicting facts as it writes, then refines evidence at query time with local graph diffusion across two retrieval channels.
- MemoryAgentBench average
- 55.1±0.9 · HippoRAG-v2 scores 49.7
- Median retrieval latency
- 95.5 ms414 ms at p99
- Cost per query
- ≤ $0.00116Across the reported columns
| Column | Comparison | ActionRAG | HippoRAG-v2 |
|---|---|---|---|
| SH-DocQA | 77.0 | 76.0 | |
| MH-DocQA | 63.0 | 66.0 | |
| EventQA | 77.1 | 67.6 | |
| MCC | 82.0 | 61.4 | |
| Recom | 18.2 | 10.2 | |
| Detective QA | 46.7 | 57.7 | |
| FC-SH | 61.3 | 54.0 | |
| FC-MH | 15.5 | 5.0 | |
| 0255075100 |
Scores as reported in the paper, which finds ActionRAG highest among retrieval-based memory agents on six of the eight columns.
Research agenda
Caller classification
Telling a customer apart from an AI assistant or a synthetic voice within the first seconds of an 8 kHz phone call, across the languages we support
Planning over weeks
Planning calls and messages over weeks using data from core banking, policy and pharmacy systems. DABstep measures the multi-step data reasoning this depends on
Deterministic and calibrated reward signals
Scoring calls deterministically where a rule can be checked, and calibrating model judges against human QA reviewers where it can’t
Reinforcement learning on verified events
Post-training open-weight models on verified events such as repaid instalments and collected refills, with each update gated on compliance evaluations
Notes from research
and engineering
All articles
Benchmark an agent on your own rubric
Score calls and the account events that followed them, then explore where post-training could improve resolution rates