TimeSage-EV measures how well agents analyze periodically released time series data, a defining challenge for agents that operate over long horizons and are expected to improve continuously.
Results
Releases
The corpus is refreshed as agencies publish new data. Each snapshot fixes a window of periods so results stay comparable; newer snapshots move the window forward.
First public release. Every easy scenario keeps its last 24 periods up to June 2026.
Get started
Python 3.10+ and uv are required; bubblewrap is optional for sandboxing the agent.
git clone https://github.com/TimeSage-Series/TimeSage-EV && cd TimeSage-EV uv sync && source .venv/bin/activate git clone https://huggingface.co/datasets/TimeSage-Series/TimeSage-EV evbench/scenarios cp .env.example .env && set -a && source .env && set +a cp experiments/example.yaml experiments/mine.yaml # one scenario, one period evsandbox run evbench/scenarios/easy/climate/noaa_cpc_ao_monthly_phase 2024-05 \ --config experiments/mine.yaml # a whole tier, periods played in order with feedback evsandbox sweep --config experiments/mine.yaml --mode episode
Task suite
Public data series from agencies such as NOAA, EIA, USDA, CDC and the BLS, spanning six domains. In each period the agent labels the latest release of an indicator as a phase, band or direction, using a rule stated in the task.
Reference
@article{yao2026timesage,
title={TimeSage-EV: A Live Benchmark for Agentic Time Series Analysis in Evolving Environments},
author={Yao, Qingren and Kong, Yaxuan and Nie, Yuqi and Li, Yichen and Zohren, Stefan and Vettoruzzo, Anna and Wen, Qingsong and Jin, Ming and Vanschoren, Joaquin},
journal={arXiv preprint arXiv:2608.14270},
year={2026}
}