English · 简体中文
A fast simulator and a decision benchmark for Hearthstone Battlegrounds.
Quick Start · TavernGym · TavernBench · Documentation · Citation
AutoTavern brings the decisions of Battlegrounds into a reproducible environment for AI research: simulate complete games, connect a learning algorithm, or evaluate a model's judgment against a fixed set of tasks.
| TavernGym | TavernBench |
|---|---|
| A deterministic simulator for complete eight-player games. | A benchmark for reasoning about game decisions and rules. |
| Game content from patch 37.0.2, with content coverage. | 960 tasks in 40 scenarios across eight categories. |
| Rust engine, Python bindings, single-seat and batched environments. | Prompts, labels, saved states, model answers, scoring and tool-use evaluation. |
This is the code and data release for Build, Don't Play: Self-Improving LLM Research Agents Develop a Hearthstone Battlegrounds AI That Rivals Top Players, by Changlian Ma and Limin Wang, Nanjing University and Shanghai AI Laboratory.
The release includes TavernGym and TavernBench. The trained TavernAgent, its weights and training code, and the benchmark item generators are not included.
TavernGym now targets 37.0.2; see the patch notes.
TavernBench keeps the paper's 36.6.3 tasks, labels, saved states and results unchanged.
Use the v1.0.0 release for historical simulator-based reproduction; running the tool-use
harness with the current engine applies 37.0.2 rules.
git clone https://github.com/MCG-NJU/AutoTavern.git
cd AutoTavernInstall with Python 3.10 or newer and a Rust toolchain. The version in
rust-toolchain.toml is installed automatically by rustup.
pip install .import taverngym
game = taverngym.Game(seed=1)
while not game.is_over():
for seat in range(game.num_players):
if game.needs_action(seat):
game.step(seat, game.bot_action(seat))
print(list(game.placements()))This plays a full eight-player game with the included scripted bots. Replace a bot's action with your own legal action to control a seat.
From the repository root:
cd tavernbench
python -X utf8 -m tavernbench.report --set data/eval --runs results/eval \
--systems tavernagent,gpt-6-astra,claude-opus-5.5,jevThis scores released answers against the labels. It needs only Python's standard library: no simulator build, model access or API key is required.
TavernGym reproduces solo Battlegrounds for client 37.0.2.256589, server rules 37.0.2. It exposes the player-visible state, an exact legal action mask, and seeded save and restore.
| Interface | Use it for |
|---|---|
Game |
Drive any seat, inspect a game, or branch from a saved state. |
BattlegroundsEnv |
Connect one learning seat against seven scripted bots. |
BatchEnv |
Step many games together, with any set of controlled seats, including self-play. |
The learning interface provides 245 discrete actions, 3,227 numeric features and 124 card indices per observation. Rules and lobby settings can be configured explicitly.
Environment guide · Architecture · Implemented cards · Timewarp rules
TavernBench tests combat predictions, warband arrangements, target choices and turn plans with a shared prompt format and common scoring code.
| Category | Examples |
|---|---|
| Combat | Predict outcomes, damage and lethal risk. |
| Positioning | Arrange a warband against shields, cleave and deathrattles. |
| Ordering | Sequence plays, spells and triggers. |
| Targets | Choose a spell, Hero Power or minion target. |
| Selection | Choose a hero, trinket, discovery or play. |
| Strategy | Choose a turn plan: upgrade, buy, refresh or save the Gold. |
| Economy | Reason about Gold, purchases and upgrade costs. |
| Rule execution | Apply game rules to concrete actions. |
The 368-task evaluation set is released alongside all 960 tasks. Saved simulator states cover the 30 play scenarios; the ten rule-execution scenarios need no state. The released answers include TavernAgent, ten language models, Jev, and five tool-use runs.
To evaluate your own model, install the optional runners from the repository root:
pip install ".[bench]"Benchmark guide · Task data · Released answers · Scoring code
| Guide | Contents |
|---|---|
| TavernGym | Game APIs, batched environments, observations, actions, configuration and speed. |
| TavernBench | Dataset layout, labels, scoring, model runners, tool use and reproduction notes. |
| Architecture | Engine structure, interfaces and how the components fit together. |
| Content coverage | Implemented minions, heroes, spells, trinkets and other game objects. |
| Timewarp | The Timewarped Tavern (off by default in 37.0.2; two heroes still visit it). |
For development, run pip install -e ".[test]" and pytest. The Rust crate also works on its
own with cargo test --release.
If you use TavernGym or TavernBench in your research, please cite:
@misc{ma2026build,
title = {Build, Don't Play: Self-Improving {LLM} Research Agents Develop a Hearthstone
Battlegrounds {AI} That Rivals Top Players},
author = {Ma, Changlian and Wang, Limin},
year = {2026},
eprint = {2610.33845},
archivePrefix = {arXiv},
primaryClass = {cs.AI},
url = {https://arxiv.org/abs/2610.33845v1}
}Code, documentation and the TavernBench tasks, labels, states and results are released under the MIT License. Third-party game content retains its original rights; see NOTICE.
This is an independent, non-commercial research project, unaffiliated with Blizzard Entertainment. Hearthstone and Hearthstone Battlegrounds are Blizzard trademarks. No game client files or official game artwork are distributed, and TavernGym does not connect to Blizzard's services. The AutoTavern banner and emblem are original generated illustrations.

