Skip to content

About

TavernGym and TavernBench: a faithful, full-content Hearthstone Battlegrounds simulator with an RL interface, and a 960-task decision benchmark. Code and data for "Build, Don't Play".

Topics

Resources

Stars

284 stars

Watchers

6 watching

Forks

Repository files navigation

AutoTavern: fantasy tavern, tactical battlefield and AI decision trees

TavernGym & TavernBench

English · 简体中文

A fast simulator and a decision benchmark for Hearthstone Battlegrounds.

Paper on arXiv License: MIT Python 3.10 or newer Rust 1.98 TavernBench: 960 tasks

Quick Start · TavernGym · TavernBench · Documentation · Citation

Overview

AutoTavern brings the decisions of Battlegrounds into a reproducible environment for AI research: simulate complete games, connect a learning algorithm, or evaluate a model's judgment against a fixed set of tasks.

TavernGym TavernBench
A deterministic simulator for complete eight-player games. A benchmark for reasoning about game decisions and rules.
Game content from patch 37.0.2, with content coverage. 960 tasks in 40 scenarios across eight categories.
Rust engine, Python bindings, single-seat and batched environments. Prompts, labels, saved states, model answers, scoring and tool-use evaluation.

This is the code and data release for Build, Don't Play: Self-Improving LLM Research Agents Develop a Hearthstone Battlegrounds AI That Rivals Top Players, by Changlian Ma and Limin Wang, Nanjing University and Shanghai AI Laboratory.

The release includes TavernGym and TavernBench. The trained TavernAgent, its weights and training code, and the benchmark item generators are not included.

TavernGym now targets 37.0.2; see the patch notes. TavernBench keeps the paper's 36.6.3 tasks, labels, saved states and results unchanged. Use the v1.0.0 release for historical simulator-based reproduction; running the tool-use harness with the current engine applies 37.0.2 rules.

Quick start

git clone https://github.com/MCG-NJU/AutoTavern.git
cd AutoTavern

Run a game

Install with Python 3.10 or newer and a Rust toolchain. The version in rust-toolchain.toml is installed automatically by rustup.

pip install .
import taverngym

game = taverngym.Game(seed=1)
while not game.is_over():
    for seat in range(game.num_players):
        if game.needs_action(seat):
            game.step(seat, game.bot_action(seat))

print(list(game.placements()))

This plays a full eight-player game with the included scripted bots. Replace a bot's action with your own legal action to control a seat.

Score the released answers

From the repository root:

cd tavernbench
python -X utf8 -m tavernbench.report --set data/eval --runs results/eval \
    --systems tavernagent,gpt-6-astra,claude-opus-5.5,jev

This scores released answers against the labels. It needs only Python's standard library: no simulator build, model access or API key is required.

TavernGym

TavernGym reproduces solo Battlegrounds for client 37.0.2.256589, server rules 37.0.2. It exposes the player-visible state, an exact legal action mask, and seeded save and restore.

Interface Use it for
Game Drive any seat, inspect a game, or branch from a saved state.
BattlegroundsEnv Connect one learning seat against seven scripted bots.
BatchEnv Step many games together, with any set of controlled seats, including self-play.

The learning interface provides 245 discrete actions, 3,227 numeric features and 124 card indices per observation. Rules and lobby settings can be configured explicitly.

Environment guide · Architecture · Implemented cards · Timewarp rules

TavernBench

TavernBench tests combat predictions, warband arrangements, target choices and turn plans with a shared prompt format and common scoring code.

Category Examples
Combat Predict outcomes, damage and lethal risk.
Positioning Arrange a warband against shields, cleave and deathrattles.
Ordering Sequence plays, spells and triggers.
Targets Choose a spell, Hero Power or minion target.
Selection Choose a hero, trinket, discovery or play.
Strategy Choose a turn plan: upgrade, buy, refresh or save the Gold.
Economy Reason about Gold, purchases and upgrade costs.
Rule execution Apply game rules to concrete actions.

The 368-task evaluation set is released alongside all 960 tasks. Saved simulator states cover the 30 play scenarios; the ten rule-execution scenarios need no state. The released answers include TavernAgent, ten language models, Jev, and five tool-use runs.

To evaluate your own model, install the optional runners from the repository root:

pip install ".[bench]"

Benchmark guide · Task data · Released answers · Scoring code

Documentation

Guide Contents
TavernGym Game APIs, batched environments, observations, actions, configuration and speed.
TavernBench Dataset layout, labels, scoring, model runners, tool use and reproduction notes.
Architecture Engine structure, interfaces and how the components fit together.
Content coverage Implemented minions, heroes, spells, trinkets and other game objects.
Timewarp The Timewarped Tavern (off by default in 37.0.2; two heroes still visit it).

For development, run pip install -e ".[test]" and pytest. The Rust crate also works on its own with cargo test --release.

Citation

If you use TavernGym or TavernBench in your research, please cite:

@misc{ma2026build,
  title  = {Build, Don't Play: Self-Improving {LLM} Research Agents Develop a Hearthstone
            Battlegrounds {AI} That Rivals Top Players},
  author = {Ma, Changlian and Wang, Limin},
  year   = {2026},
  eprint = {2610.33845},
  archivePrefix = {arXiv},
  primaryClass = {cs.AI},
  url    = {https://arxiv.org/abs/2610.33845v1}
}

License

Code, documentation and the TavernBench tasks, labels, states and results are released under the MIT License. Third-party game content retains its original rights; see NOTICE.

This is an independent, non-commercial research project, unaffiliated with Blizzard Entertainment. Hearthstone and Hearthstone Battlegrounds are Blizzard trademarks. No game client files or official game artwork are distributed, and TavernGym does not connect to Blizzard's services. The AutoTavern banner and emblem are original generated illustrations.

AutoTavern emblem
Back to the tavern

About

TavernGym and TavernBench: a faithful, full-content Hearthstone Battlegrounds simulator with an RL interface, and a 960-task decision benchmark. Code and data for "Build, Don't Play".

Topics

Resources

Stars

284 stars

Watchers

6 watching

Forks

Releases

Contributors

Languages