GPDVal — Pairwise Reward Agent Demo

Automated pairwise evaluation of AI models on real-world spreadsheet tasks from the OpenAI GDPval benchmark.

Overview

GPDVal is a two-stage evaluation framework that compares how well two AI models perform on real-world Excel-based tasks:

1. Rollout — Each model runs an agentic loop to produce a deliverable .xlsx file from reference materials
2. Judge — A third LLM compares both submissions against ground truth and a rubric, producing a verdict (A wins / B wins / tie)

The benchmark contains 220 tasks across 44 occupations (compensation analyst, merchandise planner, etc.), each requiring a professional-grade spreadsheet deliverable.

Architecture

Stage 1: ROLLOUT                          Stage 2: JUDGE
+-----------------------+                 +-----------------------+
| Task + Reference .xlsx                 | Submission A (.xlsx)  |
|        |                               | Submission B (.xlsx)  |
|        v                               | Ground Truth (.xlsx)  |
| +---------------+                      | Task Rubric           |
| | Model A       |----> output_a.xlsx   |        |              |
| | (agentic)     |                      |        v              |
| +---------------+                      | +---------------+    |
| +---------------+                      | | Judge Model   |---> winner
| | Model B       |----> output_b.xlsx   | | (agentic)     |    | justification
| | (agentic)     |                      | +---------------+    |
| +---------------+                      +-----------------------+

Both stages are agentic: the LLM calls run_python as a tool, inspects files, iterates, and signals completion with Final Answer:.

Prerequisites

• Python 3.11+
• uv (recommended) or pip

API Keys

Set in .env or environment variables, depending on which models you use:

Model Family	Key
DeepSeek	DEEPSEEK_API_KEY
OpenAI (GPT)	OPENAI_API_KEY
Anthropic (Claude)	ANTHROPIC_API_KEY

Data

Place the GDPval dataset in data/:

data/
├── data/train-00000-of-00001.parquet   # Task definitions
├── reference_files/                     # Input spreadsheets per task
└── deliverable_files/                   # Ground truth spreadsheets per task

Installation

uv sync

Or with pip:

pip install -e .

Usage

# Compare DeepSeek Chat vs DeepSeek Reasoner (default)
uv run evaluate.py --exp-name my_experiment

# Compare specific models
uv run evaluate.py --exp-name gpt_vs_claude \
    --model-a gpt-4o \
    --model-b claude-sonnet-4-6 \
    --judge-model gpt-4o

# Run a subset of tasks with more parallelism
uv run evaluate.py --exp-name quick_test --num-tasks 5 --max-workers 8

Or via the registered CLI entry point:

gpdval-eval --exp-name my_experiment

CLI Arguments

Argument	Default	Description
--exp-name	(required)	Experiment name for results directory
--model-a	deepseek-chat	Model for rollout A
--model-b	deepseek-reasoner	Model for rollout B
--judge-model	deepseek-v4-pro	Model for judging
--num-tasks	all	Number of tasks to evaluate
--seed	42	Random seed for task sampling
--max-turns	100	Max agent turns per stage
--sandbox-timeout	60	Seconds per sandbox execution
--max-workers	10	Max concurrent workers
--skip-rollout	false	Load rollouts from cache
--skip-judge	false	Load judgments from cache
--workspace	workspace	Temp directory for sandboxes
--data-dir	data	Data directory
--output	results.json	Output JSON file

Project Structure

gpdval/
├── evaluate.py                 # Entry point / orchestrator
├── pyproject.toml              # Project config
├── src/gpdval_demo/
│   ├── config.py               # LLM clients + agentic loop
│   ├── data.py                 # Task loading from parquet
│   ├── judge.py                # Pairwise judge agent
│   ├── prompts.py              # System prompts for rollout & judge
│   ├── rollout.py              # Rollout agent
│   └── sandbox.py              # Subprocess sandbox for code execution
├── data/                       # GDPval dataset (see data/README.md)
└── workspace/                  # Temp sandbox directories + caches

Output

Results are saved to a timestamped directory under results/<exp_name>/:

• rollout_results.json — Cached rollout outputs
• judge_results.json — Pairwise judgments
• Per-task directories with reference files, outputs, and trajectories

Reference

• GDPval Paper
• GDPval Blog Post
• OpenAI Evals