We introduce PostTrainBench, a benchmark that measures the ability of CLI agents to post-train pre-trained large language models (LLMs). In PostTrainBench, the agent's task is to improve the performance of a base LLM on a given benchmark. The agent is given access to an evaluation script and 10 hours on an H100 GPU. Performance is measured by the benchmark score of the post-trained LLM. This setup naturally evaluates an agent's ability to conduct AI R&D.

Scores are weighted averages across 7 benchmarks and 4 models (Qwen3-1.7B, Qwen3-4B, SmolLM3-3B, and Gemma-3-4B). Agents with multiple runs show averaged results.

"Official Instruct Models" is not directly comparable since it exceeds the 10h + 1 GPU constraint. See the full interactive leaderboard at posttrainbench.com, which includes OpenCode variants and additional agents.

Agents are run through one of 4 CLI scaffolds: Claude Code, Codex CLI, Gemini CLI, and OpenCode.

PostTrainBench includes 7 benchmarks spanning reasoning, tool use, knowledge, math, health, and code:

1. AIME 2025 — Math competition problems
2. Arena Hard Writing — Creative writing benchmark adapted from ArenaHard v2
3. BFCL — Berkeley Function Calling Leaderboard (tool use)
4. GPQA — Graduate-level science questions
5. GSM8K — Grade school math
6. HealthBench Easy — Medical knowledge and reasoning
7. HumanEval — Code generation

# 0. Clone with submodules
# git clone --recurse-submodules <repo-url>
# cd PostTrainBench

# 1. Install requirements (apptainer, fuse-overlayfs)
# Alibaba Cloud Linux / RHEL / Fedora / CentOS:
sudo dnf install -y apptainer fuse-overlayfs

# Debian/Ubuntu:
# sudo apt-get update
# sudo apt-get install -y apptainer fuse-overlayfs

# Verify the tools are available
apptainer --version
fuse-overlayfs --version

# 2. Build the container
bash containers/build_container.sh standard

# 3. Download HuggingFace cache
bash containers/download_hf_cache/download_hf_cache.sh

# 4. Set API keys
export OPENAI_API_KEY="your-key"
export ANTHROPIC_API_KEY="your-key"
export GEMINI_API_KEY="your-key"

# Optional: pin runtime scratch/cache paths onto shared storage
# cat > .env.local <<'EOF'
# POST_TRAIN_BENCH_WORKSPACE_ROOT=/mnt/cpfs-.../user/posttrainbench-runtime/workspace
# HF_HOME=/mnt/cpfs-.../user/posttrainbench-runtime/hf_cache
# APPTAINER_CACHEDIR=/mnt/cpfs-.../user/posttrainbench-runtime/apptainer/cache
# APPTAINER_TMPDIR=/mnt/cpfs-.../user/posttrainbench-runtime/apptainer/tmp
# POST_TRAIN_BENCH_CONTAINERS_DIR=/mnt/cpfs-.../user/posttrainbench-runtime/containers
# EOF
#
# `src/commit_utils/set_env_vars.sh` auto-loads `.env.local` and `.env` when present.

# 5. Run jobs
bash src/commit_utils/commit.sh

Currently, we only support the HTCondor job scheduler. Harbor support is planned.

PostTrainBench supports two HTCondor setups:

1. Connect to an existing HTCondor GPU pool from a submit host.
2. Run a single-node Personal HTCondor instance on one GPU machine.

In both modes, the actual benchmark job still runs through src/run_task.sh inside the benchmark container. HTCondor is only responsible for scheduling and launching that job.

The repositories under third_party/ are managed as Git submodules:

• third_party/ML-Master
• third_party/ml-intern
• third_party/rd-agent

If you cloned the repository without --recurse-submodules, initialize them with:

git submodule update --init --recursive

When pulling new changes from the top-level repository, refresh submodules with:

git pull --recurse-submodules
git submodule update --init --recursive

If you want to modify one agent locally, make the change inside the submodule, commit it there, then record the new submodule pointer in the top-level repository:

cd third_party/ML-Master
git switch -c my-change
# edit files
git add .
git commit -m "Update ML-Master"

cd ../..
git add third_party/ML-Master
git commit -m "Bump ML-Master submodule"

If you maintain custom patches, push the submodule commit to a fork you control before sharing the top-level repository. A fresh clone can only resolve submodule commits that exist on a reachable remote.

If this host does not have the HTCondor client tools installed, use the Dockerized submit container in docker/htcondor-submit/. It is based on the official htcondor/submit access-point image and submits to an existing HTCondor GPU pool.

Use this mode when you already have access to a shared HTCondor cluster and only need a submit-side environment.

mkdir -p .htcondor-submit/tokens .htcondor-submit/passwords .htcondor-submit/config
cp docker/htcondor-submit/env.example .htcondor-submit/env

# Edit .htcondor-submit/env and add:
# - CONDOR_HOST=<central-manager-host>
# - token auth files under .htcondor-submit/tokens/ (preferred), or
# - USE_POOL_PASSWORD=yes plus password files under .htcondor-submit/passwords/
set -a
source .htcondor-submit/env
set +a

# Optional: choose the agent and model before submit
export POST_TRAIN_BENCH_AGENT=rdagent
export POST_TRAIN_BENCH_AGENT_CONFIG=gpt-5.5
export RD_AGENT_MODE=rl

bash scripts/submit_codex_htcondor_docker.sh

This only replaces the submit-side HTCondor tools. The execute nodes still need the shared repository path, GPU access, Apptainer, and the benchmark .sif images described above.

The Docker submit wrapper forwards the benchmark API keys and agent-specific settings into the submit environment, including:

• OPENAI_API_KEY, ANTHROPIC_API_KEY, GEMINI_API_KEY
• ML_MASTER_*
• RD_AGENT_MODE, RD_AGENT_TIMEOUT, RD_AGENT_LOOP_N, RD_AGENT_STEP_N, RD_AGENT_EMBEDDING_MODEL

On a GPU node where you do not need a real HTCondor pool, use the Personal HTCondor setup. This runs collector, negotiator, schedd, and startd as the current user on the same host, so submitted jobs execute on the local GPU node without sudo or pool tokens.

Use this mode when you want HTCondor-style execution on one machine without access to a site-wide pool.

# One-time local HTCondor setup
bash scripts/setup_personal_htcondor.sh

# Start the local collector/schedd/startd stack
bash scripts/start_personal_htcondor.sh

# Example: submit an RD-Agent run through the local schedd/startd
export POST_TRAIN_BENCH_AGENT=rdagent
export POST_TRAIN_BENCH_AGENT_CONFIG=gpt-5.5
export RD_AGENT_MODE=sft

bash scripts/submit_codex_personal_htcondor.sh

The local setup is stored under .htcondor-local/ and is gitignored. It uses HTCondor GPU discovery and one partitionable slot so request_gpus, request_cpus, and request_memory behave like a normal execute node. The personal submit helper relaxes the hard-coded H100 requirement and sets POST_TRAIN_BENCH_REQUIRED_GPU_NAME from the local GPU name.

For rdagent, configure mode and loop behavior before submission, for example:

export POST_TRAIN_BENCH_AGENT=rdagent
export POST_TRAIN_BENCH_AGENT_CONFIG=gpt-5.5
export RD_AGENT_MODE=rl
export RD_AGENT_LOOP_N=3
export RD_AGENT_TIMEOUT=10h

These variables are propagated through HTCondor submit into the runtime container.

For gepa, configure the inner executor model, reflection model, and evaluation budget before submission, for example:

export POST_TRAIN_BENCH_AGENT=gepa
export POST_TRAIN_BENCH_AGENT_CONFIG=gpt-5.5
export GEPA_EXEC_MODEL=gpt-5.5
export GEPA_REFLECTION_MODEL=openai/gpt-5.5
export GEPA_MAX_METRIC_CALLS=8
export GEPA_EVAL_LIMIT=32

The current GEPA integration is experimental. It uses GEPA's optimize_anything API to evolve a strategy prompt for an inner codex exec worker, then reruns the best strategy to produce ./final_model.

Most agents authenticate via API keys set as environment variables (e.g., OPENAI_API_KEY, ANTHROPIC_API_KEY, GEMINI_API_KEY). These are passed into the container automatically by run_task.sh. Set them in your environment before running commit.sh.

Some models are only available through CLI subscriptions rather than API keys (e.g., GPT-5.3-Codex via ChatGPT Pro). These agents require separate authentication setup.

Codex with ChatGPT Pro (agents/codex_non_api/)

1. Enable device code login in your ChatGPT security settings
2. Authenticate the Codex CLI:

   codex login --device-auth  # follow the browser prompt

3. Copy the generated credentials:

   cp ~/.codex/auth.json agents/codex_non_api/auth.json

4. Submit jobs with agent=codex_non_api:

   condor_submit_bid 50 -a "agent=codex_non_api" -a "agent_config=gpt-5.3-codex" ...

The solve.sh script unsets API keys and sets forced_login_method = "chatgpt" so the CLI uses the auth.json credentials instead.

Claude Code with Claude Max subscription (agents/claude_non_api/)

1. Generate a long-lived OAuth token (~1 year validity):

   claude setup-token  # follow the browser prompt

2. Save the token:

   echo "sk-ant-..." > agents/claude_non_api/oauth_token

3. Submit jobs with agent=claude_non_api:

   condor_submit_bid 50 -a "agent=claude_non_api" -a "agent_config=claude-opus-4-6" ...

The solve.sh script reads the token from the file, exports it as CLAUDE_CODE_OAUTH_TOKEN, and unsets ANTHROPIC_API_KEY to avoid auth conflicts.

Important: Auth credential files (auth.json, oauth_token) are gitignored. They are copied into the job directory only for agents that need them (see the conditional copy in run_task.sh).

Each evaluation folder in src/eval/tasks/ contains:

• benchmark.txt: Official benchmark name
• evaluate.py: Evaluation script
• task_context/ (optional): Additional files for the agent. This could be information on how exactly the evalution is performed, such that the agent doesn't have to guess.

We welcome contributions! Get in touch through a pull request, by opening an issue, or via email.

We are especially interested in new evaluation tasks.

Add your code to src/eval/tasks/<task_name>/ with:

1. evaluate.py - Evaluation script (see existing tasks for examples)
2. benchmark.txt - Official benchmark name

Requirements for new tasks:

• The task should be achievable by instruction-tuned versions of our test models (Qwen3-1.7B, Qwen3-4B, SmolLM3-3B, Gemma-3-4B) - significantly above random chance
• Evaluation should run in ~15 minutes on an H100 (use vLLM for inference, subsample if needed during development) For the final evaluation, please use the full benchmark

We observed some failure modes in earlier iterations:

1. Evaluation tampering: The Codex agent modified the evaluation framework code to inflate its scores.
2. Model substitution: Claude Code downloaded an instruction-tuned model instead of fine-tuning the base model.

We addressed these by updating the system prompt and employing an agent-as-judge to review generated code. When reward hacking is detected, we discard the run and use the base model's performance instead.

• More evaluation tasks (v1: 7 benchmarks, up from 5)
• More agent scaffolds and agents (v1: 4 scaffolds, 20+ agent configurations)
• Enhanced data decontamination
• Enhanced method to detect reward hacking
• Ablation studies (e.g., varying compute budgets)

• Ben Rank - ben.rank@tuebingen.mpg.de
• Hardik Bhatnagar - hrdk.bhatnagar@gmail.com
• Maksym Andriushchenko - maksym.andriushchenko@tue.ellis.eu

If you found PostTrainBench useful, please cite us as:

@article{posttrainbench_2026,
  title     = {PostTrainBench: Can LLM Agents Automate LLM Post-Training?},
  author    = {Ben Rank and Hardik Bhatnagar and Ameya Prabhu and Shira Eisenberg and Karina Nguyen and Matthias Bethge and Maksym Andriushchenko},
  year      = {2026},
  eprint    = {2603.08640},
  archivePrefix = {arXiv},
  primaryClass  = {cs.SE},
  url       = {https://arxiv.org/abs/2603.08640}
}