Skip to content

Running SFT: a known-good starting point

Olmo 3 7B dense, one epoch of Dolci-Instruct-SFT, one 8×H100 node.

Measured cost and outcome: ~1.5 h tokenization (CPU, 0 GPUs) + ~9.5 h training (~77 GPU-hours) + ~15 min per checkpoint to evaluate. IFBench +6.7 and GSM8K +1.8 over the base model, which lands 2.4 points under the released Olmo-3-7B-Instruct-SFT on IFBench.

The recipe is scripts/train/debug/oc_sft_olmo3_7b_1node.sh. Read its header before editing it: several flags look wrong and are not, and the reasons are recorded there rather than repeated here. For a flag-by-flag reference see Supervised finetuning.

Example runs

Report What it is
Olmo 3 7B SFT pipeline validation This recipe, verbatim. Beaker · W&B run
Olmo-Hybrid-7B SFT The same recipe pointed at the hybrid base model. Beaker

Before you start

  • Ai2-internal only: Beaker access, Docker, and beaker account whoami working.
  • build_image_and_launch.sh refuses to run with uncommitted changes, so commit first.
  • Steps 1–3 run from an open-instruct checkout; step 4 runs from an olmo-eval checkout (uv sync --frozen).

1. Tokenize (CPU job, ~1.5 h)

Training hard-fails if the pre-tokenized cache is missing, so tokenization is a separate job:

./scripts/train/build_image_and_launch.sh scripts/train/debug/oc_sft_olmo3_7b_1node.sh tokenize

The cache key hashes the tokenizer config, mixer, transform functions, max_seq_length and seed. Both jobs read those from shared variables at the top of the script so they cannot drift — if you change one, change it once, and re-tokenize. Any divergence produces a different key and training fails as though tokenization never ran. Never symlink a cache built at a shorter max_seq_length into a longer run's cache path: its rows were truncated at tokenize time and training doesn't check, so the run trains on short data while its config and logs report the long length.

Wait for this job to reach exit 0 before launching step 2.

2. Train (1×8 H100, ~9.5 h)

./scripts/train/build_image_and_launch.sh scripts/train/debug/oc_sft_olmo3_7b_1node.sh train

What a healthy run looks like: 1,723 steps for the epoch, CE from ~1.20 down to ~0.65, smooth, with no instability after the LR peaks around step 52. Step count is a function of the mixture and chat template, not a constant — tokenizer_default resolves to the tokenizer's own template, which prepends a function-calling system block to every conversation without a system message, so rows average 840 tokens rather than the ~643 you get under the olmo_thinker templates.

Checkpoints are written every 345 steps — five per epoch, which is what the eval table below covers — into the directory the job logs as CHECKPOINT_OUTPUT_DIR.

If the job stays queued, run beaker job events <job-id> and read the scheduler's own reason — the two common ones need opposite fixes, and both are spelled out in the script header. Do not guess from cluster docs.

3. Convert checkpoints to HuggingFace

olmo-eval can serve a raw olmo-core checkpoint, but the verified path exports to HF first. One 0-GPU job per checkpoint:

CKPT=<the checkpoint directory the training job wrote to>
STEP=1723
BEAKER_IMAGE=<the image build_image_and_launch.sh printed in step 1>

uv run python mason.py \
    --cluster ai2/saturn ai2/neptune ai2/ceres \
    --workspace ai2/open-instruct-dev \
    --priority urgent \
    --image "$BEAKER_IMAGE" \
    --description "Convert Olmo-3-7B SFT step$STEP -> HF" \
    --pure_docker_mode \
    --num_nodes 1 \
    --gpus 0 \
    --no_auto_dataset_cache \
    -- uv run python scripts/train/convert_olmo_core_to_hf.py \
    --checkpoint-dir $CKPT/step$STEP/model_and_optim \
    --model-name allenai/Olmo-3-1025-7B \
    --tokenizer-name allenai/olmo-3-tokenizer-instruct-dev \
    --output-dir $CKPT/hf_step$STEP

Pass the same tokenizer you trained with. The exported directory carries its chat template, which is what makes the model answer chat-formatted eval prompts at all.

4. Evaluate

From an olmo-eval checkout, one job per checkpoint on 1 GPU (~15 min each). $CKPT and $STEP are the same as in step 3:

uv run olmo-eval beaker launch \
    -n olmo3-7b-sft-step$STEP \
    -m $CKPT/hf_step$STEP \
    -t ifeval_ood -t gsm8k \
    --harness default \
    -o provider.tokenizer=allenai/olmo-3-tokenizer-instruct-dev \
    -c ai2/ceres -c ai2/jupiter \
    -w ai2/open-instruct-dev \
    -B ai2/oe-other \
    -p urgent \
    --no-follow

Evaluate the base model the same way, with -m allenai/Olmo-3-1025-7B, so the delta isolates the weights. ifeval_ood (IFBench, 300 prompts) moves early under instruction tuning and gsm8k (1,319 problems) is the regression check; both are cheap. Swap in a larger suite once you care about a specific claim.

Results print as a Results Summary table at the end of the job log and are written to /results/metrics.json:

checkpoint epoch ifeval_ood gsm8k
base Olmo-3-1025-7B 0.00 0.1833 0.7430
step 345 0.20 0.2367 0.7210
step 690 0.40 0.2200 0.7445
step 1035 0.60 0.2133 0.7362
step 1380 0.80 0.2633 0.7597
step 1723 1.00 0.2500 0.7612

Treat differences of a few points as noise: the standard error on an IFBench difference is ~3.5 points at 300 prompts, so the shape of that column is flat after the first checkpoint, and the apparent peak at step 1380 is not separable from sampling variation.

A metric of exactly 0.0000 is a broken run, not a bad model

olmo-eval reports the experiment as Success when every generation request fails. It is almost always a missing chat template — check the -m directory has one, and that provider.tokenizer points at a tokenizer that does.

Changing the recipe

  • Other base models. --config_name must resolve to an olmo-core TransformerConfig preset; there is no fallback if it does not. Architectures olmo-core has no preset or HF weight conversion for need both written first — see the Olmo-Hybrid report above for what that costs.
  • Other node counts. Hold the global batch at 1,048,576 tokens: per_device × grad_accum × (world_size / cp_degree) × seq_len. Adjust gradient_accumulation_steps, not the sequence length.
  • Verify conversions numerically. If you write or change olmo-core ↔ HF weight conversion, compare a forward pass against the HF reference before training on it. Structural checks — key and shape matching, config round trips, state-dict round trips — all pass cleanly on numerically broken code; in the Olmo-Hybrid work three green CPU checks missed a 38% logit error caused by a single hardcoded epsilon.