U4 step3: dim_batch 16->8 to survive OOM contention with user's VS Code kernel

Run 551 was OOM-killed at n_done=36: the user's VS Code Jupyter kernel
(jsteer venv, PID 3214401) co-loaded ~1.5GB VRAM + 1.9GB RAM while the fit
sat at the 22.4/24.6GB ceiling. Clean SIGKILL with no CUDA traceback = host
OOM killer, not a CUDA OOM. dim_batch=8 halves the fit's peak footprint;
it changes only the backward schedule, not the accumulated Jacobian, so U4
exactness is preserved. Resumes from checkpoint (n_done=36), lossless.

Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
This commit is contained in:
wassname
2026-07-10 13:56:17 +08:00
co-authored by Claudypoo
parent beec9189da
commit ebb912452d
6 changed files with 8 additions and 1 deletions
+8 -1
View File
@@ -31,8 +31,15 @@ model = AutoModelForCausalLM.from_pretrained(
meta["model"], torch_dtype=torch.bfloat16).to("cuda").eval()
t0 = time.time()
# dim_batch 16 -> 8 (Claude): run 551 was OOM-killed at n_done=36 when the
# user's VS Code Jupyter kernel (jsteer venv) co-loaded ~1.5GB VRAM + 1.9GB RAM
# while this fit sat at the 22.4/24.6GB ceiling -- clean SIGKILL, no CUDA
# traceback = host OOM killer, not a CUDA OOM. dim_batch=8 halves this fit's
# peak footprint to coexist with the kernel. It only changes the backward
# SCHEDULE (2x passes), NOT the accumulated Jacobian, so U4 exactness holds.
# Resumes from the existing checkpoint (n_done=36), lossless.
jac = Jacobian.fit(model, tok, meta["prompts"], layers=meta["layers"],
dim_batch=16, max_seq_len=384,
dim_batch=8, max_seq_len=384,
checkpoint_path=str(ART / "qwen3-4b-authority.ckpt"))
logger.info(f"fit wall-time: {(time.time() - t0) / 3600:.2f} h")
jac.save(str(ART / "qwen3-4b-authority.jac"))