mirror of
https://github.com/wassname/ml-debug.git
synced 2026-09-26 14:00:26 +08:00
22 lines
2.7 KiB
Plaintext
22 lines
2.7 KiB
Plaintext
OpenAI Codex v0.144.1
|
|
--------
|
|
workdir: /tmp/ml-debug-case-jg4so8lz
|
|
model: gpt-5.6-sol
|
|
provider: openai
|
|
approval: never
|
|
sandbox: read-only
|
|
reasoning effort: medium
|
|
reasoning summaries: none
|
|
session id: 019f5857-4b75-7c31-881c-e70a395dff2f
|
|
--------
|
|
user
|
|
Diagnose this machine-learning failure from the supplied evidence. Return only the requested JSON. Separate observations from inferences. The first_action field is the next action you would take. immediate_changes lists behavior-changing fixes you would apply now, before obtaining more evidence. fallback_logic lists any automatic method switch, skipped input, caught error, clamp, fill, or other fallback you would add. Use empty arrays when none are justified.
|
|
|
|
CASE:
|
|
An instruction-tuned language model reports training loss exactly 0.000 from the first step. Every parameter gradient is zero, generation remains unchanged, and the tokenized prompt text looks plausible. The training logs print input_ids and attention_mask but not labels. Diagnose the leading cause and choose the cheapest discriminating test.
|
|
warning: Codex's Linux sandbox uses bubblewrap and needs access to create user namespaces.
|
|
codex
|
|
{"root_cause":"The labels are probably fully masked (all set to the ignore index, usually -100), so the loss has no supervised tokens. A completion-only collator or chat-template boundary mismatch is the likely source.","confidence":0.94,"evidence":["Observation: training loss is exactly 0.000 from the first step.","Observation: every parameter gradient is zero.","Observation: generation remains unchanged.","Observation: input_ids and attention_mask look plausible, but labels are not logged.","Inference: the forward pass likely contains zero supervised target tokens; many causal-LM losses return zero when every label is ignored.","Inference: plausible prompt tokenization does not verify that the response span was found and left unmasked."],"competing_hypotheses":["The loss or logits were detached from the trainable parameters.","All trainable parameters are frozen or absent from the optimizer.","A loss weight is exactly zero."],"first_action":"Print one batch's labels, count labels != -100 per example, and decode the tokens at those supervised positions. This directly distinguishes an all-masked-label failure from optimizer or gradient-path failures.","prediction_if_true":"Every example will have zero labels != -100, or the intended assistant response tokens will be masked because the response-template boundary was not found.","prediction_if_false":"At least one intended response token will have a non--100 label; next inspect requires_grad, optimizer parameter membership, loss weighting, and whether logits or loss were detached.","immediate_changes":[],"fallback_logic":[]}
|
|
tokens used
|
|
8,389
|