mirror of
https://github.com/wassname/ml_debug.git
synced 2026-07-22 12:50:56 +08:00
1 line
1.8 KiB
JSON
1 line
1.8 KiB
JSON
{"root_cause":"The critic bootstraps across a true terminal transition, likely because the return/TD target uses the reset observation's value or applies the terminal mask at the wrong timestep. This leaks post-reset reward into the preceding episode.","confidence":0.96,"evidence":["Observation: The one-step reward probe passes.","Observation: The two-step probe fails specifically when its final transition is a true terminal state.","Observation: Learned values include reward available only after reset.","Observation: Return computation is correct for continuing episodes.","Inference: The failure is localized to terminal-boundary bootstrapping rather than reward indexing or the general return recurrence."],"competing_hypotheses":["The environment's terminated flag is discarded or confused with truncated before target construction.","An off-by-one error pairs the terminal mask with the next transition rather than the transition producing the terminal state.","The rollout buffer replaces the terminal next observation with the reset observation before computing value targets."],"first_action":"Run a deterministic handcrafted two-step trajectory with rewards [0, 1], terminated flags [false, true], and an intentionally large reset-state value such as 100. Assert that the targets are [gamma, 1] and remain unchanged when the reset-state value is varied.","prediction_if_true":"The terminal-step target currently changes with the reset-state value; applying the producing transition's true-terminal mask makes it exactly 1 and removes the reset reward from earlier values.","prediction_if_false":"The targets remain invariant to the reset-state value, so the next check should trace reward, value, and mask indices through rollout storage and the backward return recurrence for an off-by-one alignment error.","immediate_changes":[],"fallback_logic":[]} |