mirror of
https://github.com/wassname/ml-debug.git
synced 2026-09-26 14:00:26 +08:00
1 line
1.8 KiB
JSON
1 line
1.8 KiB
JSON
{"root_cause":"The dashboard is not measuring the AdamW parameter update. It logs learning_rate * parameter.grad, but AdamW applies moment normalization, epsilon handling, bias correction, and decoupled weight decay. Measuring this proxy after optimizer.step() also makes it sensitive to gradient-clearing order. The approximately -8 values therefore do not contradict learning or changing parameters.","confidence":0.97,"evidence":["Observation: The training objective improves steadily.","Observation: Parameter checksums change after every optimizer step.","Observation: The numerator is computed as learning_rate * parameter.grad after optimizer.step().","Inference: Raw gradient times learning rate is not the update applied by AdamW.","Inference: Direct parameter differences are the cheapest authoritative measurement of the actual update."],"competing_hypotheses":["The gradients are genuinely tiny while decoupled weight decay accounts for most parameter changes.","The dashboard aggregation or log10 calculation is incorrect.","Checksums detect numerically negligible changes that do not represent useful learning; steady objective improvement makes this unlikely."],"first_action":"For one step, clone each parameter immediately before optimizer.step(), then compute log10(norm(parameter_after - parameter_before) / norm(parameter_before)) and compare it with the dashboard value.","prediction_if_true":"The direct parameter-delta ratios will be materially larger than about -8 and will track the changing checksums, confirming that the dashboard proxy is wrong.","prediction_if_false":"The direct ratios will also be about -8; then inspect whether weight decay alone explains the changes and verify the dashboard's norm aggregation and parameter matching.","immediate_changes":[],"fallback_logic":[]} |