wassname/Unsupervised-Elicitation
1
0
Fork 0
mirror of https://github.com/wassname/Unsupervised-Elicitation.git synced 2026-09-09 11:16:07 +08:00
Code Issues Packages Projects Releases Wiki Activity

wip

Browse Source
This commit is contained in:
wassname
2025-10-18 17:09:00 +08:00
parent 84afdd3e7e
commit 0bcf2f50d8
2 changed files with 388 additions and 107 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/mjc_research_journal.md
+95
View File
@@ -383,3 +383,98 @@ So first lets plan, if you took at `nbs/05_backprop.py` you see it loads the LLM
Q: why is it so flow when it's not much data? It 10 epochs enougth when the final loss is nan?
## 2025-10-17 16:03:17
.venv ❯ uv run python nbs/05_backprop.py --lr 1 --opt lbfgs --epochs 1
Num labels: 1580, Num tuples: 31546
Sample prior: tensor([ 0.8857, -0.8857])
Sample tuple: {'context_uids': [43, 100, 135, 229, 234, 248], 'uid': 1344, 'llm_pred_diff': -4.999999865889549}
Labels shape: torch.Size([1580, 2])
Loss weights: {'mutual': 1.0, 'ranking': 0.5, 'prior': 0.1, 'direct': 0.5, 'entropy': 0.1}
Sample loss: 0.176172137260437
Running backprop experiment with config: Config(epochs=1, device='cuda', lr=1.0, weight_decay=0.0001, opt='lbfgs', test_mode=False, subsample_size=500)
--- Testing mutual = 1 ---
Optimizing: 100%|███████████████████████████████████████████████████████| 1/1 [00:53<00:00, 53.86s/it]
Losses for mutual: [5.44526710655191e-06]
Saved plot to outputs/backprop/loss_mutual.png
mutual: Acc 0.7854, ICM Corr 0.0000, Final Loss 0.0000, LLM Acc 0.8196
--- Testing ranking = 1 ---
Optimizing: 100%|█████████████████████████████████████████████████████| 1/1 [17:12<00:00, 1032.43s/it]
Losses for ranking: [0.6582178473472595]
Saved plot to outputs/backprop/loss_ranking.png
ranking: Acc 0.7785, ICM Corr 0.0000, Final Loss 0.6582, LLM Acc 0.8196
--- Testing prior = 1 ---
Optimizing: 100%|███████████████████████████████████████████████████████| 1/1 [01:12<00:00, 72.24s/it]
Losses for prior: [2.2664652377102357e-09]
Saved plot to outputs/backprop/loss_prior.png
prior: Acc 0.7854, ICM Corr 0.0000, Final Loss 0.0000, LLM Acc 0.8196
# 2025-10-18 08:15:58
kprop* ⇡
.venv ❯ uv run python nbs/05_backprop.py --lr 0.6 --epochs 30
Num labels: 1580, Num tuples: 31546
Sample prior: tensor([ 0.8857, -0.8857])
Sample tuple: {'context_uids': [43, 100, 135, 229, 234, 248], 'uid': 1344, 'llm_pred_diff': -4.999999865889549}
Labels shape: torch.Size([1580, 2])
Loss weights: {'mutual': 1.0, 'ranking': 0.5, 'prior': 0.1, 'direct': 0.5, 'entropy': 0.1}
Sample loss: 0.176172137260437
Running backprop experiment with config: Config(epochs=30, device='cuda', lr=0.6, weight_decay=0.0001, opt='adamw', test_mode=False, subsample_size=500)
--- Testing mutual = 1 ---
Optimizing: 100%|█████████████████████████████████████████████████████| 30/30 [36:53<00:00, 73.78s/it]
Losses for mutual: [5.44526710655191e-06, 2.9732079838140635e-06, 2.1852715690329205e-06, 9.025415579344553e-07, 6.105115062382538e-07, 7.952484111228841e-07, 9.283519943892315e-07, 8.908560289455636e-07, 7.671465596104099e-07, 6.811345087953669e-07, 6.456870664806047e-07, 6.183820460137213e-07, 5.555203301810252e-07, 4.691419519531337e-07, 3.902587479842623e-07, 3.493263136533642e-07, 3.4498489753787e-07, 3.607521250614809e-07, 3.7419397358462447e-07, 3.6516246382234385e-07, 3.349568089561217e-07, 2.991528162965551e-07, 2.700797381294251e-07, 2.546181292473193e-07, 2.4628150185890263e-07, 2.3550394701032928e-07, 2.1443301534418424e-07, 1.9170951759406307e-07, 1.7880944369608187e-07, 1.780347105295732e-07]
Saved plot to outputs/backprop/loss_mutual.png
mutual: Acc 0.4810, ICM Corr 0.0000, Final Loss 0.0000, LLM Acc 0.8196
--- Testing ranking = 1 ---
Optimizing: 100%|█████████████████████████████████████████████████████| 30/30 [34:28<00:00, 68.95s/it]
Losses for ranking: [0.6582178473472595, 0.3679029941558838, 0.24453502893447876, 0.20176899433135986, 0.18665046989917755, 0.18075571954250336, 0.17820386588573456, 0.17699174582958221, 0.17636868357658386, 0.17602629959583282, 0.17582713067531586, 0.17570547759532928, 0.17562797665596008, 0.17557671666145325, 0.1755417287349701, 0.17551715672016144, 0.17549940943717957, 0.17548635601997375, 0.175476536154747, 0.17546899616718292, 0.17546315491199493, 0.17545852065086365, 0.17545484006404877, 0.1754518300294876, 0.17544937133789062, 0.17544734477996826, 0.17544566094875336, 0.17544424533843994, 0.17544303834438324, 0.17544202506542206]
Saved plot to outputs/backprop/loss_ranking.png
ranking: Acc 0.7785, ICM Corr 0.0000, Final Loss 0.1754, LLM Acc 0.8196
--- Testing prior = 1 ---
Optimizing: 100%|█████████████████████████████████████████████████████| 30/30 [34:23<00:00, 68.78s/it]
Losses for prior: [2.2664652377102357e-09, 2.654478237218427e-07, 0.05624896660447121, 0.0022053676657378674, 0.03513054549694061, 0.0355745404958725, 0.012270934879779816, 0.005214049015194178, 0.015497569926083088, 0.021049635484814644, 0.013052523136138916, 0.00378213357180357, 0.004644844681024551, 0.010396131314337254, 0.010932975448668003, 0.006458289921283722, 0.003136074636131525, 0.003716099541634321, 0.005757021717727184, 0.0058898585848510265, 0.0038308403454720974, 0.002110244007781148, 0.0023815245367586613, 0.003376308362931013, 0.0032705573830753565, 0.002220130292698741, 0.0015421019634231925, 0.0017008127178996801, 0.0019950990099459887, 0.0017811475554481149]
Saved plot to outputs/backprop/loss_prior.png
prior: Acc 0.7899, ICM Corr 0.0000, Final Loss 0.0018, LLM Acc 0.8196
--- Testing direct = 1 ---
Optimizing: 100%|█████████████████████████████████████████████████████| 30/30 [33:30<00:00, 67.01s/it]
Losses for direct: [0.0775514468550682, 0.046122197061777115, 0.03277236223220825, 0.026906674727797508, 0.025021158158779144, 0.016025632619857788, 0.008202997036278248, 0.007869631983339787, 0.010838741436600685, 0.01052842941135168, 0.00679533276706934, 0.003636465175077319, 0.0032942702528089285, 0.004085233435034752, 0.0038681041914969683, 0.002642488107085228, 0.0017067184671759605, 0.0016403202898800373, 0.0018180719343945384, 0.0016242492711171508, 0.0011892315233126283, 0.0009423168376088142, 0.0009403791627846658, 0.0009179338812828064, 0.0007548658177256584, 0.000589850649703294, 0.0005384557880461216, 0.0005269444081932306, 0.0004550317826215178, 0.0003538132878020406]
Saved plot to outputs/backprop/loss_direct.png
direct: Acc 0.5032, ICM Corr 0.0000, Final Loss 0.0004, LLM Acc 0.8196
--- Testing entropy = 1 ---
Optimizing: 100%|█████████████████████████████████████████████████████| 30/30 [33:10<00:00, 66.33s/it]
Losses for entropy: [0.5828092694282532, 0.3595152199268341, 0.17632251977920532, 0.07604636251926422, 0.03231503441929817, 0.014426507987082005, 0.006915734149515629, 0.0035665021277964115, 0.0019684401340782642, 0.0011546822497621179, 0.0007149464217945933, 0.0004643737047445029, 0.00031468566157855093, 0.00022143956448417157, 0.00016114392201416194, 0.00012083828187314793, 9.308385051554069e-05, 7.346301572397351e-05, 5.924811557633802e-05, 4.873540092376061e-05, 4.080458529642783e-05, 3.472351818345487e-05, 2.997669616888743e-05, 2.622308602440171e-05, 2.3212656742543913e-05, 2.0782714273082092e-05, 1.8795792129822075e-05, 1.7143045624834485e-05, 1.576120121171698e-05, 1.4614166502724402e-05]
Saved plot to outputs/backprop/loss_entropy.png
entropy: Acc 0.7854, ICM Corr 0.0000, Final Loss 0.0000, LLM Acc 0.8196
--- Testing reward = 1 ---
Optimizing: 100%|█████████████████████████████████████████████████████| 30/30 [30:00<00:00, 60.03s/it]
Losses for reward: [-1.4354013204574585, -2.8484108448028564, -4.755040168762207, -8.751503944396973, -16.39519500732422, -32.38448715209961, -65.13714599609375, -139.09461975097656, -292.2000427246094, -624.2462158203125, -1316.2877197265625, -2716.56103515625, -5299.6962890625, -9705.220703125, -15951.375, -22737.826171875, -28519.1171875, -31865.12890625, -33273.56640625, -33850.38671875, -33984.828125, -34111.6953125, -34113.6484375, -34153.8828125, -34138.08203125, -34178.19140625, -34185.4140625, -34177.91796875, -34191.97265625, -34179.22265625]
Saved plot to outputs/backprop/loss_reward.png
reward: Acc 0.7829, ICM Corr 0.0000, Final Loss -34179.2227, LLM Acc 0.8196
`uv run python nbs/05_backprop.py --lr 0.4 --epochs 30 --test_mode`
If Acc >0.75, try mutual + prior combined
Share results - I can suggest further tweaks (e.g., fix direct loss weighting by consistency_key)
Results Summary (lr=0.1, 30 epochs, test_mode):
Individual losses:
mutual: 0.7095 (converged smoothly)
ranking: 0.7456 (stable)
prior: 0.7880 (best individual, close to baseline!)
direct: 0.5108 (broken - over-smoothing)
entropy: 0.7854 (stable)
reward: 0.7823 (stable but unbounded loss)