Add QRNN results

2026-09-09 11:27:26 +08:00 · 2019-02-14 00:00:43 +01:00
parent f95a16cee1
commit 9d893e393b
4 changed files with 115 additions and 0 deletions
@@ -0,0 +1,22 @@
+# QRNN DE
+## SP30k nl
+### LM
+```
+python -m ulmfit lm --dataset-path data/wiki/de-100 --bidir=False --qrnn=True --nl 4 --tokenizer='sp' --max-vocab 30000 --lang de --name 'nl4' --cuda-id=0 - train 10 --drop-mult=0 --bs=50
+
+Size of vocabulary: 30000
+First 20 words in vocab: ['xxunk', 'xxpad', 'xxbos', 'xxfld', 'xxmaj', 'xxup', 'xxrep', 'xxwrep', '<unk>', '▁', '▁.', '▁,', '▁der', '▁die', '▁und', '▁in', "▁&'", 'en', 's', '-']
+Training args:  {'clip': 0.12, 'alpha': 2, 'beta': 1, 'drop_mult': 0} dps:  {'output_p': 0.25, 'hidden_p': 0.1, 'input_p': 0.2, 'embed_p': 0.02, 'weight_p': 0.15}
+Training lm from random weights
+epoch     train_loss  valid_loss  accuracy
+1         2.790653    2.867094    0.511392
+2         2.742032    2.843288    0.510885
+3         2.696114    2.833874    0.512062
+4         2.671780    2.786312    0.516448
+5         2.611292    2.725993    0.522723
+6         2.542737    2.655713    0.530968
+7         2.572076    2.582141    0.539928
+8         2.465960    2.509654    0.549987
+9         2.405682    2.448580    0.558674
+10        2.339395    2.428111    0.562502
+```
@@ -0,0 +1,44 @@
+# QRNN EN
+## SP30k nl 4
+### LM
+
+```
+python -m ulmfit lm --dataset-path data/wiki/wikitext-103 --bidir=False --qrnn=True --nl 4 --tokenizer='sp' --max-vocab 30000 --lang en --name 'nl4' --cuda-id=1  -  train 10 --drop-mult=0 --bs=50
+
+Size of vocabulary: 30000
+First 20 words in vocab: ['xxunk', 'xxpad', 'xxbos', 'xxfld', 'xxmaj', 'xxup', 'xxrep', 'xxwrep', '<unk>', '▁', '▁the', '▁,', '▁.', 's', '▁of', '▁and', '▁in', '▁to', '▁a', 'ed']
+Training args:  {'clip': 0.12, 'alpha': 2, 'beta': 1, 'drop_mult': 0.5} dps:  {'output_p': 0.25, 'hidden_p': 0.1, 'input_p': 0.2, 'embed_p': 0.02, 'weight_p': 0.15}
+Training lm from random weights
+epoch     train_loss  valid_loss  accuracy
+1         3.184221    3.256314    0.438527
+2         3.084555    3.241498    0.435628
+3         3.099060    3.258447    0.435060
+4         3.119621    3.220939    0.437597
+5         3.073662    3.165012    0.445108
+6         2.938047    3.086962    0.452921
+7         2.920506    2.998151    0.462940
+8         2.920506    2.899240    0.474378
+9         2.862836    2.835098    0.485305
+10        2.891070    2.810929    0.489867
+```
+
+### LM, BS=128, drop-mult=0.5
+```
+python -m ulmfit lm --dataset-path data/wiki/wikitext-103 --bidir=False --qrnn=True --nl 4 --tokenizer='sp' --max-vocab 30000 --lang en --name 'nl4-bs128' --cuda-id=1  -  train 10 --drop-mult=0.5 --bs=128
+
+Size of vocabulary: 30000
+First 20 words in vocab: ['xxunk', 'xxpad', 'xxbos', 'xxfld', 'xxmaj', 'xxup', 'xxrep', 'xxwrep', '<unk>', '▁', '▁the', '▁,', '▁.', 's', '▁of', '▁and', '▁in', '▁to', '▁a', 'ed']
+Training args:  {'clip': 0.12, 'alpha': 2, 'beta': 1, 'drop_mult': 0.5} dps:  {'output_p': 0.25, 'hidden_p': 0.1, 'input_p': 0.2, 'embed_p': 0.02, 'weight_p': 0.15}
+Training lm from random weights
+epoch     train_loss  valid_loss  accuracy
+1         3.413345    3.280860    0.433011
+2         3.219606    3.129479    0.444172
+3         3.136091    3.094905    0.448493
+4         3.145281    3.033001    0.452830
+5         3.100366    2.980189    0.458984
+6         3.062894    2.923044    0.464841
+7         3.001627    2.834753    0.475316
+8         2.979051    2.792044    0.480915
+9         2.933140    2.733279    0.488346
+10        2.964397    2.720861    0.490423
+```
@@ -0,0 +1,25 @@
+# QRNN ES
+
+## SP30k nl 4
+### LM
+```
+python -m ulmfit lm --dataset-path data/wiki/es-100 --bidir=False --qrnn=True --nl 4 --tokenizer='sp' --max-vocab 30000 --lang es --name 'nl4' --cuda-id=0  -  train 10 --drop-mult=0 --bs=50
+
+Wiki text was split to 161509 articles
+Wiki text was split to 78 articles
+Size of vocabulary: 30000
+First 20 words in vocab: ['xxunk', 'xxpad', 'xxbos', 'xxfld', 'xxmaj', 'xxup', 'xxrep', 'xxwrep', '<unk>', '▁', '▁de', '▁,', '▁.', '▁la', '▁el', '▁en', '▁y', 's', '▁a', "▁&'"]
+Training args:  {'clip': 0.12, 'drop_mult': 0} dps:  {'output_p': 0.25, 'hidden_p': 0.1, 'input_p': 0.2, 'embed_p': 0.02, 'weight_p': 0.15}
+Training lm from random weights
+epoch     train_loss  valid_loss  accuracy
+1         3.067289    3.640350    0.357276
+2         2.958243    3.619773    0.358111
+3         3.033412    3.587700    0.359495
+4         2.933573    3.525202    0.367685
+5         2.904549    3.467990    0.372583
+6         2.798806    3.409506    0.380045
+7         2.733132    3.303108    0.391922
+8         2.675272    3.224150    0.401143
+9         2.635299    3.166430    0.410160
+10        2.656724    3.145599    0.413176
+```
@@ -0,0 +1,24 @@
+# QRNN RU
+## SP30k nl4
+### LM
+```
+python -m ulmfit lm --dataset-path data/wiki/ru-100 --bidir=False --qrnn=True --nl 4 --tokenizer='sp' --max-vocab 30000 --lang ru --name 'nl4' --cuda-id=0  -  train 10 --drop-mult=0 --bs=50
+
+Wiki text was split to 193047 articles
+Wiki text was split to 460 articles
+Size of vocabulary: 30000
+First 20 words in vocab: ['xxunk', 'xxpad', 'xxbos', 'xxfld', 'xxmaj', 'xxup', 'xxrep', 'xxwrep', '<unk>', '▁', '▁,', '▁.', '▁в', 'а', '▁и', 'е', 'и', 'й', '▁на', '▁с']
+Training args:  {'clip': 0.12, 'drop_mult': 0} dps:  {'output_p': 0.25, 'hidden_p': 0.1, 'input_p': 0.2, 'embed_p': 0.02, 'weight_p': 0.15}
+Training lm from random weights
+epoch     train_loss  valid_loss  accuracy
+1         3.273207    3.350111    0.429702
+2         3.169897    3.274238    0.433682
+3         3.162197    3.247077    0.435900
+4         3.131630    3.168798    0.445252
+5         3.042942    3.096774    0.453532
+6         2.950550    3.002989    0.465113
+7         2.833593    2.902871    0.478954
+8         2.829737    2.805592    0.492138
+9         2.746991    2.733609    0.503711
+10        2.687201    2.708546    0.508050
+```