wassname/multifit
1
0
Fork 0
mirror of https://github.com/wassname/multifit.git synced 2026-09-10 12:12:50 +08:00
Code Issues Packages Projects Releases Wiki Activity

Add result logs

Browse Source
This commit is contained in:
Piotr Czapla
2019-02-12 15:00:29 +01:00
parent a630242f97
commit a1e7a79b57
5 changed files with 256 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
results/logs/de.md
+30
View File
@@ -483,3 +483,33 @@ epoch train_loss valid_loss accuracy
Saving models at /home/pczapla/workspace/ulmfit-multilingual/data/mldoc/de-1/models/sp30k/lstm_nl4-noise0.4.m
Loss and accuracy using (cls_last): [0.62477165, tensor(0.7717)]
```
#### 15%
```
python -m ulmfit cls --dataset-path data/mldoc/de-1 --base-lm-path data/mldoc/de-1/models/sp30k/lstm_nl4.m --lang=de --name 'nl4-noise0.15' --cuda-id=1 - train 0 --bs 40 --noise=0.15 --num-cls-epochs=2 --drop-mult-cls=0.2
Max vocab: 30000
Cache dir: /home/pczapla/workspace/ulmfit-multilingual/data/mldoc/de-1/models/sp30k
Model dir: /home/pczapla/workspace/ulmfit-multilingual/data/mldoc/de-1/models/sp30k/lstm_nl4-noise0.15.m
Loading validation /home/pczapla/workspace/ulmfit-multilingual/data/mldoc/de-1/de.dev.csv
Tokenized data loaded, lm.trn 13500, lm.val 1500
Added noise to 150 examples, only 0.85 have correct labels
Running tokenization...
Saving tokenized: cls.trn 1000, cls.val 1000
Size of vocabulary: 30000
First 20 words in vocab: ['xxunk', 'xxpad', 'xxbos', 'xxfld', 'xxmaj', 'xxup', 'xxrep', 'xxwrep', '<unk>', '▁', '▁.', '▁,', '▁der', '▁die', '▁und', '▁in', 'en', "▁&'", 's', '-']
Training args: {'tie_weights': True, 'clip': 0.12, 'bptt': 70, 'pretrained_fnames': [PosixPath('/home/pczapla/workspace/ulmfit-multilingual/data/mldoc/de-1/models/sp30k/lstm_nl4.m/lm_best'), PosixPath('/home/pczapla/workspace/ulmfit-multilingual/data/mldoc/de-1/models/sp30k/lstm_nl4.m/../itos')], 'pretrained_model': None, 'drop_mult': 0.3} dps: [0.25 0.1 0.2 0.02 0.15]
Unknown tokens 0, first 100: []
/home/pczapla/workspace/ulmfit-multilingual/data/mldoc/de-1/models/sp30k
Saving info /home/pczapla/workspace/ulmfit-multilingual/data/mldoc/de-1/models/sp30k/lstm_nl4-noise0.15.m/info.json
Starting classifier training
epoch train_loss valid_loss accuracy
1 0.836104 0.584330 0.897000
epoch train_loss valid_loss accuracy
1 0.692108 0.303470 0.930000
epoch train_loss valid_loss accuracy
1 0.653277 0.330520 0.924000
epoch train_loss valid_loss accuracy
1 0.541086 0.331944 0.922000
2 0.523274 0.335986 0.922000
Saving models at /home/pczapla/workspace/ulmfit-multilingual/data/mldoc/de-1/models/sp30k/lstm_nl4-noise0.15.m
Loss and accuracy using (cls_last): [0.28749043, tensor(0.9355)]
```
results/logs/es.md
+81
View File
@@ -0,0 +1,81 @@
````
python -m ulmfit lm --dataset-path data/wiki/es-100 --cuda-id=0 --tokenizer='sp' --nl 4 --name 'nl4' --max-vocab 30000 --lang es --qrnn=False - train 10 --bs=50 --drop_mult=0
Running tokenization
Wiki text was split to 96224 articles
Wiki text was split to 105 articles
Size of vocabulary: 30000
First 20 words in vocab: ['xxunk', 'xxpad', 'xxbos', 'xxfld', 'xxmaj', 'xxup', 'xxrep', 'xxwrep', '<unk>', '▁', '▁de', '▁,', '▁la', '▁.', '▁en', '▁el', '▁y', 's', '▁a', '▁que']
Training args: {'tie_weights': True, 'clip': 0.12, 'bptt': 70, 'pretrained_fnames': None, 'pretrained_model': None, 'drop_mult': 0} dps: [0.25 0.1 0.2 0.02 0.15]
Training lm from random weights
epoch train_loss valid_loss accuracy
1 3.269541 3.451855 0.387471
2 3.161740 3.423016 0.386158
3 3.187431 3.419638 0.388626
4 3.115763 3.357066 0.393877
5 2.996527 3.291787 0.402488
6 3.021759 3.202183 0.410873
7 2.998267 3.104373 0.422624
8 2.827225 3.006537 0.436010
9 2.784576 2.937735 0.446654
10 2.789913 2.918509 0.450055
data/wiki/es-100/models/sp30k
Saving info data/wiki/es-100/models/sp30k/lstm_nl4.m/info.json
````
### MLDoc
```
python -m ulmfit cls --dataset-path data/mldoc/es-1 --base-lm-path data/wiki/es-100/models/sp30k/lstm_nl4.m --lang=es --name 'nl4' --cuda-id=1 - train 20 --bs 40 --num-cls-epochs=2
Max vocab: 30000
Cache dir: /home/pczapla/workspace/ulmfit-multilingual/data/mldoc/es-1/models/sp30k
Model dir: /home/pczapla/workspace/ulmfit-multilingual/data/mldoc/es-1/models/sp30k/lstm_nl4.m
Loading validation /home/pczapla/workspace/ulmfit-multilingual/data/mldoc/es-1/es.dev.csv
Running tokenization...
Saving tokenized: cls.trn 13013, cls.val 1445
Running tokenization...
Saving tokenized: cls.trn 1000, cls.val 1000
Size of vocabulary: 30000
First 20 words in vocab: ['xxunk', 'xxpad', 'xxbos', 'xxfld', 'xxmaj', 'xxup', 'xxrep', 'xxwrep', '<unk>', '▁', '▁de', '▁,', '▁la', '▁.', '▁en', '▁el', '▁y', 's', '▁a', '▁que']
Training args: {'tie_weights': True, 'clip': 0.12, 'bptt': 70, 'pretrained_fnames': [PosixPath('/home/pczapla/workspace/ulmfit-multilingual/data/wiki/es-100/models/sp30k/lstm_nl4.m/lm_best'), PosixPath('/home/pczapla/workspace/ulmfit-multilingual/data/wiki/es-100/models/sp30k/lstm_nl4.m/../itos')], 'pretrained_model': None, 'drop_mult': 0.3} dps: [0.25 0.1 0.2 0.02 0.15]
Unknown tokens 0, first 100: []
Training lm from: [PosixPath('/home/pczapla/workspace/ulmfit-multilingual/data/wiki/es-100/models/sp30k/lstm_nl4.m/lm_best'), PosixPath('/home/pczapla/workspace/ulmfit-multilingual/data/wiki/es-100/models/sp30k/lstm_nl4.m/../itos')]
epoch train_loss valid_loss accuracy
1 2.805415 2.188974 0.537779
epoch train_loss valid_loss accuracy
1 2.429727 1.989691 0.569048
2 2.218828 1.794969 0.603721
3 2.015097 1.644815 0.629609
4 1.877210 1.537773 0.646898
5 1.775648 1.450283 0.660861
6 1.749334 1.377085 0.672146
7 1.601073 1.311101 0.684400
8 1.564420 1.251074 0.694900
9 1.532728 1.197607 0.704779
10 1.391921 1.145408 0.716044
11 1.379958 1.093550 0.726937
12 1.324111 1.048308 0.735890
13 1.344113 1.007926 0.745691
14 1.243085 0.969521 0.754591
15 1.230809 0.937330 0.762675
16 1.162501 0.913408 0.768044
17 1.170092 0.894892 0.773239
18 1.110860 0.884449 0.775603
19 1.115907 0.880448 0.776671
20 1.083033 0.878421 0.776931
/home/pczapla/workspace/ulmfit-multilingual/data/mldoc/es-1/models/sp30k
Saving info /home/pczapla/workspace/ulmfit-multilingual/data/mldoc/es-1/models/sp30k/lstm_nl4.m/info.json
Starting classifier training
epoch train_loss valid_loss accuracy
1 0.621574 0.391042 0.856000
epoch train_loss valid_loss accuracy
1 0.411668 0.215625 0.935000
epoch train_loss valid_loss accuracy
1 0.340519 0.222422 0.935000
epoch train_loss valid_loss accuracy
1 0.281729 0.192193 0.949000
2 0.262074 0.202975 0.945000
Saving models at /home/pczapla/workspace/ulmfit-multilingual/data/mldoc/es-1/models/sp30k/lstm_nl4.m
Loss and accuracy using (cls_best): [0.1749019, tensor(0.9515)]
```
results/logs/ja.md
+61
View File
@@ -87,4 +87,65 @@ epoch train_loss valid_loss accuracy
8 0.278896 0.358145 0.877000
Saving models at /home/pczapla/workspace/ulmfit-multilingual/data/mldoc/ja-1/models/sp30k/lstm_nl4.m
Loss and accuracy using (cls_best): [0.29789856, tensor(0.8920)]
```
### JA on 100 elements
```
python -m ulmfit cls --dataset-path data/mldoc/ja-1 --base-lm-path data/wiki/ja-100/models/sp30k/lstm_nl4.m --lang=ja --name 'nl4-100' --cuda-id=1 - train 20 --bs 40 --num-cls-epochs=8 --limit=100
Max vocab: 30000
Cache dir: /home/pczapla/workspace/ulmfit-multilingual/data/mldoc/ja-1/models/sp30k
Model dir: /home/pczapla/workspace/ulmfit-multilingual/data/mldoc/ja-1/models/sp30k/lstm_nl4-100.m
Loading validation /home/pczapla/workspace/ulmfit-multilingual/data/mldoc/ja-1/ja.dev.csv
Tokenized data loaded, lm.trn 13500, lm.val 1500
Limiting data set to: 100
Running tokenization...
Saving tokenized: cls.trn 100, cls.val 100
Size of vocabulary: 30000
First 20 words in vocab: ['xxunk', 'xxpad', 'xxbos', 'xxfld', 'xxmaj', 'xxup', 'xxrep', 'xxwrep', '<unk>', '▁', '▁、', '▁の', '▁。', '▁に', '▁を', '▁は', '▁年', '▁が', '▁)', '▁(']
Training args: {'tie_weights': True, 'clip': 0.12, 'bptt': 70, 'pretrained_fnames': [PosixPath('/home/pczapla/workspace/ulmfit-multilingual/data/wiki/ja-100/models/sp30k/lstm_nl4.m/lm_best'), PosixPath('/home/pczapla/workspace/ulmfit-multilingual/data/wiki/ja-100/models/sp30k/lstm_nl4.m/../itos')], 'pretrained_model': None, 'drop_mult': 0.3} dps: [0.25 0.1 0.2 0.02 0.15]
Unknown tokens 0, first 100: []
Training lm from: [PosixPath('/home/pczapla/workspace/ulmfit-multilingual/data/wiki/ja-100/models/sp30k/lstm_nl4.m/lm_best'), PosixPath('/home/pczapla/workspace/ulmfit-multilingual/data/wiki/ja-100/models/sp30k/lstm_nl4.m/../itos')]
epoch train_loss valid_loss accuracy
1 2.837937 2.387255 0.518590
epoch train_loss valid_loss accuracy
1 2.466900 2.193583 0.549492
2 2.232762 1.983981 0.586658
3 2.026505 1.810167 0.615649
4 1.918111 1.679784 0.636613
5 1.748909 1.577095 0.653108
6 1.708709 1.491436 0.667657
7 1.640415 1.420449 0.679619
8 1.577434 1.359511 0.690194
9 1.551961 1.302819 0.700306
10 1.475623 1.252393 0.710039
11 1.435565 1.208159 0.718740
12 1.354910 1.161781 0.727927
13 1.351157 1.123244 0.736009
14 1.299070 1.086383 0.743896
15 1.258739 1.055745 0.750383
16 1.210775 1.035209 0.754965
17 1.228421 1.018373 0.758963
18 1.179444 1.007714 0.761158
19 1.197443 1.003041 0.762068
20 1.163223 1.001939 0.762211
/home/pczapla/workspace/ulmfit-multilingual/data/mldoc/ja-1/models/sp30k
Saving info /home/pczapla/workspace/ulmfit-multilingual/data/mldoc/ja-1/models/sp30k/lstm_nl4-100.m/info.json
Starting classifier training
epoch train_loss valid_loss accuracy
1 1.269222 1.360420 0.340000
epoch train_loss valid_loss accuracy
1 0.969350 1.314497 0.400000
epoch train_loss valid_loss accuracy
1 0.832396 1.263416 0.550000
epoch train_loss valid_loss accuracy
1 0.780991 1.225439 0.600000
2 0.765755 1.183010 0.600000
3 0.749420 1.139053 0.600000
4 0.731800 1.093319 0.610000
5 0.711152 1.054695 0.610000
6 0.694611 1.029465 0.580000
7 0.680276 1.004366 0.580000
8 0.668421 0.984848 0.590000
Saving models at /home/pczapla/workspace/ulmfit-multilingual/data/mldoc/ja-1/models/sp30k/lstm_nl4-100.m
Loss and accuracy using (cls_best): [0.81621724, tensor(0.7437)]
```
results/logs/ru.md
+78
View File
@@ -0,0 +1,78 @@
# RU
## SP30k nl4
### LM
```
python -m ulmfit lm --dataset-path data/wiki/ru-100 --cuda-id=0 --tokenizer='sp' --nl 4 --name 'nl4' --max-vocab 30000 --lang ru --qrnn=False - train 10 --bs=50 --drop_mult=0
Size of vocabulary: 30000 [39/805]
First 20 words in vocab: ['xxunk', 'xxpad', 'xxbos', 'xxfld', 'xxmaj', 'xxup', 'xxrep', 'xxwrep', '<unk>', '▁', '▁,', '▁.', '▁в', 'а', '▁и', 'е', 'и', 'й', '▁на', 'х']
Training args: {'tie_weights': True, 'clip': 0.12, 'bptt': 70, 'pretrained_fnames': None, 'pretrained_model': None, 'drop_mult': 0} dps: [0.25 0.1 0.2 0.02 0.15]
Training lm from random weights
epoch train_loss valid_loss accuracy
1 3.200520 3.295865 0.436852
2 3.027569 3.168700 0.445551
3 3.007320 3.132495 0.450450
4 2.940000 3.041745 0.459344
5 2.876227 2.952338 0.469182
6 2.742553 2.860888 0.480943
7 2.684717 2.769994 0.492934
8 2.569419 2.669971 0.507300
9 2.525698 2.604086 0.516840
10 2.495174 2.591011 0.519415
data/wiki/ru-100/models/sp30k
Saving info data/wiki/ru-100/models/sp30k/lstm_nl4.m/info.json
```
### MLDoc
MultiCCA: 85.65% ulmfit: 87.27%
```
python -m ulmfit cls --dataset-path data/mldoc/ru-1 --base-lm-path data/wiki/ru-100/models/sp30k/lstm_nl4.m --lang=ru --name 'nl4-100' --cuda-id=1 - train 20 --bs 40 --num-cls-epochs=2
Max vocab: 30000
Cache dir: /home/pczapla/workspace/ulmfit-multilingual/data/mldoc/ru-1/models/sp30k
Model dir: /home/pczapla/workspace/ulmfit-multilingual/data/mldoc/ru-1/models/sp30k/lstm_nl4.m
Loading validation /home/pczapla/workspace/ulmfit-multilingual/data/mldoc/ru-1/ru.dev.csv
Running tokenization...
Saving tokenized: cls.trn 9195, cls.val 1021
Running tokenization...
Saving tokenized: cls.trn 1000, cls.val 1000
Size of vocabulary: 30000
First 20 words in vocab: ['xxunk', 'xxpad', 'xxbos', 'xxfld', 'xxmaj', 'xxup', 'xxrep', 'xxwrep', '<unk>', '▁', '▁,', '▁.', '▁в', 'а', '▁и', 'е', 'и', 'й', '▁на', 'х']
Training args: {'tie_weights': True, 'clip': 0.12, 'bptt': 70, 'pretrained_fnames': [PosixPath('/home/pczapla/workspace/ulmfit-multilingual/data/wiki/ru-100/models/sp30k/lstm_nl4.m/lm_best'), PosixPath('/home/pczapla/workspace/ulmfit-multilingual/data/wiki/ru-100/models/sp30k/lstm_nl4.m/../itos')], 'pretrained_model': None, 'drop_mult': 0.3} dps: [0.25 0.1 0.2 0.02 0.15]
Unknown tokens 0, first 100: []
Training lm from: [PosixPath('/home/pczapla/workspace/ulmfit-multilingual/data/wiki/ru-100/models/sp30k/lstm_nl4.m/lm_best'), PosixPath('/home/pczapla/workspace/ulmfit-multilingual/data/wiki/ru-100/models/sp30k/lstm_nl4.m/../itos')]
epoch train_loss valid_loss accuracy
1 2.764138 2.289755 0.552181
epoch train_loss valid_loss accuracy
1 2.414295 2.161708 0.572407
2 2.310551 2.013092 0.596075
3 2.124479 1.864450 0.620103
4 1.970015 1.723395 0.642392
5 1.883664 1.623308 0.658949
6 1.793856 1.513542 0.677954
7 1.625767 1.424582 0.693092
8 1.677054 1.335406 0.709802
9 1.578936 1.264322 0.723626
10 1.523383 1.194463 0.737942
11 1.436643 1.129712 0.750586
12 1.351507 1.072792 0.762524
13 1.357552 1.020739 0.773266
14 1.310516 0.975852 0.783653
15 1.216484 0.940323 0.791262
16 1.187942 0.909915 0.797675
17 1.141316 0.885367 0.803305
18 1.114629 0.871992 0.805929
19 1.075366 0.867010 0.807009
20 1.166387 0.865594 0.807241
/home/pczapla/workspace/ulmfit-multilingual/data/mldoc/ru-1/models/sp30k
Saving info /home/pczapla/workspace/ulmfit-multilingual/data/mldoc/ru-1/models/sp30k/lstm_nl4.m/info.json
Starting classifier training
epoch train_loss valid_loss accuracy
1 0.831180 0.610087 0.787000
epoch train_loss valid_loss accuracy
1 0.678307 0.435860 0.856000
epoch train_loss valid_loss accuracy
1 0.547668 0.399889 0.870000
epoch train_loss valid_loss accuracy
1 0.445839 0.396535 0.869000
2 0.417901 0.369961 0.882000
Saving models at /home/pczapla/workspace/ulmfit-multilingual/data/mldoc/ru-1/models/sp30k/lstm_nl4.m
Loss and accuracy using (cls_best): [0.38499942, tensor(0.8727)]
```
results/logs/zh.md
+6
View File
@@ -0,0 +1,6 @@
```
python -m ulmfit lm --dataset-path data/wiki/zh-100 --cuda-id=0 --tokenizer='sp' --nl 4 --name 'nl4' --max-vocab 60000 --lang zh --qrnn=False - train 10 --bs=50 --drop_mult=0
```