wassname/multifit
1
0
Fork 0
mirror of https://github.com/wassname/multifit.git synced 2026-09-09 11:27:26 +08:00
Code Issues Packages Projects Releases Wiki Activity
Files
tmp
multifit/results/logs/fr.md
T
Piotr Czapla c29180a08f Add some results
2019-02-20 10:22:44 +01:00

180 lines
9.4 KiB
Markdown
Raw Permalink Blame History

# FR
## SP15k QRNN nl 4
```
python -m ulmfit lm --dataset-path data/wiki/${LANG}-100 --tokenizer='sp' --nl 4 --name 'nl4' --max-vocab 15000 --lang ${LANG} --qrnn=True - train 10 --bs=50 --drop_mult=0
Max vocab: 15000
Cache dir: data/wiki/fr-100/models/sp15k
Model dir: data/wiki/fr-100/models/sp15k/qrnn_nl4.m
Wiki text was split to 174227 articles
Wiki text was split to 491 articles
Running tokenization lm...
Data lm, trn: 174227, val: 491
Size of vocabulary: 15000
First 20 words in vocab: ['xxunk', 'xxpad', 'xxbos', 'xxfld', 'xxmaj', 'xxup', 'xxrep', 'xxwrep', '<unk>', '▁', '▁de', '▁,', 's', '▁.', "'", '▁la', '▁le',
'▁et', '▁l', '▁à']
Training args: {'clip': 0.12, 'alpha': 2, 'beta': 1, 'drop_mult': 0} dps: {'output_p': 0.25, 'hidden_p': 0.1, 'input_p': 0.2, 'embed_p': 0.02, 'weight_p': 0.15}
Training lm from random weights
epoch train_loss valid_loss accuracy
1 2.881558 2.790402 0.465847
2 2.824942 2.732005 0.471660
3 2.758845 2.672040 0.478273
4 2.715069 2.602380 0.489159
5 2.677029 2.553575 0.494752
6 2.602514 2.476142 0.507337
7 2.564386 2.388670 0.518902
8 2.470835 2.304033 0.532000
9 2.366890 2.243269 0.542781
10 2.390439 2.223538 0.546622
Total time: 9:09:26
data/wiki/fr-100/models/sp15k
Saving info data/wiki/fr-100/models/sp15k/qrnn_nl4.m/info.json
```
## SP30k LSTM nl 4
### LM
```
python -m ulmfit lm --dataset-path data/wiki/fr-100 --cuda-id=1 --tokenizer='sp' --nl 4 --name 'nl4' --max-vocab 30000 \ ✘ 130
--lang fr --qrnn=False - train 10 --bs=50 --drop_mult=0
Max vocab: 30000
Cache dir: data/wiki/fr-100/models/sp30k
Model dir: data/wiki/fr-100/models/sp30k/lstm_nl4.m
Running tokenization
Wiki text was split to 113288 articles
Wiki text was split to 88 articles
Size of vocabulary: 30000
First 20 words in vocab: ['xxunk', 'xxpad', 'xxbos', 'xxfld', 'xxmaj', 'xxup', 'xxrep', 'xxwrep', '<unk>', '▁', '▁de', '▁,', '▁.', "'", 's', '▁la', '▁le', '▁et', '▁l', '▁à']
Training args: {'tie_weights': True, 'clip': 0.12, 'bptt': 70, 'pretrained_fnames': None, 'pretrained_model': None, 'drop_mult': 0} dps: [0.25 0.1 0.2 0.02 0.15]
Training lm from random weights
epoch train_loss valid_loss accuracy
1 3.120035 3.449028 0.383274
2 3.070353 3.431372 0.382520
3 3.092097 3.406521 0.384604
4 3.035016 3.356502 0.391155
5 2.936505 3.297572 0.396365
6 2.926953 3.192980 0.407546
7 2.841542 3.115280 0.417741
8 2.805254 3.008793 0.429512
9 2.681713 2.944207 0.439959
10 2.644920 2.923765 0.442415
data/wiki/fr-100/models/sp30k
Saving info data/wiki/fr-100/models/sp30k/lstm_nl4.m/info.json
```
### MLDocs
#### First run
MultiCCA 92.05, ulmfit 93.90
```
python -m ulmfit cls --dataset-path data/mldoc/fr-1 --base-lm-path data/wiki/fr-100/models/sp30k/lstm_nl4.m --lang=fr --name 'nl4' --cuda-id=1 - train 20 --bs 40
Max vocab: 30000
Cache dir: /home/pczapla/workspace/ulmfit-multilingual/data/mldoc/fr-1/models/sp30k
Model dir: /home/pczapla/workspace/ulmfit-multilingual/data/mldoc/fr-1/models/sp30k/lstm_nl4.m
Loading validation /home/pczapla/workspace/ulmfit-multilingual/data/mldoc/fr-1/fr.dev.csv
Running tokenization...
Saving tokenized: cls.trn 13500, cls.val 1500
Running tokenization...
Saving tokenized: cls.trn 1000, cls.val 1000
Size of vocabulary: 30000
First 20 words in vocab: ['xxunk', 'xxpad', 'xxbos', 'xxfld', 'xxmaj', 'xxup', 'xxrep', 'xxwrep', '<unk>', '▁', '▁de', '▁,', '▁.', "'", 's', '▁la', '▁le', '▁et', '▁l', '▁à']
Training args: {'tie_weights': True, 'clip': 0.12, 'bptt': 70, 'pretrained_fnames': [PosixPath('/home/pczapla/workspace/ulmfit-multilingual/data/wiki/fr-100/models/sp30k/lstm_nl4.m/lm_best'), PosixPath('/home/pczapla/workspace/ulmfit-multilingual/data/wiki/fr-100/models/sp30k/lstm_nl4.m/../itos')], 'pretrained_model': None, 'drop_mult': 0.3} dps: [0.25 0.1 0.2 0.02 0.15]
Unknown tokens 0, first 100: []
Training lm from: [PosixPath('/home/pczapla/workspace/ulmfit-multilingual/data/wiki/fr-100/models/sp30k/lstm_nl4.m/lm_best'), PosixPath('/home/pczapla/workspace/ulmfit-multilingual/data/wiki/fr-100/models/sp30k/lstm_nl4.m/../itos')]
epoch train_loss valid_loss accuracy
1 3.072937 2.621444 0.468314
epoch train_loss valid_loss accuracy
1 2.737065 2.485359 0.486546
2 2.625827 2.353188 0.507669
3 2.408049 2.224600 0.527609
4 2.332804 2.113603 0.544255
5 2.242967 2.016229 0.560002
6 2.162170 1.925214 0.574050
7 2.094163 1.843778 0.587944
8 2.011285 1.773228 0.599802
9 1.931492 1.708201 0.611245
10 1.883735 1.643842 0.623145
11 1.793858 1.583394 0.635366
12 1.759305 1.526640 0.646132
13 1.741412 1.474198 0.657485
14 1.675670 1.430597 0.666407
15 1.624235 1.390453 0.674829
16 1.588415 1.359892 0.681364
17 1.594124 1.336594 0.686985
18 1.567758 1.322139 0.689745
19 1.536472 1.315883 0.690974
20 1.530144 1.314872 0.691084
/home/pczapla/workspace/ulmfit-multilingual/data/mldoc/fr-1/models/sp30k
Saving info /home/pczapla/workspace/ulmfit-multilingual/data/mldoc/fr-1/models/sp30k/lstm_nl4.m/info.json
Starting classifier training
epoch train_loss valid_loss accuracy
1 0.528480 0.428756 0.853000
epoch train_loss valid_loss accuracy
1 0.365323 0.224117 0.928000
epoch train_loss valid_loss accuracy
1 0.300881 0.199623 0.936000
epoch train_loss valid_loss accuracy
1 0.217855 0.198016 0.937000
2 0.206357 0.212208 0.938000
Saving models at /home/pczapla/workspace/ulmfit-multilingual/data/mldoc/fr-1/models/sp30k/lstm_nl4.m
Loss and accuracy using (cls_best): [0.18914989, tensor(0.9390)]
```
#### Second run
MultiCCA 92.05, ulmfit 93.67
```
python -m ulmfit cls --dataset-path data/mldoc/fr-1 --base-lm-path data/wiki/fr-100/models/sp30k/lstm_nl4.m --lang=fr --name 'nl4-2nd' --cuda-id=1 - train 20 --bs 40 --num-cls-epochs=8
Max vocab: 30000
Cache dir: /home/pczapla/workspace/ulmfit-multilingual/data/mldoc/fr-1/models/sp30k
Model dir: /home/pczapla/workspace/ulmfit-multilingual/data/mldoc/fr-1/models/sp30k/lstm_nl4-2nd.m
Loading validation /home/pczapla/workspace/ulmfit-multilingual/data/mldoc/fr-1/fr.dev.csv
Tokenized data loaded, lm.trn 13500, lm.val 1500
Tokenized data loaded, cls.trn 1000, cls.val 1000
Size of vocabulary: 30000
First 20 words in vocab: ['xxunk', 'xxpad', 'xxbos', 'xxfld', 'xxmaj', 'xxup', 'xxrep', 'xxwrep', '<unk>', '▁', '▁de', '▁,', '▁.', "'", 's', '▁la', '▁le', '▁et', '▁l', '▁à']
Training args: {'tie_weights': True, 'clip': 0.12, 'bptt': 70, 'pretrained_fnames': [PosixPath('/home/pczapla/workspace/ulmfit-multilingual/data/wiki/fr-100/models/sp30k/lstm_nl4.m/lm_best'), PosixPath('/home/pczapla/workspace/ulmfit-multilingual/data/wiki/fr-100/models/sp30k/lstm_nl4.m/../itos')], 'pretrained_model': None, 'drop_mult': 0.3} dps: [0.25 0.1 0.2 0.02 0.15]
Unknown tokens 0, first 100: []
Training lm from: [PosixPath('/home/pczapla/workspace/ulmfit-multilingual/data/wiki/fr-100/models/sp30k/lstm_nl4.m/lm_best'), PosixPath('/home/pczapla/workspace/ulmfit-multilingual/data/wiki/fr-100/models/sp30k/lstm_nl4.m/../itos')]
epoch train_loss valid_loss accuracy
1 3.080331 2.625329 0.467306
epoch train_loss valid_loss accuracy
1 2.750554 2.485271 0.486776
2 2.578659 2.353940 0.507637
3 2.422981 2.224983 0.527749
4 2.342781 2.113364 0.545006
5 2.254575 2.007775 0.560709
6 2.124016 1.920536 0.575680
7 2.068470 1.847463 0.586699
8 2.013289 1.775580 0.599840
9 1.929649 1.705369 0.612201
10 1.916013 1.646228 0.623175
11 1.825515 1.586714 0.634298
12 1.795780 1.529771 0.645840
13 1.725532 1.476651 0.656197
14 1.673942 1.429790 0.666030
15 1.639384 1.392116 0.674128
16 1.605681 1.359316 0.681356
17 1.560283 1.337794 0.686116
18 1.543926 1.323153 0.689276
19 1.531950 1.318164 0.690415
20 1.494068 1.316459 0.690586
/home/pczapla/workspace/ulmfit-multilingual/data/mldoc/fr-1/models/sp30k
Saving info /home/pczapla/workspace/ulmfit-multilingual/data/mldoc/fr-1/models/sp30k/lstm_nl4-2nd.m/info.json
Starting classifier training
epoch train_loss valid_loss accuracy
1 0.537720 0.395818 0.886000
epoch train_loss valid_loss accuracy
1 0.332603 0.232112 0.930000
epoch train_loss valid_loss accuracy
1 0.267323 0.230307 0.927000
epoch train_loss valid_loss accuracy
1 0.216402 0.226042 0.930000
2 0.231040 0.232696 0.936000
3 0.182048 0.217882 0.934000
4 0.170389 0.212531 0.937000
5 0.148332 0.214293 0.937000
6 0.124968 0.210322 0.936000
7 0.117591 0.234207 0.936000
8 0.109146 0.218597 0.938000
Saving models at /home/pczapla/workspace/ulmfit-multilingual/data/mldoc/fr-1/models/sp30k/lstm_nl4-2nd.m
Loss and accuracy using (cls_best): [0.21502711, tensor(0.9367)]
```
Reference in New Issue View Git Blame Copy Permalink