Piotr Czapla
837925ff53
Imporved validate_cls & eval to pick the best model based on val accuracy
2019-03-03 13:29:49 +01:00
Piotr Czapla
7b2ac9e94b
Add ability to use random-init=True
2019-03-01 17:41:07 +01:00
Marcin
f750586114
Remove redundant bptt param
2019-02-26 18:39:38 +01:00
Piotr Czapla
c6e0373170
Make models use bptt parameter
2019-02-26 18:06:23 +01:00
Piotr Czapla
260faa703c
Correct the label smoothing implementation
2019-02-22 16:55:35 +01:00
Piotr Czapla
015f04ec08
Training with noise & label smoothing
2019-02-22 12:02:18 +01:00
Piotr Czapla
99d6b22447
Correct noise generation training + convenience functions
2019-02-20 10:24:30 +01:00
Piotr Czapla
7dc7aac327
Merge all columns in classification task into first column
...
This should fix CLS issues.
2019-02-18 21:50:04 +01:00
Piotr Czapla
9fbcf56df3
Add different learning schedules, with default to the old schedule
...
use --lr-sched=1cycle for better results
2019-02-18 21:49:16 +01:00
Piotr Czapla
c3276da062
Fixing Imdb loading
2019-02-17 23:18:12 +01:00
Piotr Czapla
119417fb6e
Disable early stopping as it was causing OOMs
2019-02-17 23:04:25 +01:00
Piotr Czapla
490c792278
Upgrade to the recent the todays version of Fastai
2019-02-17 23:03:54 +01:00
Piotr Czapla
5dced1e488
Remove bidir
2019-02-15 01:16:37 +01:00
Piotr Czapla
0f084168c1
Merge branch 'master' of https://github.com/n-waves/ulmfit-multilingual
2019-02-14 22:35:29 +01:00
Piotr Czapla
cd47b3b5dc
Fix use_moses=True for mldoc so that it is identical to wiki with uses_moses=False
...
The issue was that Moses was executed after pre_rules when use_moses = True, But when data set was pre tokenized with Moses (use_moses=False) the pre_rules were executed after.
So our wikipedia had the following processing:
- raw text
- Moses
- pre_rules
- split(' ') # fastai BaseTokenizer
- post_rules
- sentence piece
While mldoc had the following tokenziation
- raw text
- pre_rules
- Moses
- post_rules
- sentence piece
After fix I've retrained the classfiers (without finetuning) and I haven't notice huge changes in the performance. 4 languages received slight improvment 4 got a slight decrease in performance.
2019-02-14 22:35:20 +01:00
Marcin
0e2211dfaf
Merge branch 'master' into use-configs
2019-02-13 14:10:59 +01:00
Marcin
2eee051c67
Expose max length parameter
2019-02-13 13:40:05 +01:00
Piotr Czapla
3586a7dcf9
Merge branch 'pr/29'
2019-02-12 20:43:44 +01:00
Piotr Czapla
e7271f2a29
Add ability to evalulate multiple models at once
2019-02-12 15:01:01 +01:00
Marcin
8497cc1e5c
Move LM and classifier parameters to configs
2019-02-12 03:09:51 +01:00
Piotr Czapla
c63fe258d2
Fix validataion and add option to add noise to training labels
2019-02-11 10:53:25 +01:00
Piotr Czapla
26736d95de
Add code to test & train mldoc classifier
2019-02-10 09:52:58 +01:00
Tomasz Pietruszka
a1e66c39d4
tokenzier->tokenizer typo
2019-01-13 17:27:29 +01:00
Piotr Czapla
4a858f3572
Simplfy and unify input data parsing
2019-01-01 14:43:12 +01:00
Piotr Czapla
82d6a30b11
Fix SentencePiece implementation, to get 94.5% on imdb
...
- train on whole articles when tokenizer is sentencepice
- add moses tokenizer to get the same tokens on imdb as on wt103
- apply pre / post processing rules to moses tokenzier so that sentencepiece works on correctly preprocessed text (lowercased with html removed)
- add alpha implementation on xnli (no tests)
- remove vocab adaptation when swiching from wiki to imdb. As otherwise we get 50% of missing words and 93% accuracy on imdb
2018-12-27 15:14:41 +01:00
Piotr Czapla
ff30fb5642
Fastai upgrade
2018-12-27 14:57:15 +01:00
Piotr Czapla
f9394b9af1
Add callbacks to save history and best weights remove bs & drop_mult
2018-12-12 00:31:16 +01:00
Piotr Czapla
1b9d04d7ef
Expose use_test_for_validation as param to train
2018-12-09 22:45:21 +01:00
Piotr Czapla
34b1d18600
load wikipedia as articles for v & fv tok.
2018-12-09 22:42:17 +01:00
Piotr Czapla
2338618563
Make biclasifier head a hyperparameter.
2018-12-09 00:39:27 +01:00
Piotr Czapla
50ab34eea9
Fix classification scripts and give a way to test accuracy on test set
2018-12-07 16:10:20 +01:00
Piotr Czapla
2acbf15555
Tweak hyper training params of cls (drop_mul, bs, true_wd=True)
...
I've set the same hyperparams as in lesson3
2018-12-05 16:27:14 +01:00
Piotr Czapla
4454af167f
Use more text during pretraining of imdb
2018-12-05 16:26:16 +01:00
Piotr Czapla
908c3d7e8a
bug fix
2018-12-04 16:41:07 +01:00
Piotr Czapla
f25deb3049
Use trn + tst for LM training
2018-12-04 01:43:01 +01:00
Piotr Czapla
9826f6881c
Fix the way trn & val set is created in imdb
2018-12-04 01:35:59 +01:00
Piotr Czapla
039624870a
Fix loading tokenized data set in train cls
2018-12-04 01:28:03 +01:00
Piotr Czapla
82c955ce6a
Add different tokenization algorithms to train_clas
2018-12-04 00:51:21 +01:00
Piotr Czapla
c17dcce75e
spelling
2018-12-01 15:24:30 +01:00
Piotr Czapla
4b29376b44
Rewrite classifier to use changed pretrain_lm
2018-12-01 10:58:46 +01:00
Piotr Czapla
be117abac4
Make the end to end test run correctly
2018-11-24 23:51:23 +01:00
Piotr Czapla
8da47324c2
Clean ups and fixes
2018-11-22 15:32:40 +01:00
Piotr Czapla
dbd4884228
Fixes after mergin with master and updateing to newset fastai
2018-11-22 01:14:38 +01:00
Piotr Czapla
9aa877dcd0
Share trained LM between different classfiication runs
2018-11-21 18:51:42 +01:00
Piotr Czapla
979eb196d8
Change the classfication training learning rate to the one that was working te best in my exp. on bidirectional clasification
2018-11-21 18:46:05 +01:00
Piotr Czapla
7f1f8efcc3
Working version of biclassfier
2018-11-19 12:58:58 +01:00
Piotr Czapla
c821d2e783
first version of bi classifier
2018-11-19 09:59:08 +01:00
Piotr Czapla
2674a713fc
fix resuming training of classifier
2018-11-17 17:13:15 +01:00
Piotr Czapla
e97085337e
Fix dropout and classification accuracy. 0.91 on imdb
2018-11-17 16:47:17 +01:00
Piotr Czapla
aa6b59a0b3
Fix vocab size so that it remains 60k as in case of pretrain_lm + add longer lm training
2018-11-16 23:19:15 +01:00
Piotr Czapla