wassname/multifit
1
0
Fork 0
mirror of https://github.com/wassname/multifit.git synced 2026-08-22 12:10:15 +08:00
Code Issues Packages Projects Releases Wiki Activity
200 Commits 17 Branches 0 Tags
tmp
Commit Graph
200 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Piotr Czapla 837925ff53 Imporved validate_cls & eval to pick the best model based on val accuracy 2019-03-03 13:29:49 +01:00
Piotr Czapla 7b2ac9e94b Add ability to use random-init=True 2019-03-01 17:41:07 +01:00
Piotr Czapla 11b2b14523 Fix -m ulmfit tar method 2019-03-01 17:39:58 +01:00
Piotr Czapla 4742a68d28 Some logs from previous experiments 2019-02-27 20:31:38 +01:00
Marcin f750586114 Remove redundant bptt param 2019-02-26 18:39:38 +01:00
Piotr Czapla c6e0373170 Make models use bptt parameter 2019-02-26 18:06:23 +01:00
Piotr Czapla 852336c29b Merge pull request #38 from n-waves/lbl_smoothing 
Label smoothing
 2019-02-22 16:57:47 +01:00
Piotr Czapla 260faa703c Correct the label smoothing implementation 2019-02-22 16:55:35 +01:00
Piotr Czapla 015f04ec08 Training with noise & label smoothing 2019-02-22 12:02:18 +01:00
Piotr Czapla c69d31c420 New results lstm 30k 1cyc 2019-02-22 12:01:25 +01:00
Piotr Czapla 99d6b22447 Correct noise generation training + convenience functions 2019-02-20 10:24:30 +01:00
Piotr Czapla c29180a08f Add some results 2019-02-20 10:22:44 +01:00
Piotr Czapla 5529ec385b Add sentence piece char_coverage 0.99 for non european languages 
Otherwise Zh won't train on 15k
 2019-02-18 21:50:46 +01:00
Piotr Czapla 7dc7aac327 Merge all columns in classification task into first column 
This should fix CLS issues.
 2019-02-18 21:50:04 +01:00
Piotr Czapla 9fbcf56df3 Add different learning schedules, with default to the old schedule 
use --lr-sched=1cycle for better results
 2019-02-18 21:49:16 +01:00
Piotr Czapla ce6cc607ae Add saving itos.pkl so that the LM can be used to finetuning 2019-02-17 23:18:42 +01:00
Piotr Czapla c3276da062 Fixing Imdb loading 2019-02-17 23:18:12 +01:00
Piotr Czapla 119417fb6e Disable early stopping as it was causing OOMs 2019-02-17 23:04:25 +01:00
Piotr Czapla 490c792278 Upgrade to the recent the todays version of Fastai 2019-02-17 23:03:54 +01:00
Piotr Czapla 0dda4b4c2c Zeroshot MLDoc results for ulmfit trained on 10k examples 2019-02-17 19:03:02 +01:00
NAUSICAA\Julian 1ee1dd950d Merge branch 'master' of https://github.com/n-waves/ulmfit-multilingual into cls 2019-02-16 16:24:01 -03:00
Julian Eisenschlos b7e3a5e7a4 Update BERT Zero-shot results 2019-02-16 14:42:40 -03:00
Julian Eisenschlos be0511e42b Adding Bert results 2019-02-15 15:51:35 -03:00
Piotr Czapla 8733487d55 Make the validate vs train decision based on the existance of cls_last.pth istead of a model directory 2019-02-15 01:17:53 +01:00
Piotr Czapla 5dced1e488 Remove bidir 2019-02-15 01:16:37 +01:00
Piotr Czapla 5e44e442b2 Merge branch 'master' of https://github.com/n-waves/ulmfit-multilingual 2019-02-15 01:11:42 +01:00
Piotr Czapla 0e6534ad7b Expose num_lm_epochs in ulmfit eval 2019-02-15 01:11:39 +01:00
Marcin 22cb8b1660 Download pretrained models 2019-02-15 00:02:04 +01:00
Piotr Czapla 0f084168c1 Merge branch 'master' of https://github.com/n-waves/ulmfit-multilingual 2019-02-14 22:35:29 +01:00
Piotr Czapla cd47b3b5dc Fix use_moses=True for mldoc so that it is identical to wiki with uses_moses=False 
The issue was that Moses was executed after pre_rules when use_moses = True, But when data set was pre tokenized with Moses (use_moses=False) the pre_rules were executed  after.
So our wikipedia had the following processing:
- raw text
- Moses
- pre_rules
- split(' ') # fastai BaseTokenizer
- post_rules
- sentence piece

While mldoc had the following tokenziation
- raw text
- pre_rules
- Moses
- post_rules
- sentence piece

After fix I've retrained the classfiers (without finetuning) and I haven't notice huge changes in the performance. 4 languages received slight improvment 4 got a slight decrease in performance.
 2019-02-14 22:35:20 +01:00
Piotr Czapla c28c0fde16 Make ulmfit eval more secure and give more flexibility in dataset_template 
The dataset_template can use lang as additional token to construct globs patterns.
 2019-02-14 22:28:25 +01:00
Marcin 6fda7f2cda Download preprocessed wikis 2019-02-14 16:45:51 +01:00
Marcin fdac9f7ccd Save only the best LM model 2019-02-14 14:01:33 +01:00
Marcin 72e86cefc9 Add QRNN LM results for Italian 2019-02-14 12:48:04 +01:00
Marcin 0535ef169a QRNN mldoc results for de, en and es 2019-02-14 11:05:02 +01:00
Marcin b609951561 Fix path of pretrained model 2019-02-14 00:00:56 +01:00
Marcin 9d893e393b Add QRNN results 2019-02-14 00:00:43 +01:00
Marcin Kardas f95a16cee1 Merge pull request #34 from n-waves/use-configs 
Use configs
 2019-02-13 22:58:47 +01:00
Piotr Czapla 1340f4235c Improve ulmfit eval to allow for zeroshot laser evaluation 2019-02-13 15:29:55 +01:00
Piotr Czapla b14a393671 Add more results including full zeroshot results 2019-02-13 15:29:16 +01:00
NAUSICAA\Julian 4215111240 jp -> ja 2019-02-13 10:50:01 -03:00
NAUSICAA\Julian e657423e7f CLS Download Script 2019-02-13 10:41:24 -03:00
Marcin 0e2211dfaf Merge branch 'master' into use-configs 2019-02-13 14:10:59 +01:00
Marcin 2eee051c67 Expose max length parameter 2019-02-13 13:40:05 +01:00
Piotr Czapla 3586a7dcf9 Merge branch 'pr/29' 2019-02-12 20:43:44 +01:00
Piotr Czapla 4e1b76feee Add MLDoc summary & zeroshot logs 2019-02-12 15:01:42 +01:00
Piotr Czapla e72cdfb6db Add result logs (it) 2019-02-12 15:01:22 +01:00
Piotr Czapla e7271f2a29 Add ability to evalulate multiple models at once 2019-02-12 15:01:01 +01:00
Piotr Czapla a1e7a79b57 Add result logs 2019-02-12 15:00:29 +01:00
Marcin 8497cc1e5c Move LM and classifier parameters to configs 2019-02-12 03:09:51 +01:00