88 Commits
Author SHA1 Message Date
Piotr Czapla be117abac4 Make the end to end test run correctly 2018-11-24 23:51:23 +01:00
Piotr Czapla 8da47324c2 Clean ups and fixes 2018-11-22 15:32:40 +01:00
Piotr Czapla dbd4884228 Fixes after mergin with master and updateing to newset fastai 2018-11-22 01:14:38 +01:00
Piotr Czapla 934fc79384 Merge branch 'master' into bilm 2018-11-21 23:48:23 +01:00
Piotr Czapla 9aa877dcd0 Share trained LM between different classfiication runs 2018-11-21 18:51:42 +01:00
Piotr Czapla 979eb196d8 Change the classfication training learning rate to the one that was working te best in my exp. on bidirectional clasification 2018-11-21 18:46:05 +01:00
Piotr Czapla 6e3ef21b1f Add Avg BiClassifier 2018-11-21 18:44:49 +01:00
Piotr Czapla cbed02d5e0 Merge branch 'master' into bilm 2018-11-21 18:23:54 +01:00
NAUSICAA\Julian 8cb867b066 Compatibility with new fastai version 2018-11-20 19:21:02 -03:00
NAUSICAA\Julian 79691791b3 Typo fix 2018-11-20 18:53:23 -03:00
Julian Eisenschlos 88d94f38ba Merge pull request #16 from n-waves/sentencepiece_fixes
Sentencepiece Fixes
2018-11-20 17:14:41 -03:00
NAUSICAA\Julian 40a2990322 Add suggested changes to rules system 2018-11-20 09:06:03 -03:00
NAUSICAA\Julian a36c0518c6 Merge branch 'master' of https://github.com/n-waves/ulmfit-multilingual into sentencepiece_fixes 2018-11-20 08:55:37 -03:00
Aayush 429a3fe4c2 Merge pull request #13 from n-waves/models_path_fix
Models path fix, default_rules and sentencepiece support for train_clas
2018-11-20 12:25:25 +05:30
NAUSICAA\Julian 3b5629c0c5 Removing files 2018-11-20 00:46:04 -03:00
NAUSICAA\Julian 0ecc6342a6 Fixes when running sentence piece end to end 2018-11-20 00:38:44 -03:00
NAUSICAA\Julian a08881180c Merge branch 'master' of https://github.com/n-waves/ulmfit-multilingual into polyglot-lm 2018-11-19 16:33:44 -03:00
NAUSICAA\Julian 470f984ca4 Script to merge two languages 2018-11-19 16:31:38 -03:00
Piotr Czapla 2bde30a402 Add experiment with biclassifier (concat) 2018-11-19 15:31:10 +01:00
Piotr Czapla 9f526a9849 Add experiment where wd_true is set to True. 2018-11-19 15:30:28 +01:00
Aayush 73403bdcb3 fix double EOS in read_imdb 2018-11-19 19:30:11 +05:30
Aayush 6e867811db Support for running sentencepiece with train_clas
Committing into this branch. Only changes are in `read_xnli` and `read_imdb`.
2018-11-19 19:28:19 +05:30
Aayush 5ffbe8ba5c add vocab_size to sentencepiece 2018-11-19 19:13:55 +05:30
Aayush 69ed7b4169 Added preliminary test scripts for sentencepiece
Other minor changes:
- Function renaming: `test_pretrain_lm` -> `test_ulmfit_default_end_to_end`
- Delete test models after completing each test.
2018-11-19 19:11:20 +05:30
Piotr Czapla 7f1f8efcc3 Working version of biclassfier 2018-11-19 12:58:58 +01:00
Piotr Czapla c821d2e783 first version of bi classifier 2018-11-19 09:59:08 +01:00
Nirant 56a9feec7d Removed dependency note, use requirements.txt 2018-11-19 12:22:41 +05:30
Nirant K 635fb1c113 Add prepare_xnli.py 2018-11-19 06:50:50 +00:00
NAUSICAA\Julian 6292412eff Adding scripts to download and format RCV and Webis CLS X-Language datasets 2018-11-18 22:29:07 -03:00
Piotr Czapla 37b73e262f Fix bug where de-all was de-100 2018-11-17 17:14:13 +01:00
Piotr Czapla 2674a713fc fix resuming training of classifier 2018-11-17 17:13:15 +01:00
Piotr Czapla e97085337e Fix dropout and classification accuracy. 0.91 on imdb 2018-11-17 16:47:17 +01:00
Sebastian d796610452 Removed deprecated default_rules 2018-11-17 14:48:06 +00:00
Sebastian 7fe059c9a2 Fixed models path for vocabulary 2018-11-17 14:47:27 +00:00
Piotr Czapla e1418b2114 fix model dir mkdir 2018-11-17 00:11:56 +01:00
Piotr Czapla 5cc4bfd444 Add loading weights and itos, so that we can extend training 2018-11-16 23:48:22 +01:00
Piotr Czapla 895a58c88d Merge branch 'master' into bilm 2018-11-16 23:39:15 +01:00
Piotr Czapla 64ad8d8fd9 Merge pull request #8 from n-waves/sentencepiece
[WIP] Sub-word tokenization with sentencepiece
2018-11-16 23:35:01 +01:00
Piotr Czapla 36b056a465 Fix issues discovered during execution of end-to-end test. 2018-11-16 23:31:16 +01:00
Piotr Czapla a21f100ab2 Merge branch 'master' into sentencepiece 2018-11-16 23:23:49 +01:00
Piotr Czapla aa6b59a0b3 Fix vocab size so that it remains 60k as in case of pretrain_lm + add longer lm training 2018-11-16 23:19:15 +01:00
Piotr Czapla ebbf09775f Merge branch 'master' into bilm 2018-11-16 20:09:41 +01:00
Piotr Czapla 45389be413 Use drop_mult properly 2018-11-16 20:02:32 +01:00
Piotr Czapla 23b17da61e Add end to end test and improve the train_clas params 2018-11-16 19:51:29 +01:00
Piotr Czapla 977a506cfd Fix bug where LM was always intialized as QRNN despite the paramters. 2018-11-16 19:50:36 +01:00
Aayush e72b19499c fix indent 2018-11-16 23:54:48 +05:30
aayush 59c8852b5d sentencepiece for pretraining
modified:   fastai_contrib/utils.py
	modified:   ulmfit/pretrain_lm.py
2018-11-16 23:14:24 +05:30
Aayush fb2ed98ac6 Merge branch 'master' into sentencepiece 2018-11-16 23:09:06 +05:30
Piotr Czapla f0e538f621 Add support for QRNN training to train_clas 2018-11-16 17:04:52 +01:00
Piotr Czapla d4698e095c Change back temporary folder stucture
from wiki/_dumps to  wiki_dumps . I'm not sure why I've changed that.
2018-11-16 00:21:14 +01:00