NAUSICAA\Julian
a8f51900bc
Quiet wget
2018-12-19 08:40:25 -03:00
NAUSICAA\Julian
08102657bb
Remove verbose flag
2018-12-19 08:18:31 -03:00
NAUSICAA\Julian
2a36762837
Merge branch 'master' of https://github.com/n-waves/ulmfit-multilingual into datasets
2018-11-19 16:34:15 -03:00
Aayush
5ffbe8ba5c
add vocab_size to sentencepiece
2018-11-19 19:13:55 +05:30
Aayush
69ed7b4169
Added preliminary test scripts for sentencepiece
...
Other minor changes:
- Function renaming: `test_pretrain_lm` -> `test_ulmfit_default_end_to_end`
- Delete test models after completing each test.
2018-11-19 19:11:20 +05:30
Nirant
56a9feec7d
Removed dependency note, use requirements.txt
2018-11-19 12:22:41 +05:30
Nirant K
635fb1c113
Add prepare_xnli.py
2018-11-19 06:50:50 +00:00
NAUSICAA\Julian
6292412eff
Adding scripts to download and format RCV and Webis CLS X-Language datasets
2018-11-18 22:29:07 -03:00
Piotr Czapla
64ad8d8fd9
Merge pull request #8 from n-waves/sentencepiece
...
[WIP] Sub-word tokenization with sentencepiece
2018-11-16 23:35:01 +01:00
Piotr Czapla
36b056a465
Fix issues discovered during execution of end-to-end test.
2018-11-16 23:31:16 +01:00
Piotr Czapla
a21f100ab2
Merge branch 'master' into sentencepiece
2018-11-16 23:23:49 +01:00
Piotr Czapla
45389be413
Use drop_mult properly
2018-11-16 20:02:32 +01:00
Piotr Czapla
23b17da61e
Add end to end test and improve the train_clas params
2018-11-16 19:51:29 +01:00
Piotr Czapla
977a506cfd
Fix bug where LM was always intialized as QRNN despite the paramters.
2018-11-16 19:50:36 +01:00
Aayush
e72b19499c
fix indent
2018-11-16 23:54:48 +05:30
aayush
59c8852b5d
sentencepiece for pretraining
...
modified: fastai_contrib/utils.py
modified: ulmfit/pretrain_lm.py
2018-11-16 23:14:24 +05:30
Aayush
fb2ed98ac6
Merge branch 'master' into sentencepiece
2018-11-16 23:09:06 +05:30
Piotr Czapla
f0e538f621
Add support for QRNN training to train_clas
2018-11-16 17:04:52 +01:00
Piotr Czapla
d4698e095c
Change back temporary folder stucture
...
from wiki/_dumps to wiki_dumps . I'm not sure why I've changed that.
2018-11-16 00:21:14 +01:00
Piotr Czapla
363130bcff
Update to the docs
2018-11-16 00:17:41 +01:00
Piotr Czapla
87afa54f63
Merge pull request #9 from n-waves/classifier-fixes
...
Classifier fixes
2018-11-15 23:48:16 +01:00
Piotr Czapla
d5d61d31d2
Add directory structure to readme.
2018-11-15 23:47:48 +01:00
Piotr Czapla
6ba3e68f9e
Minor tweaks.
2018-11-15 23:41:02 +01:00
Piotr Czapla
08d71dd45e
Add script that fetches and prepres WT-2 WT-103
2018-11-15 23:40:50 +01:00
Piotr Czapla
9a60ef2fbd
Add assertions to train_clas and ability to limit the dataset size
2018-11-15 23:40:10 +01:00
Piotr Czapla
d8b95430b8
Chain all scripts needed to prepare a wiki dump, put .unk to new folder
2018-11-15 23:39:27 +01:00
Piotr Czapla
111fc7e4c3
Add perpare_imdb script
2018-11-15 23:37:15 +01:00
NAUSICAA\Julian
b9c587d9bd
Adding SentencePieceTokenizer
2018-11-15 19:18:42 -03:00
Nirant K
43340d03bc
Fix minor typos in train_clas
2018-11-15 18:04:15 +00:00
Aayush
1cc49e666f
Quick fixes to imports etc.
2018-11-15 23:13:16 +05:30
Aayush
3e8f9f8b5f
[WIP] Sub-word tokenization with sentencepiece
...
For resolving issue #3 .
@eisenjulian let's use this branch. I've written some skeleton code (untested currently) to be used around your tokenizer. You can commit that into fastai_contrib for our purpose.
2018-11-15 23:01:49 +05:30
Nirant K
56fc150eb1
Merge xnli
2018-11-15 15:53:37 +00:00
Nirant K
653f6e96a0
Classifier fails at fit_one_cycle
2018-11-15 15:03:52 +00:00
Nirant K
6622458fe7
Add aclImdb extractor
2018-11-15 15:03:12 +00:00
Sebastian
94fa1a3ecb
Encapsulated data reading in utils method, removed fastai processing, added doc string
2018-11-15 14:36:50 +00:00
Sebastian
300d78ee44
Consolidated methods for reading classification data, added method to read XNLI data
2018-11-15 14:34:07 +00:00
Sebastian
d5e6849cd8
Shortened XNLI download script, removed unnecessary parts; extracts now to data folder
2018-11-15 14:33:21 +00:00
Nirant K
62e6777952
Add download and unzip script
2018-11-14 15:53:59 +00:00
Nirant
898c9255e0
Fix minor typos
2018-11-14 21:07:15 +05:30
Nirant K
0fb8161061
Merge branch 'master' of https://github.com/n-waves/ulmfit-multilingual
2018-11-14 13:56:58 +00:00
Nirant K
70680c21ce
Add better logs, add multi-language flag to pretrain
...
- Added lang variable to pretrain_lm
- Tried the QRNN model with Hindi
- Added better logging and consistent model saving paths
- Fixed minor bug in validate(model, ...) to to validate(learn.model, ...)
2018-11-14 13:54:31 +00:00
Nirant
5f845724d0
Add dependencies
2018-11-14 17:13:40 +05:30
Nirant
820ff231bd
Add Download and Process Docs
2018-11-14 17:04:47 +05:30
Piotr Czapla
38ca35c044
Fix pretrain_lm to work with yesterdays changes to fastai text API
2018-11-14 11:41:26 +01:00
Piotr Czapla
e13dbea1e1
Remove missleading docs.
2018-11-12 13:14:44 +01:00
Piotr Czapla
5ad6c2ace1
Update to new fastai.master with fixed qrnn issue
2018-11-12 13:08:00 +01:00
Piotr Czapla
40ca6c200e
Refactor and temporary fix the pretrain_lm by adding learn.lr_find
...
For some reason this fixes the training issue. I though I've nailed it down but i haven't. :/ .
2018-11-10 00:32:29 +01:00
Piotr Czapla
eb1da0b630
Updated README.md
2018-11-08 21:05:48 +01:00
Piotr Czapla
12e52b182e
Initial import from fastaiv1
...
initial import from https://github.com/fastai/fastai/tree/ulmfit_v1/courses/dl2/imdb_scripts
2018-11-08 20:33:10 +01:00
Piotr Czapla
2f09109d5d
Initial commit
2018-11-08 19:52:07 +01:00
NAUSICAA\Julian