mirror of
https://github.com/wassname/multifit.git
synced 2026-09-09 11:27:26 +08:00
109 lines
3.5 KiB
Markdown
109 lines
3.5 KiB
Markdown
# ulmfit-multilingual
|
||
Repository used for collaboration on application of ulmfit for multiple languages, it helps with pertraining and uses the
|
||
fastai v1 . (The version in n-waves/fastai:ulmfit_multilingual)
|
||
|
||
# How to train classifier
|
||
|
||
```
|
||
$ LANG=en
|
||
$ python -m ulmfit lm --dataset-path data/wiki/${LANG}-100 --tokenizer='f' --nl 3 --name 'orig' --max-vocab 60000 \
|
||
--lang ${LANG} --qrnn=False - train 10 --bs=50 --drop_mult=0 --label-smoothing-eps=0.0
|
||
...
|
||
Model name: data/wiki/en-100/models/f60k/lstm_orig.m
|
||
...
|
||
|
||
$ python -m ulmfit cls --dataset-path data/imdb --base-lm-path data/wiki/${LANG}-100/models/f60k/lstm_orig.m \
|
||
--lang=${LANG} --name orig - train 20 --bs 18 --num-cls-epochs=4 --lr_sched=1cycle --label-smoothing-eps=0.1
|
||
```
|
||
|
||
You can re-evaluate classifiers by running
|
||
```
|
||
python -m ulmfit eval --glob="imdb/models/*/lstm_*.m"
|
||
```
|
||
|
||
The same command can be used to quickly trian multiple classifiers, by adding the `--name` parameter:
|
||
```
|
||
python -m ulmfit eval --glob="imdb/models/*/lstm_nl3.m" --name "nl3-my-test1" --num-cls-epochs 4 --label-smoothing-eps=0.1 --lr_sched=1cycle
|
||
```
|
||
|
||
To create a tar with model simply run
|
||
```
|
||
python -m ulmfit tar data/imdb/models/f60k/lstm_nl3.m
|
||
```
|
||
|
||
## data directory strucutre
|
||
|
||
Directory structure after changes to the way we process wiki dumps.
|
||
```
|
||
data
|
||
├── imdb
|
||
│ ├── aclImdb
|
||
│ ├── imdb_lm
|
||
│ └── tmp
|
||
├── wiki
|
||
│ ├── de-100
|
||
│ │ └── models
|
||
│ ├── de-100-unk
|
||
│ │ └── models
|
||
│ ├── de-2
|
||
│ │ └── models
|
||
│ ├── de-2-unk
|
||
│ │ └── models
|
||
│ ├── de-all
|
||
│ │ └── models
|
||
│ ├── wikitext-103
|
||
│ │ └── models
|
||
│ └── wikitext-2
|
||
│ └── models
|
||
├── wiki_dumps
|
||
├── wiki_extr
|
||
│ └── de
|
||
│ ├── AA
|
||
│ ├── AB
|
||
...
|
||
└── CC
|
||
└── xnli
|
||
├── XNLI-1.0
|
||
└── XNLI-MT-1.0
|
||
├── multinli
|
||
└── xnli
|
||
```
|
||
|
||
## how to contribute
|
||
We have a fork of fastai to propose changes to fastai.text, with a branch for this project:
|
||
https://github.com/n-waves/fastai/tree/ulmfit_multilingual
|
||
|
||
Let us know that you want to start collaboration on fastai forum thread: [Multilingual ULMFIT](https://forums.fast.ai/t/multilingual-ulmfit/28117)
|
||
and you will get access to both repositories.
|
||
|
||
- Follow the [developer installation of fastai](https://github.com/fastai/fastai#developer-install)
|
||
- Add n-waves/fastai as additional remote as described here: https://help.github.com/articles/adding-a-remote/
|
||
|
||
Here is what I did:
|
||
```bash
|
||
$ cd fastai
|
||
$ git remote add n-waves https://github.com/n-waves/fastai.git
|
||
$ git remote -v
|
||
n-waves https://github.com/n-waves/fastai.git (fetch)
|
||
n-waves https://github.com/n-waves/fastai.git (push)
|
||
origin https://github.com/fastai/fastai.git (fetch)
|
||
origin https://github.com/fastai/fastai.git (push)
|
||
|
||
$ git fetch n-waves
|
||
$ git checkout ulmfit_multilingual
|
||
Branch 'ulmfit_multilingual' set up to track remote branch 'ulmfit_multilingual' from 'n-waves'.
|
||
Switched to a new branch 'ulmfit_multilingual'
|
||
|
||
$ git push --set-upstream n-waves ulmfit_multilingual # to automatically push ulmfit_multilingual branch to the n-waves repo
|
||
```
|
||
|
||
|
||
## Running tests
|
||
|
||
To run the tests, the following data is necessary:
|
||
|
||
- wikitext-2 (prepared by `./prepare_wiki-en.sh`, along with wikitext-103)
|
||
- imdb (prepared by `./prepare_imdb.sh`)
|
||
|
||
then simply run tests, e.g. `pytest .`
|