wassname/multifit
1
0
Fork 0
mirror of https://github.com/wassname/multifit.git synced 2026-08-22 12:10:15 +08:00
Code Issues Packages Projects Releases Wiki Activity

Update README.md with an example how to train your own language model

Browse Source
This commit is contained in:
Piotr Czapla
2019-11-11 10:39:08 +01:00
parent 8f23d396b9
commit f4ece1c083
1 changed files with 22 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+22
View File
@@ -62,6 +62,28 @@ learn.save_encoder("enc")
...
```
## How to pre-train your own language model
Here is how you can per-train a 'de' language model:
From command line:
```
$ bash prepare_wiki.sh de
$ python -W ignore -m multifit new multifit_paper_version replace_ --name my_lm - train_ --pretrain-dataset data/wiki/de-100
```
This should give you the pre-trained language model in 'data/wiki/de-100/models/sp15k/my_lm'
You can later use it as follows:
```
from fastai.text import *
import multifit
exp = multifit.from_pretrained("data/wiki/de-100/models/sp15k/my_lm")
exp.finetune_lm.train_("data/cls/de-books", num_epochs=20)
exp.classifier.train_(seed=0)
```
Please note, even though `python -m multifit new ` let's you pick other configurations than `multifit_paper_version` it is not recommended.
As the `from_pretrained` do not yet detect the configuration so your training specific parameters will be overwritten with
defaults from `multifit_paper_version`.
## Reproducing the results
This repository is a rewrite of the original training scripts so it lacks all the scripts used in the paper.
We are working on a port to fastai v2.0 and then we will be adding the scripts that show how to reproduce the results.