mirror of
https://github.com/wassname/multifit.git
synced 2026-09-10 12:12:50 +08:00
Removed dependency note, use requirements.txt
This commit is contained in:
@@ -25,16 +25,6 @@ The extracted data should be in the folder `wiki_extr` -> language name e.g.`en`
|
||||
|
||||
## Create and Post Process WikiText
|
||||
|
||||
### Get the Dependencies
|
||||
|
||||
**Python Fire**: To install Python Fire with pip, run: `pip install fire`
|
||||
|
||||
To install Python Fire with conda, run: `conda install fire -c conda-forge`
|
||||
|
||||
**Moses Tokenizer**: To install Moses Tokenizer: `pip install -U sacremoses`
|
||||
|
||||
Use the Python script [create_wikitext.py](./create_wikitext.py) to process the extracted Wikipedia documents.
|
||||
|
||||
### Create and Post-Process
|
||||
If you used the automated shell script from previous step, this might look something like
|
||||
```bash
|
||||
|
||||
Reference in New Issue
Block a user