wassname/multifit
1
0
Fork 0
mirror of https://github.com/wassname/multifit.git synced 2026-09-09 11:27:26 +08:00
Code Issues Packages Projects Releases Wiki Activity

Fix -m ulmfit tar method

Browse Source
This commit is contained in:
Piotr Czapla
2019-03-01 17:39:58 +01:00
parent 4742a68d28
commit 11b2b14523
2 changed files with 11 additions and 7 deletions
Download Patch File Download Diff File Expand all files Collapse all files
results/logs/de.md
+3 -3
View File
@@ -169,10 +169,10 @@ epoch train_loss valid_loss accuracy
1 0.175013 0.280545 0.921000
2 0.178333 0.286059 0.923000
Saving models at data/mldoc/de-1/models/vf60k/lstm_nl3.m
Loss and accuracy using (cls_last): [0.28054512, tensor(0.9210)]
Loss and accuracy using (cls_best): [0.28054512, tensor(0.9210)]
Loss and accuracy using (cls_best): [0.16954255, tensor(0.9475)]
OrderedDict([('data/mldoc/de-1/models/vf60k/lstm_nl3.m', 0.9474999904632568)])
```
MultiCCA: 93.7% , ulmfit: 92.1%
MultiCCA: 93.7% , ulmfit: 94.74%
## SP30k LSTM nl 4
### LM
```
ulmfit/__main__.py
+8 -4
View File
@@ -75,14 +75,18 @@ class ULMFiT:
print(df)
def tar(self, model_path):
data_dir = (Path.cwd()/"data").resolve()
params = CLSHyperParams.from_json(model_path)
tar_name = f"models/{params.lang}-{params.tokenizer_prefix}-{params.model_name}.tar"
name = str(params.dataset_dir.resolve().relative_to(data_dir)).replace("/", "-")
tar_name = f"models/{name}-{params.tokenizer_prefix}-{params.model_name}.tar"
print("Storing model in", tar_name)
with tarfile.open(tar_name, mode="w") as tar:
for g in map(params.model_dir.glob, ['*_last.*', 'info.json', 'info.json', '../spm.*', '../itos.*',]):
for g in map(params.model_dir.glob, ['*_best.pth', 'info.json', '../spm.*', '../itos.*',]):
for f in g:
print("Adding", f, f.relative_to("data"))
tar.add(f, f.relative_to("data"))
dest = f.resolve().relative_to(Path.cwd())
print("Adding", f, dest)
tar.add(f, dest)
def eval(self, glob="mldoc/*-1/models/sp30k/lstm_nl4.m", dataset_template='${lang}-1', name="tmp-100", num_lm_epochs=0, cuda_id=0, **trn_params):
results = OrderedDict()