mirror of
https://github.com/wassname/Castor.git
synced 2026-08-20 12:00:37 +08:00
Ashutosh-Adhikari
91ed6261db
* Add Regularization Modules for LSTM * Update Reuters Trainer and Evalueator for regularization * Remove unnecessary comments * Comply with PEP8 * Comply import order with PEP8 * Fix typos in README.md * Comply with PEP8 * Add BSD 3-Clause Licence * Remove deprecated call to Variable for PyTorch 0.4 * Update dataset selection in main * Remove block comments
1.5 KiB
1.5 KiB
lstm_baseline with Regularization
Implementation of a standard LSTM using PyTorch and Torchtext for text classification baseline measurements with Regularization.
Model Type
- rand: All words are randomly initialized and then modified during training.
- static: A model with pre-trained vectors from word2vec. All words -- including the unknown ones that are initialized with zero -- are kept static and only the other parameters of the model are learned.
- non-static: Same as above but the pretrained vectors are fine-tuned for each task.
Quick Start
To run the model on Reuters dataset on static, just run the following from the Castor working directory.
python -m lstm_baseline --mode static
Dataset
We experiment the model on the following datasets.
- Reuters dataset - ModApte splits
Settings
Adam is used for training with an option of temporal averaging.
TODO
- Support ONNX export. Currently throws a ONNX export failed (Couldn't export Python operator forward_flattened_wrapper) exception.
- Add dataset results with different hyperparameters
- Parameters tuning
Regularization Module
- Regularization methods like Embedding dropout, Weight Dropped LSTM and Temporal Activation Regularization are implemented.
- Temporal Averaging is also an additional module
Acknowledgement
- The additional modules have been heavily inspired by two open source repositories: