wassname/Castor
1
0
Fork 0
mirror of https://github.com/wassname/Castor.git synced 2026-09-10 11:40:44 +08:00
Code Issues Packages Projects Releases Wiki Activity
76 Commits 6 Branches 0 Tags
mp-cnn-twitter
Commit Graph
76 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Michael Tu ab7b667ce2 Add twitter dataset 2018-02-03 19:39:08 -05:00
rosequ ef21aa975f SHARCNET tutorial (#97) 2018-01-24 20:11:26 -05:00
Victor Yang 5fd31a0434 Update path name and README for NCE-SM (#95) 
* update nce-sm

* refactor code, update torchtext

* use shared evaluation

* refactor code, use shared data loader

* refactor code

* refactor code

* refactor code according to Michael's great suggestions

* update readme and requirement

* update datasets and readme

* update data loader

* add space between +

* update path name, update readme

* update data loader and dataset name

* refactor code

* update readme
 2018-01-24 14:12:37 -05:00
Michael Tu cbd54cb382 Kim CNN README remove torchtext reset (#96) 2018-01-18 10:42:09 -05:00
Victor Yang 51d8e29525 add NCE to MP-CNN (#84) 
* update nce-sm

* refactor code, update torchtext

* use shared evaluation

* refactor code, use shared data loader

* refactor code

* refactor code

* refactor code according to Michael's great suggestions

* update readme and requirement

* update datasets and readme

* update data loader

* add space between +

* update refactor code

* add nce-mp

* remove duplicate files

* update readme, refactor code according to mp_cnn and delete duplicate code, follow PEP8 standard

* refactor code, add/delete comments

* import exit from sys
 2018-01-03 18:12:57 -05:00
Michael Tu eee160ea41 Use view instead of unsqueeze since ONNX v1.0 doesn't support it (#94) 2017-12-08 15:04:29 -05:00
rosequ 85f35bb994 connected the pipeline (#65) (#93) 
* connected the pipeline

* minor changes to api code
 2017-12-07 20:04:18 -05:00
rosequ 68e0ef45b2 Util to build w2v pytorch model (#92) 
* util to build w2v pytorch model

* added the code to build the .pt model
 2017-12-06 11:34:29 -05:00
Michael Tu a363e3d256 Add flag to support onnx (#91) 2017-12-06 09:55:02 -05:00
rosequ 0a3ce7015c Update readme (#90) 
* updated readme for easier replication

* updated cd changes

* link change
 2017-12-05 21:58:55 -05:00
rosequ 28a198f33c Update bridge (#87) (#89) 
* initial commit of the updated bridge

* moved bridge file to the root

* after CR1

* after CR2
 2017-12-05 12:23:08 -05:00
rosequ aee1541722 avoids parsing exceptions (#88) 
* avoids parsing exceptions

* remove unnecessary print
 2017-12-02 10:23:40 -05:00
MeowFei ad69d3abc2 Connected to E2E pipeline (#85) 
* added RetrieveSentences.py

* removed index

* connet to E2E pipeline

* updated code

* Some modification

* clean up

* config

* documentation

* added documentation

* update documentation

* update doc

* update doc

* add js

* updated documentation

* changed doc

* changed requirements.txt
 2017-11-25 14:39:39 -05:00
Victor Yang 61c8c0e622 update nce-sm (#78) 
* update nce-sm

* refactor code, update torchtext

* use shared evaluation

* refactor code, use shared data loader

* refactor code

* refactor code

* refactor code according to Michael's great suggestions

* update readme and requirement

* update datasets and readme

* update data loader

* add space between +
 2017-11-09 17:03:43 -05:00
Michael Tu 0722dde06e MP-CNN: Add hyperparameters used to achieve paper perf for WikiQA (#82) 2017-11-08 09:51:04 -05:00
Michael Tu aaff291edb Update Kim CNN SST-1 dataset to use torchtext 0.2.0 (#81) 2017-11-07 21:37:18 -05:00
Michael Tu 2344354cbf MP-CNN: add trainer and evaluator for WikiQA (#80) 2017-11-06 20:37:40 -05:00
Michael Tu 4132874bad Add WikiQA Dataset (#79) 2017-11-04 18:41:32 -04:00
Michael Tu 4470983504 TrecQA for MP-CNN (#77) 
* Add TrecQA dataset and modularize MP-CNN infra

* Stylistic improvements

* Fix and warn about trec_eval path issue

* Update README for MP-CNN

* Update incorrect map/mrr

* MP-CNN: address code review comments

* Create common Castor pair Dataset class

* Move map and mrr computation to Castor utils

* Make map mrr utility trec_eval path more general
 2017-11-04 18:22:40 -04:00
Michael Tu 09b3a790a2 Use torchtext for MP-CNN (#76) 
* Add SICK torchtext Dataset

* SICK dataset - torchtext postprocess into class probs

* Update model, driver, trainer, evaluator for SICK for torchtext

* MP-CNN: Fix bugs that prevent SICK from running on gpu 0

* MP-CNN: make SICK dataset w/ torchtext GPU-agnostic

* MP-CNN: support sparse features / idf overlap with torchtext

* Add MSRVID dataset with torchtext and update MP-CNN code to use it

* MP-CNN: Make torchtext deterministic by setting python random seed

* SICK and MSRVID datasets - add pair id for debug and build test vocab

* MP-CNN: Update readme to address potential module not found error

* MP-CNN: address review comments, can run on cpu
 2017-11-01 12:13:30 -04:00
Peng Shi 4dea22b40f Deterministic CNN #45 (#71) 
* Deterministic CNN #45

* Describe the data source
 2017-10-30 22:23:00 -04:00
Ralph Tang 7957dc7638 Tweak conv-rnn model (#75) 
* Tweak conv-rnn model

- Fix misplaced zero_grad()
- Tweak model hyperparams and optimization algorithm

* Fix typo

* Add new results

* Clean up extraneous code
 2017-10-29 15:55:36 -04:00
MeowFei 8fd1ddcb4c Added RetrieveSentences.py (#74) 
* added RetrieveSentences.py

* removed index
 2017-10-28 15:56:20 -04:00
MeowFei 00ef2d0fde added RetrieveSentences.py (#73) 2017-10-28 15:05:52 -04:00
rosequ 491f0b32d5 Updated sm (#72) 
+ removed redundant loss regularization
+ added script to create torch word embedding file from word2vec model
+ updated README
 2017-10-12 22:54:45 -04:00
Michael Tu 857cd43934 MP-CNN: Optionally Visualize Training for Debugging (#67) 
* MP-CNN: optionally support TensorBoard for learning curve visualization

* MP-CNN: bug with SummaryWriter comment

* MP-CNN: add instructions on how to setup tensorboard
 2017-10-05 19:15:51 -04:00
Michael Tu e61db8e7c0 Remove Kim CNN data (required for bfg repo cleaning) 2017-10-05 15:09:11 -04:00
Ralph Tang e7ca33de28 Add convolutional RNN for sentence classification (#57) 
* Add SST data preprocessing

* Add ConvRNN model

* Add LR scheduler

* Add grid search on hyperparameters

* Add random search

* Add CLI options

* Add usage to README.md

* Refactor code

* Fix randomized search parameters

* Update README.md with results

* Use Dataset and DataLoader
 2017-10-05 14:14:31 -04:00
Peng Shi 511f29a2a6 kim cnn with torchtext (#59) 
* kim cnn with torchtext

* model

* import style
 2017-10-05 09:38:07 -04:00
rosequ 36b1ddb858 SM model for WikiQA (#63) (#64) 
* support for WikiQA dataset

* parallel runs for both datasets

* minor fixes

* updated README

* removed data folder; added scripts to create dataset; updated README

* after CR

* after CR2
 2017-10-04 15:59:26 -04:00
rosequ a471cea2c5 reimplementation of SM model (#48) (#62) 
* reimplementation of SM model

* features without normalization; parallel running of different modes

* minor fix
 2017-10-03 15:02:34 -04:00
Michael Tu 6362239233 MP-CNN: early stopping (#60) 2017-09-27 19:26:20 -04:00
Michael Tu aba4dd7e89 MP-CNN: Add Additional Features (#51) 
* MP-CNN: support external features

* MP-CNN: calculate overlap in same way as SM-model

* MP-CNN: properly calculate idf overlap
 2017-09-21 22:43:31 -04:00
Matt Crane ed4dba2497 Typo fix (#54) 2017-09-22 11:30:26 +09:00
Matt Crane a601722768 Option to disable the CuDNN backend (#52) 
Some of the kernels in this backend are known to be non-deterministic. It's also not clear to me whether it's enabled in a default install or not, enabled is set to True by default, but I think ultimately it's a runtime choice.
 2017-09-22 11:19:30 +09:00
Michael Tu a2904efe5a MP-CNN Bugfixes and Improvements (#50) 
* MP-CNN: use consistent unknown vector

* MP-CNN: Make optimizer, patience, etc.. configurable

* MP-CNN: bug fixes

* MP-CNN: update README

* MP-CNN: remove unused import
 2017-09-21 15:53:54 -04:00
Michael Tu 245b032545 Set gpu seed for SM model (#46) 
Small fix to set GPU seed for SM model
 2017-09-12 14:28:18 -04:00
Matt Crane a3294339fb Add seed and thread arguments (#43) 
So that any experiments can be controlled from the command line, I've set the defaults to what they were hardcoded (for the seed) and num_threads in kim_cnn to be consistent with the default in sm_cnn.
 2017-09-01 09:30:01 -04:00
rosequ d2d958cc51 Kim CNN: Early Stop Training (#40) (#41) 
Stop training if there's no improvement in the accuracy over 5 epochs
 2017-08-30 14:45:01 -04:00
Jimmy Lin fc91a0b04a Nuking simple_qa_rnn since we have separate BuboQA repo. (#42) 
cf. https://github.com/castorini/BuboQA
 2017-08-29 14:21:09 -04:00
Matt Crane b4dc87d46f Explicitly open files with utf-8 encoding (#35) 
Same issue as castorini/data#19
 2017-08-29 14:20:18 -04:00
Michael Tu 449d715ab0 MP-CNN PyTorch Re-implementation (#37) 
Re-implementation of MP-CNN in PyTorch.
 2017-08-04 19:37:34 -04:00
Peng Shi 53e0de03d7 Kim's sentence classification model: Upgrade to Python 3 (#36) 
Kim's sentence classification model: Upgrade to Python 3, per issue #31
 2017-07-30 08:10:44 +08:00
Michael Tu a6fc10818f Fix SM Model Internal Reproducibility Bug (#34) 
* Fix SM Model reproducibility bug

vocab is in different order every time, causing unseen words to use
different random states

* Make requirements.txt usable from conda and pip

The existing torch requirement does not work with conda or pip.
Also upgrade pytorch version while we are at it.
 2017-07-20 06:52:26 +08:00
Salman Mohammed 061cc8dd09 Used torchtext to refactor relation prediction model - much cleaner! (#33) 
Refactored the code for Ferhan's relation prediction model using torchtext
 2017-07-15 07:06:10 +08:00
Michael Tu a0755e6aa3 SM Model Jupyter Notebook Tutorial (#32) 
Notebook tutorial for SM CNN.
 2017-07-04 18:28:17 -04:00
Salman Mohammed 4c081645a7 clean up the relation prediction model for Simple QA - Ferhan's paper (#28) 
+ cleaned up the model code for the simple qa directory
+ created vocab objects for pre-loading word embeddings easily
 2017-06-19 14:24:27 -04:00
Matt Crane 945b1fa6c0 Fix vocab caching issue (#29) 
With the line as-was the vocab cache was stored as b'the' rather than the, meaning that word2vec wasn't found for terms causing massive performance loss (AP 0.71 cf 0.77).
 2017-06-19 13:02:36 -04:00
Michael Tu 43dd6fdb1f GPU Support for SM Model (#26) 
Add code to using GPU for the SM Model (#25). To use the GPU if one is available, add the --cuda optional parameter when calling main.py.
 2017-06-02 08:47:13 -04:00
Gaurav Baruah 92789cb9f5 E2e sweep (#24) 
Now ensuring that the bridge process raw candidate sentences fetched from the index, exactly as was done for the best performing SM model.
 2017-05-29 19:47:42 -04:00