* Add TrecQA dataset and modularize MP-CNN infra
* Stylistic improvements
* Fix and warn about trec_eval path issue
* Update README for MP-CNN
* Update incorrect map/mrr
* MP-CNN: address code review comments
* Create common Castor pair Dataset class
* Move map and mrr computation to Castor utils
* Make map mrr utility trec_eval path more general
* Add SICK torchtext Dataset
* SICK dataset - torchtext postprocess into class probs
* Update model, driver, trainer, evaluator for SICK for torchtext
* MP-CNN: Fix bugs that prevent SICK from running on gpu 0
* MP-CNN: make SICK dataset w/ torchtext GPU-agnostic
* MP-CNN: support sparse features / idf overlap with torchtext
* Add MSRVID dataset with torchtext and update MP-CNN code to use it
* MP-CNN: Make torchtext deterministic by setting python random seed
* SICK and MSRVID datasets - add pair id for debug and build test vocab
* MP-CNN: Update readme to address potential module not found error
* MP-CNN: address review comments, can run on cpu
* MP-CNN: optionally support TensorBoard for learning curve visualization
* MP-CNN: bug with SummaryWriter comment
* MP-CNN: add instructions on how to setup tensorboard
* support for WikiQA dataset
* parallel runs for both datasets
* minor fixes
* updated README
* removed data folder; added scripts to create dataset; updated README
* after CR
* after CR2
Some of the kernels in this backend are known to be non-deterministic. It's also not clear to me whether it's enabled in a default install or not, enabled is set to True by default, but I think ultimately it's a runtime choice.
So that any experiments can be controlled from the command line, I've set the defaults to what they were hardcoded (for the seed) and num_threads in kim_cnn to be consistent with the default in sm_cnn.
* Fix SM Model reproducibility bug
vocab is in different order every time, causing unseen words to use
different random states
* Make requirements.txt usable from conda and pip
The existing torch requirement does not work with conda or pip.
Also upgrade pytorch version while we are at it.
With the line as-was the vocab cache was stored as b'the' rather than the, meaning that word2vec wasn't found for terms causing massive performance loss (AP 0.71 cf 0.77).
+ sm model no external features baseline
+ sm model with IDF weights
+ sm model with IDF weights without removing punctuation --> barely better than df/idf (a la Pytorch).
+ sm model with stemming before computing IDF weights
^ all on the TrecQA dataset
Michael Tu