* Update instructions to use Castor-models
* Consolidate requirements.txt
* Refine README with convenience scripts
* Update internal instructions
* MP-CNN working dir minor edit
* Refactor main README
* Update Anserini Dependency docs
* Update idf baseline and Kim CNN docs to use Castor-data
* Update remaining READMEs to reference Castor-data
* Change default path from data to Castor-data
* Fix wrong order of embeddings path
Some of the kernels in this backend are known to be non-deterministic. It's also not clear to me whether it's enabled in a default install or not, enabled is set to True by default, but I think ultimately it's a runtime choice.
So that any experiments can be controlled from the command line, I've set the defaults to what they were hardcoded (for the seed) and num_threads in kim_cnn to be consistent with the default in sm_cnn.
* Fix SM Model reproducibility bug
vocab is in different order every time, causing unseen words to use
different random states
* Make requirements.txt usable from conda and pip
The existing torch requirement does not work with conda or pip.
Also upgrade pytorch version while we are at it.
With the line as-was the vocab cache was stored as b'the' rather than the, meaning that word2vec wasn't found for terms causing massive performance loss (AP 0.71 cf 0.77).
Michael Tu