diff --git a/mp_cnn/README.md b/mp_cnn/README.md index 4c4132c..cbc598a 100644 --- a/mp_cnn/README.md +++ b/mp_cnn/README.md @@ -61,6 +61,22 @@ python main.py mpcnn.trecqa.model --dataset trecqa --epochs 5 --regularization 0 The paper results are reported in [Noise-Contrastive Estimation for Answer Selection with Deep Neural Networks](https://dl.acm.org/citation.cfm?id=2983872). +## WikiQA Dataset + +You also need `trec_eval` for this dataset, similar to TrecQA. + +Then, you can run: +``` +python main.py mpcnn.wikiqa.model --epochs 10 --dataset wikiqa --batch-size 64 --lr 0.0004 --regularization 0.02 +``` +| Implementation and config | map | mrr | +| -------------------------------- |:------:|:------:| +| Paper | 0.693 | 0.709 | +| PyTorch using above config | 0.693 | 0.7091 | + +The paper results are reported in [Noise-Contrastive Estimation for Answer Selection with Deep Neural Networks](https://dl.acm.org/citation.cfm?id=2983872). + + These are not the optimal hyperparameters but they are decent. This README will be updated with more optimal hyperparameters and results in the future. To see all options available, use