mirror of
https://github.com/wassname/Castor.git
synced 2026-09-11 11:51:23 +08:00
gauravbaruah
a67e2d12c4
+ sm model no external features baseline + sm model with IDF weights + sm model with IDF weights without removing punctuation --> barely better than df/idf (a la Pytorch). + sm model with stemming before computing IDF weights ^ all on the TrecQA dataset
24 lines
1.9 KiB
Plaintext
24 lines
1.9 KiB
Plaintext
Baseline_method idf condition dataset train_MAP dev_MAP test_MAP Notes/Comments
|
||
idf_sum_similarity dataset no_stopping + no_stemming TrecQA 0.7744 0.7668 0.7014 IDF is computed over terms in the train, dev and test sets
|
||
idf_sum_similarity dataset with_stopping + with_stemming TrecQA 0.7078 0.7404 0.6816
|
||
idf_sum_similarity dataset with_stopping + with_stemming + stop(punct) TrecQA 0.7074 0.7337 0.6805 also stopping punctuation
|
||
idf_sum_similarity dataset no_stopping + no_stemming WikiQA 0.2429 0.2489 0.2219 There is less overlap between questions and answers in the WikiQA dataset
|
||
idf_sum_similarity dataset with_stopping + with_stemming WikiQA 0.2232 0.2472 0.2025
|
||
idf_sum_similarity dataset with_stopping + with_stemming + stop(punct) WikiQA 0.2236 0.2472 0.2025
|
||
|
||
idf_sum_similarity Corpus with_stopping + with_stemming TrecQA 0.7032 0.7281 0.672 IDF is computed over documents in disks1-5 and aquaint
|
||
idf_sum_similarity Corpus with_stopping + with_stemming WikiQA 0.2212 0.2461 0.2005 IDF is computed over documents in Wikipedia
|
||
|
||
sm_model NA no_idf_overlap_features TrecQA 0.597 0.6918 0.6445
|
||
sm_model Dataset with_corrected_idf_overlap_features TrecQA 0.919 0.7927 0.7384 S&M model uses stopping only
|
||
sm_model Dataset with_corrected_idf_overlap_features – stop(punct) TrecQA 0.8325 0.7948 0.7562 did not remove punctuation
|
||
sm_model Dataset with_stemmed_idf_overlap_features TrecQA 0.9414 0.8206 0.7363
|
||
sm_model Corpus with_corrected_idf_overlap_features TrecQA
|
||
sm_model Dataset with_fractional_idf_overlap_features TrecQA
|
||
sm_model Corpus with_fractional_idf_overlap_features TrecQA
|
||
sm_model NA no_idf_overlap_features WikiQA
|
||
sm_model Dataset with_corrected_idf_overlap_features WikiQA
|
||
sm_model Corpus with_corrected_idf_overlap_features WikiQA
|
||
sm_model Dataset with_fractional_idf_overlap_features WikiQA
|
||
sm_model Corpus with_fractional_idf_overlap_features WikiQA
|