Castor/baseline_results.tsv at lintool-patch-3

mirror of https://github.com/wassname/Castor.git synced 2026-09-09 11:13:20 +08:00

Files

T

gauravbaruah a67e2d12c4 Ext feats bug fix (#19 )

+ sm model no external features baseline
+ sm model with IDF weights
+ sm model with IDF weights without removing punctuation --> barely better than df/idf (a la Pytorch).
+ sm model with stemming before computing IDF weights
^ all on the TrecQA dataset

2017-04-18 12:32:43 -04:00

1.9 KiB

Raw Permalink Blame History

1	Baseline_method	idf	condition	dataset	train_MAP	dev_MAP	test_MAP	Notes/Comments
2	idf_sum_similarity	dataset	no_stopping + no_stemming	TrecQA	0.7744	0.7668	0.7014	IDF is computed over terms in the train, dev and test sets
3	idf_sum_similarity	dataset	with_stopping + with_stemming	TrecQA	0.7078	0.7404	0.6816
4	idf_sum_similarity	dataset	with_stopping + with_stemming + stop(punct)	TrecQA	0.7074	0.7337	0.6805	also stopping punctuation
5	idf_sum_similarity	dataset	no_stopping + no_stemming	WikiQA	0.2429	0.2489	0.2219	There is less overlap between questions and answers in the WikiQA dataset
6	idf_sum_similarity	dataset	with_stopping + with_stemming	WikiQA	0.2232	0.2472	0.2025
7	idf_sum_similarity	dataset	with_stopping + with_stemming + stop(punct)	WikiQA	0.2236	0.2472	0.2025
8
9	idf_sum_similarity	Corpus	with_stopping + with_stemming	TrecQA	0.7032	0.7281	0.672	IDF is computed over documents in disks1-5 and aquaint
10	idf_sum_similarity	Corpus	with_stopping + with_stemming	WikiQA	0.2212	0.2461	0.2005	IDF is computed over documents in Wikipedia
11
12	sm_model	NA	no_idf_overlap_features	TrecQA	0.597	0.6918	0.6445
13	sm_model	Dataset	with_corrected_idf_overlap_features	TrecQA	0.919	0.7927	0.7384	S&M model uses stopping only
14	sm_model	Dataset	with_corrected_idf_overlap_features – stop(punct)	TrecQA	0.8325	0.7948	0.7562	did not remove punctuation
15	sm_model	Dataset	with_stemmed_idf_overlap_features	TrecQA	0.9414	0.8206	0.7363
16	sm_model	Corpus	with_corrected_idf_overlap_features	TrecQA
17	sm_model	Dataset	with_fractional_idf_overlap_features	TrecQA
18	sm_model	Corpus	with_fractional_idf_overlap_features	TrecQA
19	sm_model	NA	no_idf_overlap_features	WikiQA
20	sm_model	Dataset	with_corrected_idf_overlap_features	WikiQA
21	sm_model	Corpus	with_corrected_idf_overlap_features	WikiQA
22	sm_model	Dataset	with_fractional_idf_overlap_features	WikiQA
23	sm_model	Corpus	with_fractional_idf_overlap_features	WikiQA

1.9 KiB Raw Permalink Blame History

1.9 KiB

Raw Permalink Blame History