Skip to content

Commit

Permalink
Upgrade to Lucene 7.6 (#514)
Browse files Browse the repository at this point in the history
  • Loading branch information
@lintool
lintool committed Dec 18, 2018
1 parent af0a479 commit e71df7a
Show file tree
Hide file tree
Showing 44 changed files with 719 additions and 1,177 deletions.
4 changes: 2 additions & 2 deletions docs/experiments-car17.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -63,11 +63,11 @@ With the above commands, you should be able to replicate the following results:

MAP | BM25 | BM25+RM3 | BM25+AX | QL | QL+RM3 | QL+AX |
:---------------------------------------|-----------|-----------|-----------|-----------|-----------|-----------|
All Topics | 0.1650 | 0.1343 | 0.1318 | 0.1515 | 0.1211 | 0.1083 |
All Topics | 0.1689 | 0.1386 | 0.1355 | 0.1516 | 0.1198 | 0.1082 |


RECIP_RANK | BM25 | BM25+RM3 | BM25+AX | QL | QL+RM3 | QL+AX |
:---------------------------------------|-----------|-----------|-----------|-----------|-----------|-----------|
All Topics | 0.2270 | 0.1852 | 0.1817 | 0.2085 | 0.1672 | 0.1503 |
All Topics | 0.2321 | 0.1907 | 0.1857 | 0.2085 | 0.1653 | 0.1501 |


4 changes: 2 additions & 2 deletions docs/experiments-core17.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -63,11 +63,11 @@ With the above commands, you should be able to replicate the following results:

MAP | BM25 | BM25+RM3 | BM25+AX | QL | QL+RM3 | QL+AX |
:---------------------------------------|-----------|-----------|-----------|-----------|-----------|-----------|
All Topics | 0.1996 | 0.2639 | 0.2719 | 0.1928 | 0.2427 | 0.2498 |
All Topics | 0.1977 | 0.2596 | 0.2700 | 0.1913 | 0.2405 | 0.2514 |


P30 | BM25 | BM25+RM3 | BM25+AX | QL | QL+RM3 | QL+AX |
:---------------------------------------|-----------|-----------|-----------|-----------|-----------|-----------|
All Topics | 0.4207 | 0.4880 | 0.4900 | 0.4327 | 0.4640 | 0.4813 |
All Topics | 0.4160 | 0.4820 | 0.4927 | 0.4373 | 0.4580 | 0.4827 |


4 changes: 2 additions & 2 deletions docs/experiments-core18.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -62,11 +62,11 @@ With the above commands, you should be able to replicate the following results:

MAP | BM25 | BM25+RM3 | BM25+AX | QL | QL+RM3 | QL+AX |
:---------------------------------------|-----------|-----------|-----------|-----------|-----------|-----------|
All Topics | 0.2487 | 0.2911 | 0.2919 | 0.2504 | 0.2754 | 0.2976 |
All Topics | 0.2491 | 0.2952 | 0.2921 | 0.2522 | 0.2759 | 0.2975 |


P30 | BM25 | BM25+RM3 | BM25+AX | QL | QL+RM3 | QL+AX |
:---------------------------------------|-----------|-----------|-----------|-----------|-----------|-----------|
All Topics | 0.3640 | 0.4087 | 0.4033 | 0.3620 | 0.3773 | 0.4067 |
All Topics | 0.3580 | 0.4200 | 0.4007 | 0.3627 | 0.3753 | 0.4073 |


24 changes: 12 additions & 12 deletions docs/experiments-cw09b.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -110,29 +110,29 @@ With the above commands, you should be able to replicate the following results:

MAP | BM25 | BM25+RM3 | BM25+AX | QL | QL+RM3 | QL+AX |
:---------------------------------------|-----------|-----------|-----------|-----------|-----------|-----------|
TREC 2010 Web Track: Topics 51-100 | 0.1094 | 0.1075 | 0.0966 | 0.1027 | 0.1060 | 0.1088 |
TREC 2011 Web Track: Topics 101-150 | 0.1095 | 0.1146 | 0.0996 | 0.0971 | 0.0961 | 0.0914 |
TREC 2012 Web Track: Topics 151-200 | 0.1072 | 0.1318 | 0.1242 | 0.1035 | 0.1132 | 0.1215 |
TREC 2010 Web Track: Topics 51-100 | 0.1126 | 0.1171 | 0.0928 | 0.1060 | 0.1117 | 0.1086 |
TREC 2011 Web Track: Topics 101-150 | 0.1094 | 0.1142 | 0.0974 | 0.0958 | 0.0964 | 0.0879 |
TREC 2012 Web Track: Topics 151-200 | 0.1106 | 0.1382 | 0.1315 | 0.1069 | 0.1167 | 0.1212 |


P30 | BM25 | BM25+RM3 | BM25+AX | QL | QL+RM3 | QL+AX |
:---------------------------------------|-----------|-----------|-----------|-----------|-----------|-----------|
TREC 2010 Web Track: Topics 51-100 | 0.2653 | 0.2604 | 0.2521 | 0.2417 | 0.2507 | 0.2618 |
TREC 2011 Web Track: Topics 101-150 | 0.2540 | 0.2713 | 0.2420 | 0.2220 | 0.2207 | 0.2267 |
TREC 2012 Web Track: Topics 151-200 | 0.2180 | 0.2387 | 0.2313 | 0.2013 | 0.2040 | 0.2100 |
TREC 2010 Web Track: Topics 51-100 | 0.2681 | 0.2819 | 0.2354 | 0.2431 | 0.2611 | 0.2618 |
TREC 2011 Web Track: Topics 101-150 | 0.2513 | 0.2700 | 0.2393 | 0.2147 | 0.2147 | 0.2167 |
TREC 2012 Web Track: Topics 151-200 | 0.2167 | 0.2473 | 0.2553 | 0.2080 | 0.2053 | 0.2140 |


NDCG20 | BM25 | BM25+RM3 | BM25+AX | QL | QL+RM3 | QL+AX |
:---------------------------------------|-----------|-----------|-----------|-----------|-----------|-----------|
TREC 2010 Web Track: Topics 51-100 | 0.1328 | 0.1423 | 0.1715 | 0.1132 | 0.1314 | 0.1470 |
TREC 2011 Web Track: Topics 101-150 | 0.1914 | 0.1861 | 0.1877 | 0.1635 | 0.1608 | 0.1592 |
TREC 2012 Web Track: Topics 151-200 | 0.0976 | 0.1308 | 0.1187 | 0.0862 | 0.1075 | 0.1055 |
TREC 2010 Web Track: Topics 51-100 | 0.1354 | 0.1545 | 0.1637 | 0.1143 | 0.1362 | 0.1454 |
TREC 2011 Web Track: Topics 101-150 | 0.1890 | 0.1823 | 0.1833 | 0.1619 | 0.1520 | 0.1509 |
TREC 2012 Web Track: Topics 151-200 | 0.1014 | 0.1329 | 0.1441 | 0.0868 | 0.1059 | 0.1030 |


ERR20 | BM25 | BM25+RM3 | BM25+AX | QL | QL+RM3 | QL+AX |
:---------------------------------------|-----------|-----------|-----------|-----------|-----------|-----------|
TREC 2010 Web Track: Topics 51-100 | 0.0717 | 0.0784 | 0.1007 | 0.0586 | 0.0649 | 0.0802 |
TREC 2011 Web Track: Topics 101-150 | 0.0947 | 0.1081 | 0.1064 | 0.0842 | 0.0921 | 0.0879 |
TREC 2012 Web Track: Topics 151-200 | 0.1382 | 0.2179 | 0.1921 | 0.1315 | 0.1574 | 0.1583 |
TREC 2010 Web Track: Topics 51-100 | 0.0733 | 0.0865 | 0.0981 | 0.0599 | 0.0649 | 0.0742 |
TREC 2011 Web Track: Topics 101-150 | 0.0959 | 0.1042 | 0.1091 | 0.0849 | 0.0865 | 0.0820 |
TREC 2012 Web Track: Topics 151-200 | 0.1304 | 0.2224 | 0.2355 | 0.1305 | 0.1475 | 0.1558 |


16 changes: 8 additions & 8 deletions docs/experiments-cw12.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -73,25 +73,25 @@ With the above commands, you should be able to replicate the following results:

MAP | BM25 | BM25+RM3 | QL | QL+RM3 |
:---------------------------------------|-----------|-----------|-----------|-----------|
[TREC 2013 Web Track: Topics 201-250](http://trec.nist.gov/data/web2013.html)| 0.1673 | 0.1489 | 0.1438 | 0.1235 |
[TREC 2014 Web Track: Topics 251-300](http://trec.nist.gov/data/web2014.html)| 0.2432 | 0.2468 | 0.2401 | 0.2331 |
[TREC 2013 Web Track: Topics 201-250](http://trec.nist.gov/data/web2013.html)| 0.1695 | 0.1498 | 0.1493 | 0.1280 |
[TREC 2014 Web Track: Topics 251-300](http://trec.nist.gov/data/web2014.html)| 0.2469 | 0.2496 | 0.2467 | 0.2383 |


P30 | BM25 | BM25+RM3 | QL | QL+RM3 |
:---------------------------------------|-----------|-----------|-----------|-----------|
[TREC 2013 Web Track: Topics 201-250](http://trec.nist.gov/data/web2013.html)| 0.2827 | 0.2347 | 0.2507 | 0.2047 |
[TREC 2014 Web Track: Topics 251-300](http://trec.nist.gov/data/web2014.html)| 0.4500 | 0.4200 | 0.4367 | 0.4013 |
[TREC 2013 Web Track: Topics 201-250](http://trec.nist.gov/data/web2013.html)| 0.2767 | 0.2407 | 0.2613 | 0.2207 |
[TREC 2014 Web Track: Topics 251-300](http://trec.nist.gov/data/web2014.html)| 0.4533 | 0.4180 | 0.4380 | 0.4107 |


NDCG20 | BM25 | BM25+RM3 | QL | QL+RM3 |
:---------------------------------------|-----------|-----------|-----------|-----------|
[TREC 2013 Web Track: Topics 201-250](http://trec.nist.gov/data/web2013.html)| 0.2066 | 0.1757 | 0.1905 | 0.1557 |
[TREC 2014 Web Track: Topics 251-300](http://trec.nist.gov/data/web2014.html)| 0.2646 | 0.2435 | 0.2327 | 0.2168 |
[TREC 2013 Web Track: Topics 201-250](http://trec.nist.gov/data/web2013.html)| 0.2086 | 0.1836 | 0.1993 | 0.1611 |
[TREC 2014 Web Track: Topics 251-300](http://trec.nist.gov/data/web2014.html)| 0.2578 | 0.2430 | 0.2228 | 0.2258 |


ERR20 | BM25 | BM25+RM3 | QL | QL+RM3 |
:---------------------------------------|-----------|-----------|-----------|-----------|
[TREC 2013 Web Track: Topics 201-250](http://trec.nist.gov/data/web2013.html)| 0.1213 | 0.0915 | 0.1169 | 0.0859 |
[TREC 2014 Web Track: Topics 251-300](http://trec.nist.gov/data/web2014.html)| 0.1737 | 0.1741 | 0.1451 | 0.1344 |
[TREC 2013 Web Track: Topics 201-250](http://trec.nist.gov/data/web2013.html)| 0.1284 | 0.0974 | 0.1232 | 0.0913 |
[TREC 2014 Web Track: Topics 251-300](http://trec.nist.gov/data/web2014.html)| 0.1630 | 0.1713 | 0.1321 | 0.1407 |


16 changes: 8 additions & 8 deletions docs/experiments-cw12b13.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -88,25 +88,25 @@ With the above commands, you should be able to replicate the following results:

MAP | BM25 | BM25+RM3 | BM25+AX | QL | QL+RM3 | QL+AX |
:---------------------------------------|-----------|-----------|-----------|-----------|-----------|-----------|
[TREC 2013 Web Track: Topics 201-250](http://trec.nist.gov/data/web2013.html)| 0.0457 | 0.0440 | 0.0411 | 0.0389 | 0.0314 | 0.0354 |
[TREC 2014 Web Track: Topics 251-300](http://trec.nist.gov/data/web2014.html)| 0.0219 | 0.0192 | 0.0177 | 0.0228 | 0.0202 | 0.0189 |
[TREC 2013 Web Track: Topics 201-250](http://trec.nist.gov/data/web2013.html)| 0.0468 | 0.0450 | 0.0435 | 0.0397 | 0.0319 | 0.0359 |
[TREC 2014 Web Track: Topics 251-300](http://trec.nist.gov/data/web2014.html)| 0.0224 | 0.0189 | 0.0180 | 0.0235 | 0.0205 | 0.0186 |


P30 | BM25 | BM25+RM3 | BM25+AX | QL | QL+RM3 | QL+AX |
:---------------------------------------|-----------|-----------|-----------|-----------|-----------|-----------|
[TREC 2013 Web Track: Topics 201-250](http://trec.nist.gov/data/web2013.html)| 0.2000 | 0.1767 | 0.1800 | 0.1720 | 0.1420 | 0.1513 |
[TREC 2014 Web Track: Topics 251-300](http://trec.nist.gov/data/web2014.html)| 0.1293 | 0.1113 | 0.1173 | 0.1313 | 0.1160 | 0.1180 |
[TREC 2013 Web Track: Topics 201-250](http://trec.nist.gov/data/web2013.html)| 0.2113 | 0.1787 | 0.1840 | 0.1767 | 0.1373 | 0.1513 |
[TREC 2014 Web Track: Topics 251-300](http://trec.nist.gov/data/web2014.html)| 0.1273 | 0.1133 | 0.1107 | 0.1373 | 0.1173 | 0.1167 |


NDCG20 | BM25 | BM25+RM3 | BM25+AX | QL | QL+RM3 | QL+AX |
:---------------------------------------|-----------|-----------|-----------|-----------|-----------|-----------|
[TREC 2013 Web Track: Topics 201-250](http://trec.nist.gov/data/web2013.html)| 0.1242 | 0.1197 | 0.1245 | 0.1158 | 0.0852 | 0.1117 |
[TREC 2014 Web Track: Topics 251-300](http://trec.nist.gov/data/web2014.html)| 0.1190 | 0.1002 | 0.0969 | 0.1133 | 0.0959 | 0.0999 |
[TREC 2013 Web Track: Topics 201-250](http://trec.nist.gov/data/web2013.html)| 0.1286 | 0.1228 | 0.1287 | 0.1107 | 0.0880 | 0.1143 |
[TREC 2014 Web Track: Topics 251-300](http://trec.nist.gov/data/web2014.html)| 0.1185 | 0.1012 | 0.0964 | 0.1177 | 0.1024 | 0.1001 |


ERR20 | BM25 | BM25+RM3 | BM25+AX | QL | QL+RM3 | QL+AX |
:---------------------------------------|-----------|-----------|-----------|-----------|-----------|-----------|
[TREC 2013 Web Track: Topics 201-250](http://trec.nist.gov/data/web2013.html)| 0.0821 | 0.0777 | 0.0915 | 0.0764 | 0.0511 | 0.0705 |
[TREC 2014 Web Track: Topics 251-300](http://trec.nist.gov/data/web2014.html)| 0.1237 | 0.1014 | 0.0959 | 0.1040 | 0.0911 | 0.0994 |
[TREC 2013 Web Track: Topics 201-250](http://trec.nist.gov/data/web2013.html)| 0.0838 | 0.0879 | 0.0943 | 0.0769 | 0.0568 | 0.0780 |
[TREC 2014 Web Track: Topics 251-300](http://trec.nist.gov/data/web2014.html)| 0.1201 | 0.1039 | 0.0929 | 0.1091 | 0.1036 | 0.0896 |


12 changes: 6 additions & 6 deletions docs/experiments-disk12.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -90,15 +90,15 @@ With the above commands, you should be able to replicate the following results:

MAP | BM25 | BM25+RM3 | BM25+AX | QL | QL+RM3 | QL+AX |
:---------------------------------------|-----------|-----------|-----------|-----------|-----------|-----------|
TREC-1 Ad Hoc Track: Topics 51-100 | 0.2254 | 0.2607 | 0.2675 | 0.2188 | 0.2500 | 0.2519 |
TREC-2 Ad Hoc Track: Topics 101-150 | 0.2003 | 0.2579 | 0.2708 | 0.2013 | 0.2475 | 0.2606 |
TREC-3 Ad Hoc Track: Topics 151-200 | 0.2571 | 0.3224 | 0.3349 | 0.2530 | 0.3019 | 0.3113 |
TREC-1 Ad Hoc Track: Topics 51-100 | 0.2273 | 0.2617 | 0.2640 | 0.2189 | 0.2478 | 0.2501 |
TREC-2 Ad Hoc Track: Topics 101-150 | 0.2010 | 0.2600 | 0.2722 | 0.2015 | 0.2485 | 0.2593 |
TREC-3 Ad Hoc Track: Topics 151-200 | 0.2580 | 0.3227 | 0.3318 | 0.2518 | 0.2996 | 0.3103 |


P30 | BM25 | BM25+RM3 | BM25+AX | QL | QL+RM3 | QL+AX |
:---------------------------------------|-----------|-----------|-----------|-----------|-----------|-----------|
TREC-1 Ad Hoc Track: Topics 51-100 | 0.4493 | 0.4813 | 0.5167 | 0.4453 | 0.4687 | 0.4967 |
TREC-2 Ad Hoc Track: Topics 101-150 | 0.4213 | 0.4580 | 0.4787 | 0.4153 | 0.4427 | 0.4660 |
TREC-3 Ad Hoc Track: Topics 151-200 | 0.4740 | 0.5100 | 0.5160 | 0.4647 | 0.5013 | 0.5160 |
TREC-1 Ad Hoc Track: Topics 51-100 | 0.4533 | 0.4867 | 0.5067 | 0.4520 | 0.4653 | 0.4953 |
TREC-2 Ad Hoc Track: Topics 101-150 | 0.4280 | 0.4580 | 0.4753 | 0.4207 | 0.4453 | 0.4740 |
TREC-3 Ad Hoc Track: Topics 151-200 | 0.4740 | 0.5040 | 0.5100 | 0.4580 | 0.4933 | 0.5167 |


12 changes: 6 additions & 6 deletions docs/experiments-gov2.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -90,15 +90,15 @@ With the above commands, you should be able to replicate the following results:

MAP | BM25 | BM25+RM3 | BM25+AX | QL | QL+RM3 | QL+AX |
:---------------------------------------|-----------|-----------|-----------|-----------|-----------|-----------|
[TREC 2004 Terabyte Track: Topics 701-750](http://trec.nist.gov/data/terabyte04.html)| 0.2673 | 0.2974 | 0.2735 | 0.2636 | 0.2770 | 0.2638 |
[TREC 2005 Terabyte Track: Topics 751-800](http://trec.nist.gov/data/terabyte05.html)| 0.3366 | 0.3846 | 0.3669 | 0.3264 | 0.3610 | 0.3670 |
[TREC 2006 Terabyte Track: Topics 801-850](http://trec.nist.gov/data/terabyte06.html)| 0.3055 | 0.3438 | 0.3061 | 0.2957 | 0.3160 | 0.3112 |
[TREC 2004 Terabyte Track: Topics 701-750](http://trec.nist.gov/data/terabyte04.html)| 0.2689 | 0.2943 | 0.2665 | 0.2681 | 0.2806 | 0.2666 |
[TREC 2005 Terabyte Track: Topics 751-800](http://trec.nist.gov/data/terabyte05.html)| 0.3390 | 0.3800 | 0.3664 | 0.3303 | 0.3628 | 0.3646 |
[TREC 2006 Terabyte Track: Topics 801-850](http://trec.nist.gov/data/terabyte06.html)| 0.3080 | 0.3356 | 0.3069 | 0.2996 | 0.3173 | 0.3084 |


P30 | BM25 | BM25+RM3 | BM25+AX | QL | QL+RM3 | QL+AX |
:---------------------------------------|-----------|-----------|-----------|-----------|-----------|-----------|
[TREC 2004 Terabyte Track: Topics 701-750](http://trec.nist.gov/data/terabyte04.html)| 0.4837 | 0.5347 | 0.5082 | 0.4667 | 0.4878 | 0.4837 |
[TREC 2005 Terabyte Track: Topics 751-800](http://trec.nist.gov/data/terabyte05.html)| 0.5520 | 0.5960 | 0.5947 | 0.5160 | 0.5673 | 0.5880 |
[TREC 2006 Terabyte Track: Topics 801-850](http://trec.nist.gov/data/terabyte06.html)| 0.4900 | 0.5227 | 0.5007 | 0.4753 | 0.4853 | 0.5007 |
[TREC 2004 Terabyte Track: Topics 701-750](http://trec.nist.gov/data/terabyte04.html)| 0.4864 | 0.5313 | 0.4986 | 0.4755 | 0.4952 | 0.4932 |
[TREC 2005 Terabyte Track: Topics 751-800](http://trec.nist.gov/data/terabyte05.html)| 0.5540 | 0.5873 | 0.5933 | 0.5347 | 0.5720 | 0.5840 |
[TREC 2006 Terabyte Track: Topics 801-850](http://trec.nist.gov/data/terabyte06.html)| 0.4907 | 0.5160 | 0.5033 | 0.4720 | 0.4773 | 0.4920 |


0 comments on commit e71df7a

Please sign in to comment.