Skip to content

capreolus-ir/trecrun

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

31 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

License Worfklow PyPI version fury.io Code style: black

TRECRun

TRECRun is a library for working with TREC run files, with an API heavily inspired by PyTerrier's pipeline operators.

API Operator Description
TRECRun(results) Create a TRECRun object from a dictionary of results or a path to a run file in TREC format.
add(self, other), subtract, multiply, divide +, -, *, / Perform the given operation between self's document scores and other, which can be a TRECRun or a scalar.
topk(self, k) % Retain only the top-k documents for each qid after sorting by score.
intersect(self, other) & Retain only the queries and documents that appear in both self and other.
concat(self, other) Concat the documents in other and self, with those in other appearing at the end. Their scores will be modified to accomplish this.
normalize(self, method='rr') Normalize scores in self using RRF (rr), sklearn's min-max scaling (minmax), or sklearn's scaling (standard).
write_trec_run(self, outf) Write self to outfn in TREC format.
evaluate(self, qrels, metrics) Compute metrics for self using qrels and return a dict mapping metric names to their values for each QID. Metrics are computed by ir_measures.