Translator Confidence Score

Synonyms

g-score

Definition

Definition: The Confidence Score combines the scores assigned to a given result by multiple reasoning agents, with an expected range of [0,1] for each score. The Confidence Score invokes an algorithm that implements the following logic: let score(ara[i]) represent the score returned by ARA i for a given answer; the Confidence Score is then given by 1 - \product_i (1 - score(ara[i])). In natural language, the algorithm treats 1 - score(ara[i]) as the "doubt" that ARA i had in the answer and then considers each ARA as independent from the others, so it multiplies these "doubts" together before subtracting from 1 to return a final Confidence Score. This is essentially equivalent to geometric (or logarithmic) pooling with constant weights as described here.