-
Notifications
You must be signed in to change notification settings - Fork 2
Translator Confidence Score
g-score
Definition: The Confidence Score combines the scores assigned to a given result by multiple reasoning agents, with an expected range of [0,1]
for each score. The Confidence Score invokes an algorithm that implements the following logic: let score(ara[i])
represent the score returned by ARA i for a given answer; the Confidence Score is then given by 1 - \product_i (1 - score(ara[i]))
. In natural language, the algorithm treats 1 - score(ara[i])
as the "doubt" that ARA i had in the answer and then considers each ARA as independent from the others, so it multiplies these "doubts" together before subtracting from 1 to return a final Confidence Score. This is essentially equivalent to geometric (or logarithmic) pooling with constant weights as described here.