Model confidence metrics

Model confidence metrics in Refiner functions and Refiner UDFs indicate how confident the system is about the information it extracts.

Understanding the model’s confidence in its predictions can help you prioritize refining data sources, or take measures to validate output that doesn’t meet required confidence levels.

The Value object includes integrated model confidence metrics.

Data types

WordConfidence – A dictionary type capturing the actual word (val) and its corresponding model confidence (confidence).

class WordConfidence(TypedDict):
    val: str       # the word
    confidence: float # corresponding model confidence

ValConfidence – A universal type encapsulating model confidence metrics across all value objects. For complex data structures, this type mirrors the object’s structure but replaces the values with their respective confidence metrics.
```
ValConfidence = Union[List[WordConfidence], List[ValConfidence], Dict[str, ValConfidence]]
```

Functions

get_model_confs() – Retrieves model confidence metrics for each word. For complex data types, it mirrors the object’s structure.
```
def get_model_confs() -> ValConfidence
```
get_avg_model_conf() – Computes the average model confidence score. Returns None if there are no model confidence metrics.
```
def get_avg_model_conf() -> float | None
```
get_max_model_conf() – Identifies the word with the highest confidence score.
```
def get_max_model_conf() -> WordConfidence | None
```
get_min_model_conf() – Identifies the word with the lowest confidence score.
```
def get_min_model_conf() -> WordConfidence | None
```