Oracle

The Oracle provides ground-truth labels for candidates. It can operate in two modes: offline (using a dataset to query labels) or online (using a model to generate labels, simulating real experiments). The oracle is used during the active learning loop to evaluate selected candidates and provide feedback for training the surrogate model.

class alf_core.oracle.oracle.Oracle(scorer)

Bases: object

Oracle model is used to evaluate new candidates proposed by the search/optimiser process. For offline optimization tasks, the oracle is the dataset. For online optimization tasks, the oracle is a model.

evaluate(candidates, state)

Evaluate candidates and return their labels.

Parameters:

• candidates (list[Candidate]) – List of Candidate objects to evaluate.

• state (State) – Current task state (updated with evaluation time).

Returns:

• Candidates paired with their evaluated labels

• Updated state with oracle_time metric

Return type:

tuple[LabelledCandidates, State]

get_metrics()

Get metrics from the underlying scorer if available.

Return type:

dict[str, Union[float, int, number]]

Returns:

Dictionary of metric names to values. Returns empty dict if the module doesn’t provide metrics.