Zero-Shot Task

The ZeroShotTask evaluates pre-trained models without additional training. It uses the model to make predictions on test data and computes evaluation metrics. This task is useful for evaluating oracle models or pre-trained surrogate models without fine-tuning.

class alf_core.tasks.zeroshot_task.ZeroShotTask(**kwargs)

Bases: BaseTask

Zero-shot evaluation task for pre-trained models.

Evaluates a pre-trained surrogate model on the test set without any training or fine-tuning.

run(state, state_loggers)

Run the zero-shot evaluation task.

Evaluates the pre-trained surrogate model on the test set without any training. Saves predictions and logs metrics.

Parameters:

• state (State) – Task state with dataset and pre-trained surrogate model.

• state_loggers (list[StateLogger]) – List of StateLogger for recording the state.

Return type:

None