Design Task

The DesignTask runs a multi-round active learning optimisation loop. In each round, the surrogate model is trained on the current training data, the optimiser selects promising candidates, the oracle evaluates them, and the newly labelled candidates are added to the training set. This iterative process continues for a specified number of acquisition rounds.

After all rounds complete, DesignTask computes summary metrics — which includes auc_top_k, the normalised area under the per-round top-k mean curve — and emits them as a experiment_summary log entry. This gives a single sample-efficiency score for the full experiment. For regression tasks the curve is the top_k_mean of all candidates acquired so far; for classification tasks it is the per-round test-set accuracy. The summary is skipped silently when fewer than two rounds produce a valid value.

class alf_core.tasks.design_task.DesignTask(**kwargs)

Bases: BaseTask

Multi-round design task for iteratively optimising the surrogate model. Performs multiple rounds of candidate acquisition, evaluation, and model training.

run(state, state_loggers, optimizer, oracle)

Run the multi-round design task.

Executes multiple rounds of active learning: 1. Optimizer proposes candidates (ask) 2. Oracle evaluates candidates 3. Surrogate model is updated with new data (tell) 4. Model is evaluated on test set 5. Metrics are logged

The loop continues for num_acq_rounds or until termination conditions are met.

After all rounds complete, logs end-of-experiment aggregate metrics under the round name experiment_summary.

Parameters:

• state (State) – Initial task state with dataset and surrogate.

• state_loggers (list[StateLogger]) – List of StateLogger for recording the state.

• optimizer (Optimizer) – Optimizer for candidate acquisition.

• oracle (Oracle) – Oracle for evaluating candidate labels.

Return type:

None

run_initial_train_round(state, state_loggers)

Run the initial train round on the train and validation sets.

Parameters:

• state (State) – Task state with dataset and surrogate.

• state_loggers (list[StateLogger]) – List of StateLogger for recording the state.

Return type:

State

Returns:

Updated state with surrogate fine-tuned on the train and validation sets.