Base Dataset

The BaseDataset class is an abstract base class that defines the interface for all datasets in the framework. It manages data loading through the abstract load_dataset() method, splits data into train/validation/test/candidate_pool sets, updates splits with newly acquired candidates, and provides labels for candidates from the original dataset (used by the oracle in offline settings).

BaseDatasetConfig requires a problem_type field (ProblemType.REGRESSION, ProblemType.BINARY, or ProblemType.MULTICLASS) that drives metric selection, model output dimensionality, and split strategy validation. Note that split_type="stratified" is only valid for classification tasks and will raise a validation error if used with ProblemType.REGRESSION.

class alf_core.dataset.base_dataset.BaseDataset(config)

Bases: ABC

Base class for all datasets.

property candidate_pool: LabelledCandidates

Get the candidate pool split.

Returns:

Candidate pool available for acquisition.

Raises:

AssertionError – If dataset hasn’t been split yet.

determine_num_classes()

Determine the number of output neurons based on problem type and labels.

Raises:

• RuntimeError – If the raw dataset is not loaded.

• ValueError – If the dataset is empty (no labels found).

Returns:

Number of output neurons. For REGRESSION and BINARY, returns 1 and 2 respectively.

Return type:

int

get_metrics()

Get summary metrics for all dataset splits.

Returns:

• “num_{split}”: Number of samples in each split

• ”{split}_mean”: Mean label value for each split

Return type:

dict[str, Union[float, int, number]]

abstractmethod load_dataset()

Load the raw dataset.

This method must be implemented by subclasses to load data from their specific source.

Return type:

LabelledCandidates

Returns:

The loaded dataset with candidates and labels.

query(candidates)

Query labels for the given candidates from the raw dataset.

Parameters:

candidates (list[Candidate]) – List of Candidate objects to query.

Return type:

LabelledCandidates

Returns:

Candidates paired with their labels from the dataset.

Raises:

• AssertionError – If dataset hasn’t been loaded yet.

• ValueError – If any candidate’s data is not found in the dataset.

property raw_dataset: LabelledCandidates

Get the full labelled dataset before splitting.

Returns:

The complete labelled dataset loaded by load_dataset.

Raises:

AssertionError – If the dataset has not been set up yet.

save_splits(output_path)

Save the dataset splits to a file.

Parameters:

output_path (str | PathLike) – Path to the directory to save the dataset splits to

Return type:

None

set_metadata()

Set metadata for the dataset.

Subclasses can override this method to compute and store dataset-specific metadata. Called automatically during setup().

Return type:

None

setup()

Setup the dataset by loading and splitting it.

Loads the raw dataset, splits it according to the split configuration, and sets metadata. This must be called before accessing dataset splits.

Return type:

None

property test_dataset: LabelledCandidates

Get the test dataset split.

Returns:

Test dataset.

Raises:

AssertionError – If dataset hasn’t been split yet.

property train_dataset: LabelledCandidates

Get the training dataset split.

Returns:

Training dataset.

Raises:

AssertionError – If dataset hasn’t been split yet.

update_splits(acquired_candidates)

Update dataset splits with newly acquired candidates.

Removes acquired candidates from the candidate pool and distributes them between train and validation splits according to the split ratio.

Parameters:

acquired_candidates (LabelledCandidates) – Newly acquired candidates with labels.

Return type:

None

property validation_dataset: LabelledCandidates

Get the validation dataset split.

Returns:

Validation dataset.

Raises:

AssertionError – If dataset hasn’t been split yet.

class alf_core.dataset.base_dataset.BaseDatasetConfig(**data)

Bases: BaseModel

Configuration for BaseDataset.

name

Name identifier for the dataset.

modality

Data modality (validated against Modality enum).

seed

Random seed for reproducibility.

train_ratio

Fraction of data for training (0-1).

validation_frac

Fraction of training data held out for validation (0-1).

test_ratio

Fraction of data for testing (0-1).

split_type

Type of split.

max_candidate_pool

Optional maximum size of the candidate pool.

max_candidate_pool: int | None

modality: Modality

model_config: ClassVar[ConfigDict] = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

name: str

problem_type: ProblemType

seed: int

split_type: Literal['random', 'low_vs_high', 'stratified']

test_ratio: Annotated[float, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=0), Le(le=1)])]

train_ratio: Annotated[float, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=0), Le(le=1)])]

validate_config()

Validate dataset configuration.

Return type:

Self

Returns:

The validated configuration instance.

Raises:

• ValueError – If train_ratio + test_ratio exceeds 1.

• ValueError – If split_type is “stratified” with a REGRESSION problem_type.

validation_frac: Annotated[float, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=0), Le(le=1)])]