Splitting Utils

The alf_core.dataset.splitting_utils module provides utility functions for splitting datasets into train, validation, test, and candidate pool sets. It supports three splitting strategies: random splitting (split_type="random"), low-vs-high splitting that separates top and bottom performers by label value (split_type="low_vs_high"), and stratified splitting that preserves class proportions across splits for classification tasks (split_type="stratified"). Stratified splitting is only valid with ProblemType.BINARY or ProblemType.MULTICLASS.

alf_core.dataset.splitting_utils.split_dataset(split_type, dataset, train_size, validation_size, test_size, candidate_pool_size, seed)[source]

Split dataset into train, validation, test, and candidate pool.

Parameters:

  • split_type (Literal['random', 'low_vs_high', 'stratified']) – Type of split to perform (“random”, “low_vs_high”, or “stratified”).

  • dataset (LabelledCandidates) – Dataset to split.

  • train_size (int) – Number of samples for training set.

  • validation_size (int) – Number of samples for validation set.

  • test_size (int) – Number of samples for test set.

  • candidate_pool_size (int) – Number of samples for candidate pool.

  • seed (int) – Random seed for reproducibility.

Return type:

dict[str, LabelledCandidates]

Returns:

Dictionary with keys “train”, “validation”, “test”, and “candidate_pool”.

Raises:

ValueError – If split_type is not one of the supported split types.

alf_core.dataset.splitting_utils.split_low_vs_high(dataset, train_size, validation_size, test_size, candidate_pool_size, seed)[source]

Split dataset with low-scoring candidates in train/val, high-scoring in test/pool.

Sorts candidates by label value, assigns low-scoring candidates to train/validation and high-scoring candidates to test/candidate_pool. Within each group, candidates are randomly shuffled.

Parameters:

  • dataset (LabelledCandidates) – Dataset to split.

  • train_size (int) – Number of samples for training set (from low-scoring group).

  • validation_size (int) – Number of samples for validation set (from low-scoring group).

  • test_size (int) – Number of samples for test set (from high-scoring group).

  • candidate_pool_size (int) – Number of samples for candidate pool (from high-scoring group).

  • seed (int) – Random seed for shuffling within groups.

Return type:

dict[str, LabelledCandidates]

Returns:

Dictionary with keys “train”, “validation”, “test”, and “candidate_pool”.

alf_core.dataset.splitting_utils.split_random(dataset, train_size, validation_size, test_size, candidate_pool_size, seed)[source]

Split dataset randomly into train, validation, test, and candidate pool.

Shuffles the dataset and splits it sequentially into the specified sizes. Remaining samples go to the candidate pool.

Parameters:

  • dataset (LabelledCandidates) – Dataset to split.

  • train_size (int) – Number of samples for training set.

  • validation_size (int) – Number of samples for validation set.

  • test_size (int) – Number of samples for test set.

  • candidate_pool_size (int) – Number of samples for candidate pool.

  • seed (int) – Random seed for shuffling.

Return type:

dict[str, LabelledCandidates]

Returns:

Dictionary with keys “train”, “validation”, “test”, and “candidate_pool”.

alf_core.dataset.splitting_utils.split_stratified(dataset, train_size, validation_size, test_size, candidate_pool_size, seed)[source]

Split dataset preserving class proportions across all splits.

For each class in the dataset, samples are proportionally distributed across train, validation, test, and candidate pool splits. Within each class the samples are randomly shuffled before assignment.

Parameters:

  • dataset (LabelledCandidates) – Dataset to split. Labels must be integer class indices.

  • train_size (int) – Total number of samples for training set.

  • validation_size (int) – Total number of samples for validation set.

  • test_size (int) – Total number of samples for test set.

  • candidate_pool_size (int) – Total number of samples for candidate pool.

  • seed (int) – Random seed for reproducibility.

Return type:

dict[str, LabelledCandidates]

Returns:

Dictionary with keys “train”, “validation”, “test”, and “candidate_pool”.

Raises:

ValueError – If any split requests more samples than remain after earlier splits.