Model Utilities

Shared utilities for ALF model implementations.

transform_data (in alf_tools.models.utils.data_utils) applies input normalisation and output standardisation to training data in a single call, returning the transformed tensors and fitted normaliser objects for later use at inference time.

BotorchModelWrapper bridges ALF’s BaseModel interface and native BoTorch models to BoTorch acquisition functions, automatically routing through either the BoTorch posterior method or ALF’s predict() depending on the model type.

build_from_target (in alf_tools.models.utils.config_utils) instantiates any GPyTorch prior or constraint from a serialisable _target_ config dict, enabling fully configuration-driven kernel setup.

The alf_tools.models.utils.botorch_utils module provides conversion utilities between ALF’s data structures (Candidate, Predictions) and BoTorch’s expected formats (tensors, posteriors).

Data transformation utilities for ALF model implementations.

alf_tools.models.utils.data_utils.transform_data(train_x, labels, normalise_inputs_strategy, standardise_outputs, label_dtype, device, feature_dtype=None)[source]

Apply input normalisation and output standardisation to training data.

Parameters:

  • train_x (Tensor) – Training features tensor.

  • labels (ndarray) – Training labels array.

  • normalise_inputs_strategy (Optional[Literal['minmax', 'zscore']]) – Which input normalisation to apply: minmax, zscore, or None to disable input normalisation.

  • standardise_outputs (bool) – Whether to apply Z-score standardisation to output labels.

  • label_dtype (dtype) – Data type for the output labels.

  • device (device) – Device to move tensors to.

  • feature_dtype (dtype | None) – Data type for the input features. None (the default) preserves the input dtype of train_x, except in the normalised branch where the numpy round-trip casts to float32.

Return type:

tuple[Tensor, Tensor, InputNormaliser | InputStandardiser | None, OutputStandardiser | None]

Returns:

Transformed training features, training labels, and the fitted normalisers.

Universal wrapper for models to work with BoTorch acquisition functions.

This module provides a unified interface that allows both native BoTorch models and ALF BaseModel instances to work seamlessly with BoTorch acquisition functions. This essentially wraps the ALF models to be used as BoTorch models

class alf_tools.models.utils.botorch_model_wrapper.BotorchModelWrapper(model)[source]

Bases: Model

Universal wrapper for BoTorch acquisition functions compatibility.

Accepts either a native BoTorch Model (direct pass-through to posterior()) or an ALF BaseModel (adapts predict() to BoTorch’s posterior() interface). The wrapper detects the model type automatically.

ALF models split into two posterior paths:

Joint-capable ALF models that expose a trained botorch_model attribute (e.g. GPModel) delegate posterior() to the inner BoTorch model’s native posterior(). This yields a true joint covariance and fully honours observation_noise, posterior_transform and output_indices, all of which are forwarded to the inner model. q-based acquisitions such as qLogNoisyExpectedImprovement are therefore evaluated with the exact joint covariance, and num_outputs/batch_shape are delegated to the underlying BoTorch model.

Marginal-only (predict-only) ALF models route posterior() through predict(), which yields a diagonal (per-point independent) posterior. q-based acquisitions over q>1 are evaluated under a per-point independence approximation — baseline-candidate correlations are zero — rather than with the exact joint covariance. On this path the observation_noise flag is ignored: the returned posterior always carries whatever variance predict() reports. posterior_transform and output_indices cannot be honoured and raise NotImplementedError if passed. Such models report the single-output defaults num_outputs=1 and batch_shape=torch.Size([]).

Models that don’t provide prediction variances (e.g., CNNModel, deterministic models) cannot be used with BoTorch acquisition functions. Expected Improvement and Upper Confidence Bound require uncertainty estimates; attempting to use such models raises a ValueError.

Parameters:

model (Model | BaseModel) – Either a BoTorch BotorchModel or an ALF BaseModel instance. If BaseModel, it must provide prediction variances.

Raises:

  • TypeError – If the model is neither a BoTorch Model nor ALF BaseModel.

  • ValueError – If a BaseModel doesn’t provide variances in predictions (raised during posterior()).

property batch_shape: Size

The batch shape of the model.

Returns:

Batch shape for native BoTorch models, or for ALF models exposing a trained botorch_model (delegated to it). Returns an empty batch shape for marginal-only ALF models.

property num_outputs: int

The number of outputs of the model.

Returns:

Number of outputs for native BoTorch models, or for ALF models exposing a trained botorch_model (delegated to it). Returns 1 for marginal-only ALF models (single-output predict()).

posterior(X, output_indices=None, observation_noise=False, posterior_transform=None)[source]

Compute the posterior distribution at input points.

Parameters:

  • X (Tensor) – Input tensor of shape (batch_size, q, d) or (batch_size, d) where d is the input dimension and q is number of points.

  • output_indices (list[int] | None) – Optional list of output indices for multi-output models. Not supported for ALF models.

  • observation_noise (bool | Tensor) – Whether to include observation noise in predictions. Can be bool or Tensor for observed noise. Ignored for ALF models — predict() decides the variance semantics (noise-inclusive for GPModel).

  • posterior_transform (PosteriorTransform | None) – Optional posterior transformation. Not supported for ALF models.

Return type:

Posterior

Returns:

Posterior distribution with mean and variance at input points.

Raises:

  • ValueError – If ALF BaseModel doesn’t provide variances.

  • NotImplementedError – If posterior_transform or output_indices is passed for an ALF BaseModel — predict()-based posteriors cannot apply them, and ignoring them silently would corrupt acquisition values (e.g. a minimisation transform would be dropped).

property provides_joint_posterior: bool

Whether this model can produce a true joint posterior over q>1 points.

Returns:

True for native BoTorch models and for ALF models exposing a trained botorch_model; False for marginal-only predict()-based models (and for untrained models, which cannot yet supply one).

alf_tools.models.utils.botorch_model_wrapper.resolve_botorch_model(model)[source]

Return the joint-posterior BoTorch Model a surrogate provides, if any.

This is the single capability check used across the BoTorch integration: native BoTorch Model instances are returned as-is; ALF BaseModel instances are expected to expose a trained botorch_model attribute (the documented joint-posterior contract), which is returned when available.

Parameters:

model (Model | BaseModel) – A native BoTorch Model or an ALF BaseModel.

Return type:

Model | None

Returns:

The joint-posterior BoTorch model, or None for marginal-only models — those without a botorch_model, or whose botorch_model is unavailable because the model is untrained.

Config instantiation utilities for configurations with _target_ keys.

alf_tools.models.utils.config_utils.build_from_target(cfg, expected_base=None)[source]

Instantiate a GPyTorch object from a _target_ config dict.

Supports any GPyTorch prior or constraint. The dict must contain a _target_ key with a fully-qualified class path; all other keys are passed as constructor kwargs.

Parameters:

  • cfg (dict | None) – Dict with _target_ (e.g. "gpytorch.priors.LogNormalPrior") and any constructor kwargs. None returns None.

  • expected_base (type | tuple[type, ...] | None) – Base class (or tuple of base classes) the resolved target must subclass. When None, the target is validated against the union of gpytorch.priors.Prior and gpytorch.constraints.Interval.

Raises:

ValueError – If _target_ is missing, not fully qualified, not in the allowed module list, names a private attribute (leading underscore), does not exist in the module, does not resolve to a class, does not subclass expected_base, or if the constructor rejects the provided kwargs.

Return type:

object | None

Returns:

Instantiated object, or None if cfg is None.

Example:

prior = build_from_target({
    "_target_": "gpytorch.priors.GammaPrior",
    "concentration": 3.0,
    "rate": 6.0,
})

Utility functions for BoTorch integration with ALF.

This module provides conversion utilities between ALF’s data structures (Candidates, Predictions) and BoTorch’s expected formats (tensors, posteriors).

alf_tools.models.utils.botorch_utils.candidates_to_tensor(candidates, device=None, dtype=torch.float64)[source]

Convert a list of ALF Candidates to a PyTorch tensor.

This function extracts the data from each candidate and stacks them into a single tensor suitable for BoTorch operations.

Parameters:

  • candidates (list[Candidate]) – List of Candidate objects. Each candidate’s data should be a numpy array or torch tensor of shape (d,) where d is the feature dimension.

  • device (device | None) – Optional device to place the tensor on. If None, uses CPU.

  • dtype (dtype) – Desired data type of the output tensor. Default is torch.float64.

Return type:

Tensor

Returns:

Tensor of shape (n, d) where n is the number of candidates and d is the feature dimension.

Raises:

ValueError – If candidates have inconsistent shapes or unsupported data types.

Example

>>> candidates = [
...     Candidate(data=np.array([1.0, 2.0]), modality=Modality.TABULAR),
...     Candidate(data=np.array([3.0, 4.0]), modality=Modality.TABULAR)
... ]
>>> X = candidates_to_tensor(candidates)
>>> print(X.shape)
torch.Size([2, 2])
alf_tools.models.utils.botorch_utils.get_bounds_tensor(bounds, device=None, dtype=torch.float64)[source]

Convert bounds to BoTorch format.

BoTorch expects bounds as a tensor of shape (2, d) where the first row contains lower bounds and the second row contains upper bounds.

Parameters:

  • bounds (ndarray | list[tuple[float, float]]) –

    Bounds in one of the following formats: - numpy array of shape (2, d): [[lower_1, …, lower_d],

    [upper_1, …, upper_d]]

    • list of tuples: [(lower_1, upper_1), …, (lower_d, upper_d)]

  • device (device | None) – Optional device to place the tensor on. If None, uses CPU.

  • dtype (dtype) – Desired data type of the output tensor. Default is torch.float64.

Raises:

ValueError – If bounds are not in a valid format or if lower bounds are not <= upper bounds.

Return type:

Tensor

Returns:

Tensor of shape (2, d) in BoTorch format.

Example

>>> bounds = [(0.0, 1.0), (0.0, 1.0)]
>>> bounds_tensor = get_bounds_tensor(bounds)
>>> print(bounds_tensor.shape)
torch.Size([2, 2])
alf_tools.models.utils.botorch_utils.predictions_to_posterior(predictions, device=None, dtype=torch.float64)[source]

Convert ALF Predictions to a BoTorch GPyTorchPosterior.

This allows using ALF’s surrogate model predictions with BoTorch’s acquisition functions.

Parameters:

  • predictions (Predictions) – ALF Predictions object containing means and variances.

  • device (device | None) – Optional device to place tensors on. If None, uses CPU.

  • dtype (dtype) – Desired data type of the output tensor. Default is torch.float64.

Return type:

GPyTorchPosterior

Returns:

BoTorch GPyTorchPosterior wrapping the predictions.

Raises:

  • ValueError – If predictions don’t contain variances (required for posterior).

  • RuntimeError – If variances contain significantly negative values, which may indicate a problem with the surrogate model.

Example

>>> means = np.array([1.0, 2.0, 3.0])
>>> variances = np.array([0.1, 0.2, 0.15])
>>> predictions = Predictions(means=means, variances=variances)
>>> posterior = predictions_to_posterior(predictions)
>>> print(posterior.mean.shape)
torch.Size([3, 1])
alf_tools.models.utils.botorch_utils.tensor_to_candidates(X, modality=Modality.TABULAR, features=None)[source]

Convert a PyTorch tensor to a list of ALF Candidates.

Parameters:

  • X (Tensor) – Tensor of shape (n, d) where n is the number of candidates and d is the feature dimension.

  • modality (Modality) – Modality to assign to the candidates. Default is TABULAR for continuous optimization.

  • features (dict[str, Any] | None) – Optional dictionary of features to assign to all candidates.

Return type:

list[Candidate]

Returns:

List of n Candidate objects.

Raises:

ValueError – If X is not a 2D tensor.

Example

>>> X = torch.tensor([[1.0, 2.0], [3.0, 4.0]])
>>> candidates = tensor_to_candidates(X)
>>> print(len(candidates))
2