Conformer Selection¶

class mlipaudit.benchmarks.conformer_selection.conformer_selection.ConformerSelectionBenchmark(force_field: ForceField | Calculator, data_input_dir: str | PathLike = './data', run_mode: RunMode | Literal['dev', 'fast', 'standard'] = RunMode.STANDARD)¶

Benchmark for small organic molecule conformer selection.

name¶

The unique benchmark name that should be used to run the benchmark from the CLI and that will determine the output folder name for the result file. The name is conformer_selection.

Type:: str

category¶

A string that describes the category of the benchmark, used for example, in the UI app for grouping. Default, if not overridden, is “General”. This benchmark’s category is “Small Molecules”.

Type:: str

result_class¶

A reference to the type of BenchmarkResult that will determine the return type of self.analyze(). The result class type is ConformerSelectionResult.

Type:: type[mlipaudit.benchmark.BenchmarkResult] | None

model_output_class¶

A reference to the ConformerSelectionModelOutput class.

Type:: type[mlipaudit.benchmark.ModelOutput] | None

required_elements¶

The set of element types that are present in the benchmark’s input files.

Type:: set[str] | None

skip_if_elements_missing¶

Whether the benchmark should be skipped entirely if there are some element types that the model cannot handle. If False, the benchmark must have its own custom logic to handle missing element types. For this benchmark, the attribute is set to True.

Type:: bool

__init__(force_field: ForceField | Calculator, data_input_dir: str | PathLike = './data', run_mode: RunMode | Literal['dev', 'fast', 'standard'] = RunMode.STANDARD) → None¶

Initializes the benchmark.

Parameters:

force_field – The force field model to be benchmarked.
data_input_dir – The local input data directory. Defaults to “./data”. If the subdirectory “{data_input_dir}/{benchmark_name}” exists, the benchmark expects the relevant data to be in there, otherwise it will download it from HuggingFace.
run_mode – Whether to run the standard benchmark length, a faster version, or a very fast development version. Subclasses should ensure that when RunMode.DEV, their benchmark runs in a much shorter timeframe, by running on a reduced number of test cases, for instance. Implementing RunMode.FAST being different from RunMode.STANDARD is optional and only recommended for very long-running benchmarks. This argument can also be passed as a string “dev”, “fast”, or “standard”.

Raises:

ChemicalElementsMissingError – If initialization is attempted with a force field that cannot perform inference on the required elements.
ValueError – If force field type is not compatible.

run_model() → None¶

Run a single point energy calculation for each structure.

The calculation is performed as a batched inference using the MLIP force field directly. The energy profile is stored in the model_output attribute.

analyze() → ConformerSelectionResult¶

Calculates the MAE, RMSE and Spearman correlation.

The results are returned. For a correct representation of the energy differences, the lowest energy conformer of the reference data is set to zero for the reference and inference energy profiles.

Returns:: A ConformerSelectionResult object with the benchmark results.
Raises:: RuntimeError – If called before run_model().

class mlipaudit.benchmarks.conformer_selection.conformer_selection.ConformerSelectionResult(*, failed: bool = False, score: Annotated[float | None, Ge(ge=0), Le(le=1)] = None, molecules: list[ConformerSelectionMoleculeResult], avg_mae: Annotated[float, Ge(ge=0)] | None = None, avg_rmse: Annotated[float, Ge(ge=0)] | None = None)¶

Results object for small molecule conformer selection benchmark.

molecules¶

The individual results for each molecule in a list.

Type:: list[mlipaudit.benchmarks.conformer_selection.conformer_selection.ConformerSelectionMoleculeResult]

avg_mae¶

The MAE values for all molecules that didn’t fail averaged. Is None in the case all the inferences failed.

Type:: float | None

avg_rmse¶

The RMSE values for all molecules that didn’t fail averaged. Is None in the case all the inferences failed.

Type:: float | None

failed¶

Whether all the simulations or inferences failed and no analysis could be performed. Defaults to False.

Type:: bool

score: The final score for the benchmark between: 0 and 1.

class mlipaudit.benchmarks.conformer_selection.conformer_selection.ConformerSelectionMoleculeResult(*, molecule_name: str, mae: Annotated[float, Ge(ge=0)] | None = None, rmse: Annotated[float, Ge(ge=0)] | None = None, spearman_correlation: Annotated[float | None, Ge(ge=-1.0), Le(le=1.0)] = None, spearman_p_value: Annotated[float | None, Ge(ge=0), Le(le=1)] = None, predicted_energy_profile: list[float] | None = None, reference_energy_profile: list[float] | None = None, failed: bool = False)¶

Results object for small molecule conformer selection benchmark for a single molecule. Will have attributes set to None if the inference failed.

molecule_name¶

The molecule’s name.

Type:: str

mae¶

The MAE between the predicted and reference energy profiles of the conformers.

Type:: float | None

rmse¶

The RMSE between the predicted and reference energy profiles of the conformers.

Type:: float | None

spearman_correlation¶

The spearman correlation coefficient between predicted and reference energy profiles.

Type:: float | None

spearman_p_value¶

The spearman p value between predicted and reference energy profiles.

Type:: float | None

predicted_energy_profile¶

The predicted energy profile for each conformer.

Type:: list[float] | None

reference_energy_profile¶

The reference energy profiles for each conformer.

Type:: list[float] | None

failed¶

Whether the inference failed on the molecule.

Type:: bool

class mlipaudit.benchmarks.conformer_selection.conformer_selection.ConformerSelectionModelOutput(*, molecules: list[ConformerSelectionMoleculeModelOutput], num_failed: int = 0)¶

Stores model outputs for the conformer selection benchmark.

molecules¶

Results for each molecule.

Type:: list[mlipaudit.benchmarks.conformer_selection.conformer_selection.ConformerSelectionMoleculeModelOutput]

num_failed¶

The number of molecules on which inference failed.

Type:: int

class mlipaudit.benchmarks.conformer_selection.conformer_selection.ConformerSelectionMoleculeModelOutput(*, molecule_name: str, predicted_energy_profile: list[float] | None = None, failed: bool = False)¶

Stores model outputs for the conformer selection benchmark for a given molecule.

molecule_name¶

The molecule’s name.

Type:: str

predicted_energy_profile¶

The predicted energy profile for the conformers. Is None if the inference failed on the molecule.

Type:: list[float] | None

failed¶

Whether the inference failed on the molecule.

Type:: bool