Dihedral Scan

class mlipaudit.benchmarks.dihedral_scan.dihedral_scan.DihedralScanBenchmark(force_field: ForceField | Calculator, data_input_dir: str | PathLike = './data', run_mode: RunMode | Literal['dev', 'fast', 'standard'] = RunMode.STANDARD)

Benchmark for small organic molecule dihedral scan.

name

The unique benchmark name that should be used to run the benchmark from the CLI and that will determine the output folder name for the result file. The name is dihedral_scan.

Type:

str

category

A string that describes the category of the benchmark, used for example, in the UI app for grouping. Default, if not overridden, is “General”. This benchmark’s category is “Small Molecules”.

Type:

str

result_class

A reference to the type of BenchmarkResult that will determine the return type of self.analyze(). The result class is DihedralScanResult.

Type:

type[mlipaudit.benchmark.BenchmarkResult] | None

model_output_class

A reference to the DihedralScanModelOutput class.

Type:

type[mlipaudit.benchmark.ModelOutput] | None

required_elements

The set of atomic element types that are present in the benchmark’s input files.

Type:

set[str] | None

skip_if_elements_missing

Whether the benchmark should be skipped entirely if there are some atomic element types that the model cannot handle. If False, the benchmark must have its own custom logic to handle missing atomic element types. For this benchmark, the attribute is set to True.

Type:

bool

__init__(force_field: ForceField | Calculator, data_input_dir: str | PathLike = './data', run_mode: RunMode | Literal['dev', 'fast', 'standard'] = RunMode.STANDARD) → None

Initializes the benchmark.

Parameters:

• force_field – The force field model to be benchmarked.

• data_input_dir – The local input data directory. Defaults to “./data”. If the subdirectory “{data_input_dir}/{benchmark_name}” exists, the benchmark expects the relevant data to be in there, otherwise it will download it from HuggingFace.

• run_mode – Whether to run the standard benchmark length, a faster version, or a very fast development version. Subclasses should ensure that when RunMode.DEV, their benchmark runs in a much shorter timeframe, by running on a reduced number of test cases, for instance. Implementing RunMode.FAST being different from RunMode.STANDARD is optional and only recommended for very long-running benchmarks. This argument can also be passed as a string “dev”, “fast”, or “standard”.

Raises:

• ChemicalElementsMissingError – If initialization is attempted with a force field that cannot perform inference on the required elements.

• ValueError – If force field type is not compatible.

run_model() → None

Run a single point energy calculation for each conformer for each fragment.

The calculation is performed as a batched inference using the MLIP force field directly. The predicted energies are then stored as a list corresponding to the conformers for each fragment.

analyze() → DihedralScanResult

Calculates the RMSD between the MLIP and reference structures.

The MAE and RMSE are calculated for each structure in the inference_results attribute. The results are stored in the analysis_results attribute. The results contain the MAE, RMSE and inference energy profile along the dihedral.

Returns:

A DihedralScanResult object with the benchmark results.

Raises:

RuntimeError – If called before run_model().

class mlipaudit.benchmarks.dihedral_scan.dihedral_scan.DihedralScanResult(*, failed: bool = False, score: Annotated[float | None, Ge(ge=0), Le(le=1)] = None, avg_mae: Annotated[float, Ge(ge=0)] | None = None, avg_rmse: Annotated[float, Ge(ge=0)] | None = None, avg_pearson_r: Annotated[float, Ge(ge=-1.0), Le(le=1.0)] = None, avg_pearson_p: Annotated[float, Ge(ge=0), Le(le=1)] = None, mae_barrier_height: Annotated[float, Ge(ge=0)] | None = None, fragments: list[DihedralScanFragmentResult])

Results object for the dihedral scan benchmark.

avg_mae

The avg mae across all fragments.

Type:

float | None

avg_rmse

The avg rmse across all fragments.

Type:

float | None

avg_pearson_r

The avg Pearson correlation coefficient across all fragments.

Type:

float

avg_pearson_p

The avg Pearson p-value across all fragments.

Type:

float

mae_barrier_height

The MAE of the barrier heights across all fragments.

Type:

float | None

fragments

A list of results objects per fragment.

Type:

list[mlipaudit.benchmarks.dihedral_scan.dihedral_scan.DihedralScanFragmentResult]

failed

Whether all the simulations or inferences failed and no analysis could be performed. Defaults to False.

Type:

bool

score

The final score for the benchmark between 0 and 1.

Type:

float | None

class mlipaudit.benchmarks.dihedral_scan.dihedral_scan.DihedralScanFragmentResult(*, fragment_name: str, mae: Annotated[float, Ge(ge=0)] | None = None, rmse: Annotated[float, Ge(ge=0)] | None = None, pearson_r: Annotated[float, Ge(ge=-1.0), Le(le=1.0)] = None, pearson_p: Annotated[float, Ge(ge=0), Le(le=1)] = None, barrier_height_error: Annotated[float, Ge(ge=0)] | None = None, predicted_energy_profile: list[float] | None = None, reference_energy_profile: list[float] | None = None, distance_profile: list[float] | None = None, failed: bool = False)

Stores individual fragment results.

fragment_name

The name of the fragment.

Type:

str

mae

The mean absolute error between the predicted energy and the reference energies for all the conformers.

Type:

float | None

rmse

The root mean square error between the predicted energy and the reference energies for all the conformers.

Type:

float | None

pearson_r

The Pearson correlation coefficient between the predicted and reference energies for all the conformers.

Type:

float

pearson_p

The p-value of the Pearson correlation coefficient between the predicted and reference energies for all the conformers.

Type:

float

barrier_height_error

The absolute difference between the predicted and reference barrier height.

Type:

float | None

predicted_energy_profile

The aligned predicted energies for each conformer in kcal/mol.

Type:

list[float] | None

reference_energy_profile

The reference energies for each conformer in kcal/mol.

Type:

list[float] | None

distance_profile

The torsion angle for each conformer.

Type:

list[float] | None

failed

Whether the inference failed for the fragment. Defaults to false.

Type:

bool

class mlipaudit.benchmarks.dihedral_scan.dihedral_scan.DihedralScanModelOutput(*, fragments: list[FragmentModelOutput], num_failed: int = 0)

Stores energy predictions per fragment per conformer.

fragments

A list of predictions per fragment.

Type:

list[mlipaudit.benchmarks.dihedral_scan.dihedral_scan.FragmentModelOutput]

num_failed

The number of molecules on which inference failed.

Type:

int

class mlipaudit.benchmarks.dihedral_scan.dihedral_scan.FragmentModelOutput(*, fragment_name: str, energy_predictions: list[float] | None = None, failed: bool = False)

Stores energy predictions per conformer. Will have attributes set to None if the inference failed.

fragment_name

The name of the fragment.

Type:

str

energy_predictions

The list of energy predictions for each conformer of the fragment.

Type:

list[float] | None

failed

Whether the inference failed on the molecule.

Type:

bool