Noncovalent Interactions

class mlipaudit.benchmarks.noncovalent_interactions.noncovalent_interactions.NoncovalentInteractionsBenchmark(force_field: ForceField | Calculator, data_input_dir: str | PathLike = './data', run_mode: RunMode | Literal['dev', 'fast', 'standard'] = RunMode.STANDARD)

Benchmark for noncovalent interactions.

name

The unique benchmark name that should be used to run the benchmark from the CLI and that will determine the output folder name for the result file. The name is noncovalent_interactions.

Type:

str

category

A string that describes the category of the benchmark, used for example, in the UI app for grouping. Default, if not overridden, is “General”. This benchmark’s category is “Small Molecules”.

Type:

str

result_class

A reference to the type of BenchmarkResult that will determine the return type of self.analyze(). The result class type is NoncovalentInteractionsResult.

Type:

type[mlipaudit.benchmark.BenchmarkResult] | None

model_output_class

A reference to the NoncovalentInteractionsResult class.

Type:

type[mlipaudit.benchmark.ModelOutput] | None

required_elements

The set of atomic element types that are present in the benchmark’s input files.

Type:

set[str] | None

skip_if_elements_missing

Whether the benchmark should be skipped entirely if there are some atomic element types that the model cannot handle. If False, the benchmark must have its own custom logic to handle missing atomic element types. For this benchmark, the attribute is set to False.

Type:

bool

__init__(force_field: ForceField | Calculator, data_input_dir: str | PathLike = './data', run_mode: RunMode | Literal['dev', 'fast', 'standard'] = RunMode.STANDARD) → None

Initializes the benchmark.

Parameters:

• force_field – The force field model to be benchmarked.

• data_input_dir – The local input data directory. Defaults to “./data”. If the subdirectory “{data_input_dir}/{benchmark_name}” exists, the benchmark expects the relevant data to be in there, otherwise it will download it from HuggingFace.

• run_mode – Whether to run the standard benchmark length, a faster version, or a very fast development version. Subclasses should ensure that when RunMode.DEV, their benchmark runs in a much shorter timeframe, by running on a reduced number of test cases, for instance. Implementing RunMode.FAST being different from RunMode.STANDARD is optional and only recommended for very long-running benchmarks. This argument can also be passed as a string “dev”, “fast”, or “standard”.

Raises:

• ChemicalElementsMissingError – If initialization is attempted with a force field that cannot perform inference on the required elements.

• ValueError – If force field type is not compatible.

run_model() → None

Run a single point energy calculation for each structure.

The calculation is performed as a batched inference using the mlip force field directly. This benchmark will skip structures with unseen elements.

analyze() → NoncovalentInteractionsResult

Calculate the total interaction energies and their abs. deviations.

This calculation will yield the MLIP total interaction energy and energy profile and the abs. deviation compared to the reference data.

Returns:

A NoncovalentInteractionsResult object with the benchmark results.

Raises:

RuntimeError – If called before run_model().

class mlipaudit.benchmarks.noncovalent_interactions.noncovalent_interactions.NoncovalentInteractionsResult(*, failed: bool = False, score: Annotated[float | None, Ge(ge=0), Le(le=1)] = None, systems: list[NoncovalentInteractionsSystemResult], n_skipped_unallowed_elements: int = 0, num_failed: int = 0, mae_interaction_energy_all: float, rmse_interaction_energy_all: float, rmse_interaction_energy_subsets: dict[str, float], mae_interaction_energy_subsets: dict[str, float], rmse_interaction_energy_datasets: dict[str, float], mae_interaction_energy_datasets: dict[str, float])

Results object for the noncovalent interactions benchmark.

systems

The systems results for those that were successfully run.

Type:

list[mlipaudit.benchmarks.noncovalent_interactions.noncovalent_interactions.NoncovalentInteractionsSystemResult]

n_skipped_unallowed_elements

The number of structures skipped due to unallowed elements.

Type:

int

num_failed

The number of structures that failed running inference.

Type:

int

mae_interaction_energy_all

The MAE of the interaction energy over all tested systems.

Type:

float

rmse_interaction_energy_all

The RMSE of the interaction energy over all tested systems.

Type:

float

rmse_interaction_energy_subsets

The RMSE of the interaction energy per subset.

Type:

dict[str, float]

mae_interaction_energy_subsets

The MAE of the interaction energy per subset.

Type:

dict[str, float]

rmse_interaction_energy_datasets

The RMSE of the interaction energy per dataset.

Type:

dict[str, float]

mae_interaction_energy_datasets

The MAE of the interaction energy per dataset.

Type:

dict[str, float]

failed

Whether all the simulations or inferences failed and no analysis could be performed. Defaults to False.

Type:

bool

score

The final score for the benchmark between 0 and 1.

Type:

float | None

class mlipaudit.benchmarks.noncovalent_interactions.noncovalent_interactions.NoncovalentInteractionsSystemResult(*, system_id: str, structure_name: str, dataset: str, group: str, reference_interaction_energy: float | None = None, mlip_interaction_energy: float | None = None, deviation: float | None = None, reference_energy_profile: list[float] | None = None, energy_profile: list[float] | None = None, distance_profile: list[float] | None = None, failed: bool = False)

Results object for the noncovalent interactions benchmark for a single bi-molecular system.

system_id

The system id.

Type:

str

structure_name

The structure name.

Type:

str

dataset

The dataset name.

Type:

str

group

The group name.

Type:

str

reference_interaction_energy

The reference interaction energy.

Type:

float | None

mlip_interaction_energy

The MLIP interaction energy.

Type:

float | None

deviation

The deviation between the reference and MLIP interaction energies.

Type:

float | None

reference_energy_profile

The reference energy profile.

Type:

list[float] | None

energy_profile

The MLIP energy profile.

Type:

list[float] | None

distance_profile

The distance profile.

Type:

list[float] | None

failed

Whether running the model failed on the system. Defaults to False.

Type:

bool

class mlipaudit.benchmarks.noncovalent_interactions.noncovalent_interactions.NoncovalentInteractionsModelOutput(*, systems: list[NoncovalentInteractionsSystemModelOutput], n_skipped_unallowed_elements: int, skipped_structures: list[str] = [], failed_structures: list[str] = [])

Model output for the noncovalent interactions benchmark.

systems

List of model outputs for each successfully run system.

Type:

list[mlipaudit.benchmarks.noncovalent_interactions.noncovalent_interactions.NoncovalentInteractionsSystemModelOutput]

n_skipped_unallowed_elements

The number of structures skipped due to unallowed elements.

Type:

int

skipped_structures

The list of skipped structures due to unallowed elements.

Type:

list[str]

failed_structures

The list of structures for which inference failed.

Type:

list[str]

class mlipaudit.benchmarks.noncovalent_interactions.noncovalent_interactions.NoncovalentInteractionsSystemModelOutput(*, system_id: str, energy_profile: list[float] | None = None)

Model output for a bi-molecular system.

system_id

The unique id of the system.

Type:

str

energy_profile

The energy profile of the system. Is None if any of the inferences failed when calculating the energy profile.

Type:

list[float] | None