Configuration¶

This page documents the configuration files used by the chemreporter CLI. Each CLI command (process, query, export) reads a YAML config file, which is validated at runtime against a corresponding schema. These schemas are defined in the chemreporter library as Pydantic classes, linked in the table below; follow those links to the Config Schemas API section further down this page for a detailed, auto-generated reference of every field, its type, and its default value. These example configs can be found in the ChemReporter GitHub repository, and their content is also shown further below on this page.

Command	Example config file	Schema
`process`	`example_configs/process.yaml`	`chemreporter.config_schemas.ProcessDBConfig`
`query`	`example_configs/query.yaml`	`chemreporter.config_schemas.QueryDBConfig`
`export`	`example_configs/export.yaml`	`chemreporter.config_schemas.ExportHDF5Config`

Example YAML config for “process” step¶

The config below is passed to chemreporter process -c <config>.yaml and controls how raw source datasets are ingested into the Query Database.

# Path to the source database to process (local path or s3:// URI)
source_database_path: "/path/to/source_dataset/"

# Name and split of the source database (used to build entry keys)
database_name: "omol25"
split_name: "train"

# Format of the source database files
# One of: aselmdb_omol, aselmdb_omat, aselmdb_oc, aselmdb_odac, aselmdb_omc, xyz, xyz_oc
database_format: "aselmdb_omol"

# Path to the ChemReporter query database where the new entries are appended
query_database_path: "/path/to/query_database/"

# Graph-based processing (RDKit SMILES, fingerprints, bio substructure counts)
graph_based_processing:
  enable: True
  nb_atoms_limit: 600         # skip graph features for structures above this size
  subsets_skip_list:          # subsets to skip for graph features
    - elytes

# Number of structures processed per chunk - lower this if memory is tight
processing_chunk_size: 50000

# Directory of per-database DFT metadata (basis set, functional, dispersion correction)
source_database_metadata:
  basis_set: "def2-TZVPD"
  functional: "ωB97M-V"
  correction_term: "VV10"

Note that the field graph_based_processing is validated by chemreporter.config_schemas.GraphBasedProcessingConfig.

Example YAML config for “query” step¶

The config below is passed to chemreporter query -c <config>.yaml and controls how the Query Database is filtered into a data subset.

# Path to the ChemReporter query database where the queries will be executed
query_database_path: "/path/to/query_database/"

# Path to the results file where the filtered keys will be saved
results_path:  "/path/to/results/filtered_keys.npy"

# The query to be executed
query:  database_name = 'omol25' AND max_force_norm < 15 AND net_force_norm < 1.0e-3

# Sample actions to be executed - remove if not needed
sampling:
  n_samples: 100000
  method: random
  seed: 42 # for reproducibility

For query syntax and filtering examples, see Query Examples.

Example YAML config for “export” step¶

The config below is passed to chemreporter export -c <config>.yaml and controls how the filtered subset is written out to an HDF5 file.

# Path to the ChemReporter query database
query_database_path : "/path/to/query_database/"

# Path to the output HDF5 file that will be created
output_path :  "/path/to/output/export.hdf5"

# Path to the keys file produced by the query command
keys_path: "/path/to/results/filtered_keys.npy"

num_workers: 1
num_files_to_export: 20

Config Schemas API¶

class chemreporter.config_schemas.ExportHDF5Config(*, query_database_path: str, keys_path: str, output_path: str, num_workers: Annotated[int, Ge(ge=1)] = 1, extras_fields: List[str] | None = None, num_files_to_export: Annotated[int, Ge(ge=1)] = 1)[source]¶

Bases: BaseModel

Configuration for the export command.

Variables:

query_database_path (str) – The (local or remote) path to the query database.
keys_path (str) – The (local or remote) path to the numpy file containing the keys.
output_path (str) – The (local or remote) path to the output HDF5 file.
num_workers (int) – The number of workers to use during export. (set multiprocessing to True if over 1).
extras_fields (List[str] | None) – The extra fields to include in the output HDF5 file.
num_files_to_export (int) – The number of files to export. (Setting this to 1 might lead to memory issues on large datasets). Defaults to 1.

extras_fields: List[str] | None¶

keys_path: str¶

model_config = {}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

num_files_to_export: int¶

num_workers: int¶

output_path: str¶

query_database_path: str¶

class chemreporter.config_schemas.GraphBasedProcessingConfig(*, enable: bool = False, nb_atoms_limit: int = 200, subsets_skip_list: list[str] = <factory>)[source]¶

Bases: BaseModel

Configuration class for graph-based processing.

Variables:

enable (bool) – Whether to enable graph-based processing. Defaults to False.
nb_atoms_limit (int) – The maximum number of atoms to process. Defaults to 200.
subsets_skip_list (list[str]) – A list of subsets to skip.

enable: bool¶

model_config = {}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

nb_atoms_limit: int¶

subsets_skip_list: list[str]¶

class chemreporter.config_schemas.ProcessDBConfig(*, source_database_path: str, query_database_path: str, database_name: ~typing.Annotated[str, ~annotated_types.MinLen(min_length=1)], split_name: ~typing.Annotated[str, ~annotated_types.MinLen(min_length=1)] = 'other', database_format: str, processing_chunk_size: ~typing.Annotated[int, ~annotated_types.Gt(gt=0)] = 50000, source_database_metadata: ~chemreporter.config_schemas.SourceDatabaseMetadata = <factory>, graph_based_processing: ~chemreporter.config_schemas.GraphBasedProcessingConfig = <factory>)[source]¶

Bases: BaseModel

Configuration for the process command.

Variables:

source_database_path (str) – The (local or remote) path to the source database.
query_database_path (str) – The (local or remote) path to the query database.
database_name (str) – The name of the database. (User provided, will be used for all lines in the database).
split_name (str) – The name of the split. (User provided, will be used for all lines in the database). Defaults to “other”.
database_format (str) – The format of the database.
processing_chunk_size (int) – The size of the processing chunk. Defaults to 50000.
source_database_metadata (chemreporter.config_schemas.SourceDatabaseMetadata) – The DFT metadata (basis set, functional, correction term) for the source database.
graph_based_processing (chemreporter.config_schemas.GraphBasedProcessingConfig) – The nested configuration for graph-based processing.

database_format: str¶

database_name: str¶

graph_based_processing: GraphBasedProcessingConfig¶

model_config = {}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

processing_chunk_size: int¶

query_database_path: str¶

source_database_metadata: SourceDatabaseMetadata¶

source_database_path: str¶

split_name: str¶

class chemreporter.config_schemas.QueryDBConfig(*, query_database_path: str, results_path: str, query: ~typing.Annotated[str, ~annotated_types.MinLen(min_length=1)], sampling: ~typing.Dict[str, ~typing.Any] = <factory>, actions: ~typing.List[~typing.Any] = <factory>, restrict_to: ~chemreporter.config_schemas.RestrictToConfig | None = None)[source]¶

Bases: BaseModel

Configuration for the query command.

Variables:

query_database_path (str) – The path to the query database.
results_path (str) – The path to the results directory.
query (str) – The query to execute.
sampling (Dict[str, Any]) – Sampling options applied to the query results. Recognized keys: “method” (str, built-in sampler “random” or “all”, or a path to a plugin file defining “custom_sampling_function”; defaults to “random”), “n_samples” (int, number of entries to sample; if omitted, all matching entries are returned), “seed” (int, optional seed for built-in random sampling), “kwargs” (dict, extra arguments forwarded to a custom sampler), and “required_columns” (str or list of str, columns required by the custom sampler function. If not provided, only entry_keys are passed). Example: {“n_samples”: 100000, “method”: “random”}.
actions (List[Any]) – Post-query actions to run. Each item is either an action name (str) to run with no extra arguments, or a single-key dict mapping the action name to its argument. Supported action names: “make_statistics” and “make_histograms” (argument: list of column names), and “extract_smiles” (no argument). Example: [“extract_smiles”, {“make_statistics”: [“logp”, “num_atoms”]}].
restrict_to (chemreporter.config_schemas.RestrictToConfig | None) – The restrict to configuration.

actions: List[Any]¶

model_config = {}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

query: str¶

query_database_path: str¶

restrict_to: RestrictToConfig | None¶

results_path: str¶

sampling: Dict[str, Any]¶

class chemreporter.config_schemas.RestrictToConfig(*, columns: list[str], path_to_values: Annotated[str, MinLen(min_length=1)])[source]¶

Bases: BaseModel

Allowlist filter applied after the SQL query (semi-join).

Variables:

columns (list[str]) – The columns to filter on.
path_to_values (str) – The path to a single “.npy” or “.npz” file with the values to filter on. A “.npy” file only supports a single column. A “.npz” file supports one or more columns, stored as one named 1D array per column (key = column name).

Example

1. Single column, “.npy” file columns: [“smiles”] path_to_values: path_to_file.npy File content: 1D array of smiles values, e.g. [“C1CCCCC1”, “C1CCCCCC1”]

2. Multiple columns, “.npz” file columns: [“smiles”, “charge”] path_to_values: path_to_file.npz File content: one named 1D array per column, e.g. smiles=[“C1CCCCC1”, “C1CCCCCC1”], charge=[0, 1]

columns: list[str]¶

model_config = {}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

path_to_values: str¶

class chemreporter.config_schemas.SourceDatabaseMetadata(*, basis_set: str = '', functional: str = '', correction_term: str = '')[source]¶

Bases: BaseModel

Per-source-dataset DFT metadata (basis set, functional, dispersion correction).

Metadata is optional, but if provided, every entry processed from the source database is set to these values in the query database.

Variables:

basis_set (str) – The basis set used for the DFT calculations.
functional (str) – The functional used for the DFT calculations.
correction_term (str) – The correction term used for the DFT calculations.

basis_set: str¶

correction_term: str¶

functional: str¶

model_config = {}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].