queens.drivers package#

Drivers.

Modules for calling external simulation software.

Submodules#

queens.drivers.function module#

Function Driver.

class Function[source]#

Bases: Driver

Driver to run an python function.

function#

Function to evaluate.

Type:

function

function_requires_job_id#

True if function requires job_id

Type:

bool

function_requires_num_procs#

True if function requires num_procs

Type:

bool

function_requires_experiment_dir#

True if function requires experiment_dir

Type:

bool

function_requires_experiment_name#

True if function requires experiment_name

Type:

bool

__init__(parameters, function, external_python_module_function=None, worker_log_level=None)[source]#

Initialize Function object.

Parameters:

  • parameters (Parameters) – Parameters object

  • function (callable, str) – Function or name of example function provided by QUEENS. If the function contains the arguments job_id, num_procs, experiment_dir, or experiment_name, they will be passed to the function when calling the run method. If the function contains a **kwargs parameter, all the arguments will be passed. Thus, these argument names should be avoided for non-related arguments.

  • external_python_module_function (Path | str) – Path to external module with function

  • worker_log_level (int | str | None) – Logging level of the job log files (default: None, i.e., no log files are written)

static function_wrapper(function)[source]#

Wrap the function to be used.

This wrapper calls the function by a kwargs dict only and reshapes the output as needed. This way if called in a pool, the reshaping is also done by the workers.

Parameters:

function (function) – Function to be wrapped

Returns:

reshaped_output_function (function) – Wrapped function

run(sample, job_id, num_procs, experiment_dir, experiment_name)[source]#

Run the driver.

Parameters:

  • sample (np.ndarray) – Input sample

  • job_id (int) – Job ID

  • num_procs (int) – number of processors

  • experiment_dir (Path) – Path to QUEENS experiment directory.

  • experiment_name (str) – name of QUEENS experiment.

Returns:

Result and potentially the gradient

Return type:

dict

queens.drivers.jobscript module#

Driver to run a jobscript.

class JobOptions[source]#

Bases: object

Dataclass for job options.

All the attributes of this dataclass can be injected into input and jobscript files. The input files dictionary will be flattened, such that the input files are injected by their key.

__init__(job_dir, output_dir, output_file, job_id, num_procs, experiment_dir, experiment_name, input_files)#
Parameters:

  • job_dir (Path)

  • output_dir (Path)

  • output_file (Path)

  • job_id (int)

  • num_procs (int)

  • experiment_dir (Path)

  • experiment_name (str)

  • input_files (dict)

Return type:

None

add_data_and_to_dict(additional_data)[source]#

Add additional options to the job options dict.

Parameters:

additional_data (dict) – Additional data to combine with the job options.

Returns:

dict – Dict combining the job options and the additional data.

experiment_dir: Path#
experiment_name: str#
input_files: dict#
job_dir: Path#
job_id: int#
num_procs: int#
output_dir: Path#
output_file: Path#
to_dict()[source]#

Create a job options dict.

Returns:

dict – Dict containing all the data.

class Jobscript[source]#

Bases: Driver

Driver to run an executable with a jobscript.

input_templates#

Read in simulation input template as string.

Type:

Path

data_processor#

Instance of data processor class.

Type:

obj

gradient_data_processor#

Instance of data processor class for gradient data.

Type:

obj

jobscript_template#

Read-in jobscript template.

Type:

str

jobscript_options#

Dictionary containing jobscript options.

Type:

dict

jobscript_file_name#

Jobscript file name (default: ‘jobscript.sh’).

Type:

str

raise_error_on_jobscript_failure#

Whether to raise an error for a non-zero jobscript exit code.

Type:

bool

__init__(parameters, input_templates, jobscript_template, executable, files_to_copy=None, data_processor=None, gradient_data_processor=None, jobscript_file_name='jobscript.sh', extra_options=None, raise_error_on_jobscript_failure=True, worker_log_level=20)[source]#

Initialize Jobscript object.

Parameters:

  • parameters (Parameters) – Parameters object.

  • input_templates (str, Path, dict) – (Local) path(s) to simulation input template.

  • jobscript_template (str, Path) – (Local) path to jobscript template or read-in jobscript template.

  • executable (str, Path) – Path to main executable of respective software. Is a remote path when using the Cluster scheduler.

  • files_to_copy (list, opt) – Files or directories to copy to experiment_dir.

  • data_processor (obj, opt) – Instance of data processor class.

  • gradient_data_processor (obj, opt) – Instance of data processor class for gradient data.

  • jobscript_file_name (str, opt) – Jobscript file name (default: ‘jobscript.sh’).

  • extra_options (dict, opt) – Extra options to inject into jobscript template.

  • raise_error_on_jobscript_failure (bool, opt) – Whether to raise an error for a non-zero jobscript exit code.

  • worker_log_level (int | str | None) – Logging level of the job log files (default: logging.INFO). None switches them off.

static create_input_templates_dict(input_templates)[source]#

Cast input templates into a dict.

Parameters:

input_templates (str, Path, dict) – Input template(s).

Returns:

dict – Dict containing input file names and template paths.

static get_read_in_jobscript_template(jobscript_template)[source]#

Get the jobscript template contents.

If the provided jobscript template is a Path or a string of a path and a valid file, the corresponding file is read.

Parameters:

jobscript_template (str, Path) – Path to jobscript template or read-in jobscript template.

Returns:

str – Read-in jobscript template

prepare_input_files(sample_dict, experiment_dir, input_files)[source]#

Prepare and parse data to input files.

Parameters:

  • sample_dict (dict) – Dict containing sample.

  • experiment_dir (Path) – Path to QUEENS experiment directory.

  • input_files (dict) – Dict with name and path of the input file(s).

run(sample, job_id, num_procs, experiment_dir, experiment_name)[source]#

Run the driver.

Parameters:

  • sample (np.array) – Input sample.

  • job_id (int) – Job ID.

  • num_procs (int) – Number of processors.

  • experiment_dir (Path) – Path to QUEENS experiment directory.

  • experiment_name (str) – Name of QUEENS experiment.

Returns:

Result and potentially the gradient.

Return type:

dict

read_log_file_tail(log_file, number_of_lines)[source]#

Read the last lines of a log file.

Parameters:

  • log_file (Path) – Path to the log file.

  • number_of_lines (int) – Maximum number of lines to read.

Returns:

The requested log file tail and whether the log file was truncated.

Return type:

tuple[str, bool]

queens.drivers.mpi module#

Convenience wrapper around Jobscript Driver.

class Mpi[source]#

Bases: Jobscript

Driver to run a generic MPI run.

__init__(parameters, input_templates, executable, files_to_copy=None, data_processor=None, gradient_data_processor=None, mpi_cmd='/usr/bin/mpirun --bind-to none', worker_log_level=20)[source]#

Initialize MPI object.

Parameters:

  • parameters (Parameters) – Parameters object

  • input_templates (str, Path, dict) – path to simulation input template

  • executable (str, Path) – path to main executable of respective software

  • files_to_copy (list, opt) – files or directories to copy to experiment_dir

  • data_processor (Callable, opt) – data processor

  • gradient_data_processor (Callable, opt) – data processor class for gradient data

  • mpi_cmd (str, opt) – mpi command

  • worker_log_level (int | str | None) – Logging level of the job log files (default: logging.INFO). None switches them off.