queens.schedulers package#

Schedulers.

Modules for scheduling and submitting computational jobs.

Submodules#

queens.schedulers.cluster module#

Cluster scheduler for QUEENS runs.

class Cluster(experiment_name, workload_manager, walltime, remote_connection, num_jobs=1, min_jobs=0, num_procs=1, num_nodes=1, queue=None, cluster_internal_address=None, restart_workers=False, allowed_failures=5, verbose=True, experiment_base_dir=None, overwrite_existing_experiment=False)[source]#

Bases: Dask

Cluster scheduler for QUEENS.

copy_files_to_experiment_dir(paths)[source]#

Copy file to experiment directory.

Parameters:

paths (Path, list) – Paths to files or directories that should be copied to experiment directory

local_experiment_dir(experiment_name, experiment_base_dir, overwrite_existing_experiment)[source]#

Get the local experiment directory.

Parameters:

  • experiment_name (str) – name of the current experiment

  • experiment_base_dir (str, Path) – Base directory for the simulation outputs

  • overwrite_existing_experiment (bool) – If true, continue and overwrite experiment directory. If false, prompt user for confirmation before continuing and overwriting.

Returns:

experiment_dir (Path) – Path to local experiment directory.

remote_experiment_dir(experiment_name, experiment_base_dir, overwrite_existing_experiment)[source]#

Get experiment directory on remote host.

Parameters:

  • experiment_name (str) – name of the current experiment

  • experiment_base_dir (str, Path) – Remote base directory for the simulation outputs

  • overwrite_existing_experiment (bool) – If true, continue and overwrite experiment directory. If false, prompt user for confirmation before continuing and overwriting.

Returns:

experiment_dir (Path) – Path to experiment directory on remote host.

restart_worker(worker)[source]#

Restart a worker.

This method retires a dask worker. The Client.adapt method of dask takes cares of submitting new workers subsequently.

Parameters:

worker (str, tuple) – Worker to restart. This can be a worker address, name, or a both.

timedelta_to_str(timedelta_obj)[source]#

Format a timedelta object to str.

This function seems unnecessarily complicated, but unfortunately the datetime library does not

support this formatting for timedeltas. Returns the format HH:MM:SS.

Parameters:

timedelta_obj (datetime.timedelta) – Timedelta object to format

Returns:

str – String of the timedelta object

queens.schedulers.local module#

Local scheduler for QUEENS runs.

class Local(experiment_name, num_jobs=1, num_procs=1, restart_workers=False, verbose=True, experiment_base_dir=None, overwrite_existing_experiment=False)[source]#

Bases: Dask

Local scheduler class for QUEENS.

restart_worker(worker)[source]#

Restart a worker.

Parameters:

worker (str, tuple) – Worker to restart. This can be a worker address, name, or a both.

queens.schedulers.pool module#

Pool scheduler for QUEENS runs.

class Pool(experiment_name, num_jobs=1, verbose=True, experiment_base_dir=None, overwrite_existing_experiment=False)[source]#

Bases: Scheduler

Pool scheduler class for QUEENS.

pool#

Multiprocessing pool.

Type:

pathos pool

evaluate(samples: Iterable, function: SchedulerCallableSignature, job_ids: Iterable = None) → dict[source]#

Submit jobs to driver.

Parameters:

  • samples (np.array) – Array of samples

  • function (Callable) – Callable to evaluate in the scheduler

  • job_ids (lst, opt) – List of job IDs corresponding to samples

Returns:

result_dict (dict) – Dictionary containing results