queens.models.surrogate_models package#

Surrogate models.

Modules for surrogate models.

Subpackages#

queens.models.surrogate_models.utils package
- Submodules
- queens.models.surrogate_models.utils.kernel_utils_jitted module

Submodules#

queens.models.surrogate_models.bayesian_neural_network module#

Implementation of a Bayesian Neural Network.

class GaussianBayesianNeuralNetworkModel(training_iterator=None, testing_iterator=None, eval_fit=None, error_measures=None, plotting_options=None, num_posterior_samples=None, num_samples_statistics=None, adams_training_rate=None, nodes_per_hidden_layer_lst=None, activation_per_hidden_layer_lst=None, num_epochs=None, optimizer_seed=None, verbosity_on=None)[source]#

Bases: SurrogateModel

A Bayesian Neural network.

Class for creating a Bayesian neural network with Gaussian conditional distribution based on Tensorflow Probability.

The network can handle heteroskedastic noise and an arbitrary nonlinear functional. As we use Tensorflow variational layers and learn the mean and variance function of a Gaussian distribution, the network is able to handle epistemic and aleatory uncertainty.

num_posterior_samples#

Number of posterior sample functions (realizations of Bayesian neural network).

Type:: int

num_samples_statistics#

Number of samples to approximate posterior statistics.

Type:: int

bnn_model#

Tensorflow based Bayesian neural network model.

Type:: tf.model

num_epochs#

Number of training epochs for variational optimization.

Type:: int

optimizer_seed#

Random seed used for initialization of stochastic gradient decent optimizer.

Type:: int

verbosity_on#

Boolean for model verbosity during training (True=verbose).

Type:: bool

model_realizations_lst#

List with different neural network realizations (epistemic uncertainty).

Type:: lst

adams_training_rate#

Training rate for the ADAMS gradient decent optimizer

Type:: float

nodes_per_hidden_layer_lst#

List containing number of nodes per hidden layer of the Bayesian Neural Network. The length of the list defines the deepness of the model and the values the width of the individual layers.

Type:: lst

activation_per_hidden_layer_lst#

List with strings encoding the activation function that shall be used for the respective hidden layer of the Bayesian Neural Network

Type:: lst

grad(samples, upstream_gradient)[source]#

Evaluate gradient of model w.r.t. current set of input samples.

Consider current model f(x) with input samples x, and upstream function g(f). The provided upstream gradient is \(\frac{\partial g}{\partial f}\) and the method returns \(\frac{\partial g}{\partial f} \frac{df}{dx}\).

Parameters:

samples (np.array) – Input samples
upstream_gradient (np.array) – Upstream gradient function evaluated at input samples \(\frac{\partial g}{\partial f}\)

Returns:

gradient (np.array) – Gradient w.r.t. current set of input samples \(\frac{\partial g}{\partial f} \frac{df}{dx}\)

static mean_field_variational_distribution(kernel_size, bias_size=0, dtype=None)[source]#

Variational distribution that approximates the true posterior.

Here, we use a Gaussian mean field approach, such that every parameter in the NN is approximated with a Gaussian distribution, parameterized by a mean and variance.

Parameters:

kernel_size (int) – Number of weight parameters in the NN
bias_size (int) – Number of bias parameters in the NN
dtype (str) – DataType string

Returns:

mean_field_variational_distr (obj) – Tensorflow Probability mean field variational
distribution for weights and biases. Each node (with weight and bias)
that are individually optimized via their mean and variance.
Hence the parameterization of the variational distribution has n * 2 parameters with
n being the number of nodes in the NN.

static negative_log_likelihood(y, random_variable_y)[source]#

Evaluate the negative log.-likelihood of the random variable.

Negative logarithmic likelihood of (tensorflow) random variable random_variable_y evaluated at location y.

Parameters:

y (float) – Value/Realization of the random variable
random_variable_y (obj) – Tensorflow probability random variable object

Returns:

negloglik (float) – Negative logarithmic likelihood of random_variable_y at y

predict(x_test, support='y', full_cov=False)[source]#

Make a prediction with the Bayesian neural network.

Parameters:

x_test (np.array) – Testing input vector for which the posterior distribution, respectively point estimates should be predicted
support (str) – String to determine which output
full_cov (bool) – Boolean flag for prediction posterior covariance matrix (True) or posterior variance (False) at x_test

Returns:

output (dict) – Dictionary with posterior output statistics

predict_f(x_test, full_cov=False)[source]#

Predict the posterior statistics of the latent function ‘f’.

Predict the posterior mean, variance or covariance of the posterior distribution w.r.t. the latent function ‘f’ at test points x_test. This prediction only accounts for the epistemic uncertainty and neglects the aleatory uncertainty.

Parameters:

x_test (np.array) – Testing input vector for which the posterior distribution, respectively point estimates should be predicted
full_cov (bool) – Boolean flag for prediction posterior covariance matrix (True) or posterior variance (False) at x_test

Returns:

output (dict) – Dictionary with posterior output statistics

predict_f_samples(x_test, num_samples)[source]#

Predict posterior samples of the latent function ‘f’.

Sample the latent function (without noise), that is in this case realizations of the mean predictions of the Bayesian neural network which are themselves uncertain.

Parameters:

x_test (np.array) – Input testing point at which the samples should be realized
num_samples (int) – Number of posterior samples of the latent function ‘f’

Returns:

samples (np.array) – Samples of the latent function at locations x_test

predict_y(x_test, full_cov=False)[source]#

Predict the posterior mean, variance/covariance.

Predictions are conducted for the posterior distribution of the random output variable at x_test (w.r.t. y), that combines epistemic and aleatory uncertainty.

Parameters:

x_test (np.array) – Testing input vector for which the posterior distribution, respectively point estimates should be predicted
full_cov (bool) – Boolean flag for prediction posterior covariance matrix (True) or posterior variance (False) at x_test

Returns:

output (dict) – Dictionary with posterior output statistics

predict_y_samples(x_test, num_samples)[source]#

Sample from the posterior distribution of ‘y’.

Sampling from the posterior (w.r.t. y) that combines epistemic and aleatory uncertainty. This will generally lead to noisy samples in contrast to smooth samples from the latent function as noise assumption is conditionally independent for each point x_test.

Parameters:

x_test (np.array) – Testing input locations for the posterior distribution
num_samples (int) – Number of posterior samples at x_test

Returns:

samples (np.array) – Posterior samples w.r.t. y

static prior_trainable(kernel_size, bias_size=0, dtype=None)[source]#

Specify the prior over keras.layers.Dense, kernel and bias.

Note: This is a special hybrid case of prior which is actually trainable. See “Empirical Bayes” for more background on this topic.

Parameters:

kernel_size (int) – Number of weight parameters in the NN
bias_size (int) – Number of bias parameters in the NN
dtype (str) – DataType string

Returns:

prior (obj) – Tensorflow probability Gaussian prior distribution over biases and
weights of the Bayesian NN, taking the mean of the Gaussian as an input
variable for the prior distribution and fixing the variance

setup(x_train, y_train)[source]#: Setup surrogate model.

train()[source]#

Train the Bayesian neural network.

Train the Bayesian neural network using the previous defined optimizers in the model build and configuration. We allow Tensorflow’s early stopping here to stop the optimization routine when the loss function starts to increase again over several iterations.

queens.models.surrogate_models.gaussian_neural_network module#

Gaussian Neural Network regression model.

class GaussianNeuralNetworkModel(num_epochs=None, batch_size=None, adams_training_rate=None, optimizer_seed=None, verbosity_on=None, nodes_per_hidden_layer_lst=None, activation_per_hidden_layer_lst=None, kernel_initializer=None, nugget_std=None, loss_plot_path=False, refinement_epochs_decay=0.75, data_scaling=None, mean_function_type='zero')[source]#

Bases: SurrogateModel

Class for creating a neural network that parameterizes a Gaussian.

The network can handle heteroskedastic noise and an arbitrary nonlinear functions.

nn_model#

Tensorflow based Bayesian neural network model

Type:: tf.model

num_epochs#

Number of training epochs for variational optimization

Type:: int

optimizer_seed#

Random seed used for initialization of stochastic gradient decent optimizer

Type:: int

verbosity_on#

Boolean for model verbosity during training. True=verbose

Type:: bool

batch_size#

Size of data-batch (smaller than the training data size)

Type:: int

scaler_x#

Scaler for inputs

Type:: obj

scaler_y#

Scaler for outputs

Type:: obj

loss_plot_path#

Path to determine whether loss plot should be produced (yes if provided). Plot will be saved at path location.

Type:: str

num_refinements#

Number of refinements

Type:: int

refinement_epochs_decay#

Decrease of epochs in refinements

Type:: float

mean_function#

Mean function of the Gaussian Neural Network

Type:: function

gradient_mean_function#

Gradient of the mean function of the Gaussian Neural Network

Type:: function

adams_training_rate#

Training rate for the ADAMS gradient decent optimizer

Type:: float

nodes_per_hidden_layer#

List containing number of nodes per hidden layer of the Neural Network. The length of the list defines the deepness of the model and the values the width of the individual layers.

Type:: lst

activation_per_hidden_layer#

List with strings encoding the activation function that shall be used for the respective hidden layer of the Neural Network

Type:: list

kernel_initializer#

Type of kernel initialization for neural network

Type:: str

nugget_std#

Nugget standard deviation for robustness

Type:: float

grad(samples, upstream_gradient)[source]#

Evaluate gradient of model w.r.t.

current set of input samples.

static negative_log_likelihood(y, random_variable_y)[source]#

Negative log-likelihood of (tensorflow) random variable.

Parameters:

y (float) – Value/Realization of the random variable
random_variable_y (obj) – Tensorflow probability random variable object

Returns:

negative_log_likelihood (float) – Negative logarithmic likelihood of random_variable_y at y

predict(x_test, support='y', gradient_bool=False)[source]#

Predict the output distribution at x_test.

Parameters:

x_test (np.array) – Testing input vector for which the posterior distribution, respectively point estimates should be predicted
support (str, optional) – String to define the support of the output distribution - ‘y’: Conditional distribution is defined on the output space - ‘f’: Conditional distribution is defined on the latent space
gradient_bool (bool, optional) – Boolean to configure whether gradients should be returned as well

Returns:

output (dict) – Dictionary with posterior output statistics

predict_and_gradient(x_test)[source]#

Predict the mean, variance and their gradients at x_test.

Parameters:: x_test (np.array) – Testing input vector for which the posterior distribution, respectively point estimates should be predicted
Returns:: output (dict) – Dictionary with posterior output statistics

predict_y(x_test)[source]#

Predict the posterior mean and variance.

Prediction is conducted w.r.t. to the output space “y”.

Parameters:: x_test (np.array) – Testing input vector for which the posterior distribution, respectively point estimates should be predicted
Returns:: output (dict) – Dictionary with posterior output statistics

setup(x_train, y_train)[source]#

Setup surrogate model.

Parameters:

x_train (np.array) – training inputs
y_train (np.array) – training outputs

train()[source]#

Train the Bayesian neural network.

We ues the previous defined optimizers in the model build and configuration. We allow tensorflow’s early stopping here to stop the optimization routine when the loss function starts to increase again over several iterations.

update_training_data(x_train, y_train)[source]#

Update the training data of the model.

Parameters:

x_train (np.array) – Training input array
y_train (np.array) – Training output array

queens.models.surrogate_models.gp_approximation_gpflow module#

Gaussian process implementation in GPFlow.

class GPFlowRegressionModel(training_iterator=None, testing_iterator=None, eval_fit=None, error_measures=None, plotting_options=None, number_posterior_samples=None, seed_optimizer=42, seed_posterior_samples=None, restart_min_value=0, restart_max_value=5, number_restarts=10, number_training_iterations=100, dimension_lengthscales=None, train_likelihood_variance=True)[source]#

Bases: SurrogateModel

Class for creating GP regression model based on GPFlow.

This class constructs a GP regression, using a GPFlow model.

number_posterior_samples#

Number of posterior samples.

Type:: int

number_restarts#

Number of restarts.

Type:: int

number_training_iterations#

Number of iterations in optimizer for training.

Type:: int

number_input_dimensions#

Dimensionality of random features.

Type:: int

restart_min_value#

Minimum value for restart.

Type:: int

restart_max_value#

Maximum value for restart.

Type:: int

model#

GPFlow based Gaussian process model.

Type:: GPFlow.models.GPR

dimension_lengthscales#

Dimension of lengthscales.

Type:: int

train_likelihood_variance#

if true, likelihood variance is trained

Type:: bool

scaler_x#

Scaler for inputs.

Type:: sklearn scaler object

scaler_y#

Scaler for outputs.

Type:: sklearn scaler object

assign_hyperparameters(hyperparameters, transform=False)[source]#

Assign untransformed (constrained) hyperparameters to model.

Parameters:

hyperparameters (np.ndarray) – Hyperparameters of GP
transform (bool) – If True, hyperparameters are transformed from unconstrained to constrained representation

get_dimension_hyperparameters()[source]#

Return the dimension of the hyperparameters.

Returns:: dimension_hyperparameters (int) – Dimension of hyperparameters

grad(samples, upstream_gradient)[source]#

Evaluate gradient of model w.r.t. current set of input samples.

Parameters:

samples (np.array) – Input samples
upstream_gradient (np.array) – Upstream gradient function evaluated at input samples \(\frac{\partial g}{\partial f}\)

Returns:

gradient (np.array) – Gradient w.r.t. current set of input samples \(\frac{\partial g}{\partial f} \frac{df}{dx}\)

predict(x_test, support='y', full_cov=False)[source]#

Predict the posterior distribution at x_new.

Options:

f(x_test): predict the latent function values

y(x_test): predict values of the new observations (including noise)

Parameters:

x_test (np.ndarray) – New inputs where to make predictions
support (str) – Probabilistic support of random process (default: ‘y’)
full_cov (bool) – Boolean that specifies whether the entire posterior covariance matrix should be returned or only the posterior variance

Returns:

output (dict) – Dictionary with mean, variance, and possibly
posterior samples at *x_test*

setup(x_train, y_train)[source]#

Setup surrogate model.

Parameters:

x_train (np.array) – training inputs
y_train (np.array) – training outputs

train()[source]#: Train the GP by maximizing the likelihood.

transform_hyperparameters(hyperparameters)[source]#

Transform hyperparameters.

Transform hyperparameters from unconstrained to constrained representation.

Parameters:: hyperparameters (np.ndarray) – Unconstrained representation of hyperparameters
Returns:: hyperparameters (np.ndarray) – Constrained representation of hyperparameters

queens.models.surrogate_models.gp_approximation_gpflow_svgp module#

Module implementing a scalable GPFlow-based SVGP model.

This module defines the GPflowSVGPModel class, which is designed to facilitate scalable Gaussian Process regression using variational inference as outlined by Hensman et al. (2015). The model leverages TensorFlow for optimization and supports minibatch training, making it suitable for large datasets.

Key features include: - Configurable number of inducing points and minibatch size for efficient training. - Customizable options for training the locations of inducing points and the likelihood variance. - Scalable to high-dimensional input spaces with support for custom lengthscales. - Integration with various evaluation metrics and plotting options.

The class provides methods for model setup, training, gradient evaluation, and prediction, allowing users to easily integrate the SVGP model into their machine learning workflows.

class GPflowSVGPModel(training_iterator=None, testing_iterator=None, eval_fit=None, error_measures=None, plotting_options=None, number_posterior_samples=None, mini_batch_size=100, number_training_iterations=10000, seed=41, number_inducing_points=100, dimension_lengthscales=None, train_inducing_points_location=False, train_likelihood_variance=True)[source]#

Bases: SurrogateModel

Class for creating SVGP regression model based on GPFlow.

Key reference:: J. Hensman, A. Matthews, and Z. Ghahramani, “Scalable Variational Gaussian Process Classification” in Artificial Intelligence and Statistics, Feb. 2015, pp. 351–360. https://proceedings.mlr.press/v38/hensman15.html

number_posterior_samples#

Number of posterior samples.

Type:: int

mini_batch_size#

Minibatch size to speed up computation of ELBO.

Type:: int

number_training_iterations#

Number of iterations in optimizer for training.

Type:: int

training_data#

List of training datasets.

Type:: list

number_input_dimensions#

Dimensionality of random features/input dimension.

Type:: int

model#

List of GPFlow based stochastic variational GP (SVGP).

Type:: list

scaler_x#

Scaler for inputs.

Type:: sklearn scaler object

scaler_y#

Scaler for outputs.

Type:: sklearn scaler object

dimension_output#

Dimensionality of the output (quantities of interest).

Type:: int

seed#

random seed

Type:: int

num_inducing_points#

Number of inducing points

Type:: int

dimension_lengthscales#

Dimension of lengthscales

Type:: int

train_inducing_points_location#

if true, location of inducing points is trained

Type:: bool

train_likelihood_variance#

if true, likelihood variance is trained

Type:: bool

grad(samples, upstream_gradient)[source]#

Evaluate gradient of model w.r.t. current set of input samples.

Parameters:

samples (np.array) – Input samples
upstream_gradient (np.array) – Upstream gradient function evaluated at input samples \(\frac{\partial g}{\partial f}\)

Returns:

gradient (np.array) – Gradient w.r.t. current set of input samples \(\frac{\partial g}{\partial f} \frac{df}{dx}\)

predict(x_test, support='f', full_cov=False)[source]#

Predict the posterior distribution at x_test.

Options:

‘f’: predict the latent function values
‘y’: predict values of the new observations (including noise)

Parameters:

x_test (np.ndarray) – New inputs where to make predictions
support (str) – Probabilistic support of random process (default: ‘y’)
full_cov (bool) – Boolean that specifies whether the entire posterior covariance matrix should be returned or only the posterior variance

Returns:

output (dict) – Dictionary with mean, variance, and possibly
posterior samples at *x_test*

setup(x_train, y_train)[source]#

Setup surrogate model.

Parameters:

x_train (np.array) – training inputs
y_train (np.array) – training outputs

train()[source]#: Train the GP.

queens.models.surrogate_models.gp_approximation_jitted module#

A fast, jitted version of Gaussian Process regression.

class GPJittedModel(stochastic_optimizer, initial_hyper_params_lst=None, kernel_type=None, data_scaling=None, mean_function_type='zero', plot_refresh_rate=None, noise_var_lb=None)[source]#

Bases: SurrogateModel

A jitted Gaussian process implementation using numba.

It just-in-time compiles linear algebra operations. The GP also allows to specify a Gamma hyper-prior or the length scale, but only computes the MAP estimate and does not marginalize the hyper-parameters.

k_mat_inv#

Inverse of the assembled covariance matrix.

Type:: np.array

cholesky_k_mat#

Lower Cholesky decomposition of the covariance matrix.

Type:: np.array

k_mat#

Assembled covariance matrix of the GP.

Type:: np.array

partial_derivatives_hyper_params#

List of partial derivatives of the kernel function w.r.t. the hyper-parameters.

Type:: list

mean_function#

Mean function of the GP

Type:: function

gradient_mean_function#

Gradient of the mean function of the GP

Type:: function

stochastic_optimizer#

Stochastic optimizer object.

Type:: obj

scaler_x#

Scaler for inputs.

Type:: obj

scaler_y#

Scaler for outputs.

Type:: obj

grad_log_evidence_value#

Current gradient of the log marginal likelihood w.r.t. the parameterization.

Type:: np.array

hyper_params#

List of hyper-parameters

Type:: list

noise_variance_lower_bound#

Lower bound for Gaussian noise variance in RBF kernel.

Type:: float

plot_refresh_rate#

Refresh rate of the plot (every n-iterations).

Type:: int

kernel_type#

Type of kernel function.

Type:: str

get_state()[source]#

Get the current hyper-parameters of the model.

Returns:

state_dict (dict) – Dictionary with the current state settings
of the probabilistic mapping object

grad(samples, upstream_gradient)[source]#

Evaluate gradient of model w.r.t. current set of input samples.

Parameters:

samples (np.array) – Input samples
upstream_gradient (np.array) – Upstream gradient function evaluated at input samples \(\frac{\partial g}{\partial f}\)

Returns:

gradient (np.array) – Gradient w.r.t. current set of input samples \(\frac{\partial g}{\partial f} \frac{df}{dx}\)

static gradient_identity_multi_fidelity_mean_fun(samples)[source]#: Return gradient of mean function for multi-fidelity.

static gradient_zero_mean_fun(_samples)[source]#: Return gradient of zero mean function.

static identity_multi_fidelity_mean_fun(samples)[source]#: Return identity mean function for multi-fidelity.

log_evidence()[source]#

Log evidence/log marginal likelihood of the GP.

Returns:: log_evidence (float) – Evidence of the GP for current choice of hyper-parameters

predict(x_test, support='f', gradient_bool=False)[source]#

Predict the posterior distribution of the trained GP at x_test.

Parameters:

x_test (np.array) – Testing matrix for GP with row-wise (vector-valued) testing points
support (str) –
Type of support for which the GP posterior is computed; If: - ‘f’: Posterior w.r.t. the latent function f - ‘y’: Latent function is marginalized such that posterior is defined

w.r.t. the output y (introduces extra variance)
gradient_bool (bool, optional) – Boolean to configure whether gradients should be returned as well

Returns:

output (dict) – Output dictionary containing the posterior of the GP

set_state(state_dict)[source]#

Update and set new hyper-parameters for the model.

Parameters:: state_dict (dict) – Dictionary with the current state settings of the probabilistic mapping object

setup(x_train, y_train)[source]#

Setup surrogate model.

Parameters:

x_train (np.array) – training inputs
y_train (np.array) – training outputs

train()[source]#

Train the Gaussian Process.

Training is conducted by maximizing the evidence/marginal likelihood by minimizing the negative log evidence.

valid_kernels_dict = {'matern_3_2': (CPUDispatcher(<function matern_3_2>), CPUDispatcher(<function posterior_mean_matern_3_2>), CPUDispatcher(<function posterior_var_matern_3_2>), CPUDispatcher(<function grad_log_evidence_matern_3_2>), <function grad_posterior_mean_matern_3_2>, <function grad_posterior_var_matern_3_2>), 'squared_exponential': (CPUDispatcher(<function squared_exponential>), CPUDispatcher(<function posterior_mean_squared_exponential>), CPUDispatcher(<function posterior_var_squared_exponential>), CPUDispatcher(<function grad_log_evidence_squared_exponential>), CPUDispatcher(<function grad_posterior_mean_squared_exponential>), CPUDispatcher(<function grad_posterior_var_squared_exponential>))}#

static zero_mean_fun(_samples)[source]#: Return zero mean function.

queens.models.surrogate_models.gp_heteroskedastic_gpflow module#

Implementation of a heteroskedastic Gaussian process models using GPFlow.

class HeteroskedasticGPModel(training_iterator=None, testing_iterator=None, eval_fit=None, error_measures=None, plotting_options=None, num_posterior_samples=None, num_inducing_points=None, num_epochs=None, adams_training_rate=None, random_seed=None, num_samples_stats=None)[source]#

Bases: SurrogateModel

Class for creating heteroskedastic GP based regression model.

Class for creating heteroskedastic GP based regression model based on GPFlow.

This class constructs a GP regression, currently using a GPFlow model. Currently, a lot of parameters are still hard coded, which will be improved in the future.

The basic idea of this latent variable GP model can be found in [1-3].

num_posterior_samples#

Number of posterior GP samples (realizations of posterior GP).

Type:: int

num_inducing_points#

Number of inducing points for variational GPs.

Type:: int

model#

Gpflow based heteroskedastic Gaussian process model.

Type:: gpf.model

optimizer#

Tensorflow optimization object.

Type:: obj

adams_training_rate#

Training rate for the ADAMS gradient decent optimizer

Type:: float

num_epochs#

Number of training epochs for variational optimization.

Type:: int

random_seed#

Random seed used for initialization of stochastic gradient decent optimizer.

Type:: int

posterior_cov_mat_y#

Posterior covariance matrix of heteroskedastic GP w.r.t. the y-coordinate.

Type:: np.array

num_samples_stats#

Number of samples used to calculate empirical variance/covariance.

Type:: int

References

[1]: https://gpflow.github.io/GPflow/2.6.3/notebooks/advanced/heteroskedastic.html

[2]: Saul, A. D., Hensman, J., Vehtari, A., & Lawrence, N. D. (2016, May).: Chained gaussian processes. In Artificial Intelligence and Statistics (pp. 1431-1440).
[3]: Hensman, J., Fusi, N., & Lawrence, N. D. (2013). Gaussian processes for big data.: arXiv preprint arXiv:1309.6835.

grad(samples, upstream_gradient)[source]#

Evaluate gradient of model w.r.t. current set of input samples.

Parameters:

samples (np.array) – Input samples
upstream_gradient (np.array) – Upstream gradient function evaluated at input samples \(\frac{\partial g}{\partial f}\)

Returns:

gradient (np.array) – Gradient w.r.t. current set of input samples \(\frac{\partial g}{\partial f} \frac{df}{dx}\)

predict(x_test, support=None, full_cov=False)[source]#

Predict the posterior distribution at the given test inputs.

Predict the posterior distribution at Xnew with respect to the data y.

Parameters:

x_test (np.array) – Inputs at which to evaluate or test the latent function ‘f’
support (str) – Predict w.r.t. the latent variable ‘f’ or ‘y’; not needed here as only ‘y’ makes sense
full_cov (bool) – Boolean that specifies whether the entire posterior covariance matrix should be returned or only the posterior variance

Returns:

output (dict) – Dictionary with mean, variance, and possibly
posterior samples at *Xnew*

predict_f_samples(x_test, num_samples)[source]#

Produce samples from the posterior latent function Xnew.

Parameters:

x_test (np.array) – Inputs at which to evaluate latent function ‘f’
num_samples (int) – Number of posterior realizations of GP

Returns:

post_samples (np.array) – Posterior samples of latent functions at x_test
(the latter might be a vector/matrix of points)

predict_y(x_test, full_cov=False)[source]#

Compute the posterior distribution at test inputs considering ‘y’.

Compute the posterior distribution at x_test with respect to the data ‘y’.

Parameters:

x_test (np.array) – Inputs at which to evaluate latent function ‘f’
full_cov (bool) – Boolean to decide if we want posterior variance (full_cov=False) or the full posterior covariance matrix (full_cov=True)

Returns:

output (dict) – Dictionary with mean, variance, and possibly
posterior samples of latent function at *x_test*

setup(x_train, y_train)[source]#

Setup surrogate model.

Parameters:

x_train (np.array) – training inputs
y_train (np.array) – training outputs

train()[source]#

Train the variational.

Train the variational by minimizing the variational loss in variational EM step.

queens.models.surrogate_models.surrogate_model module#

Surrogate model class.

class SurrogateModel(training_iterator=None, testing_iterator=None, eval_fit=None, error_measures=None, plotting_options=None)[source]#

Bases: Model

Surrogate model class.

training_iterator#

Iterator to evaluate the subordinate model with the purpose of getting training data

Type:: Iterator

testing_iterator#

Iterator to evaluate the subordinate model with the purpose of getting testing data

Type:: Iterator

eval_fit#

How to evaluate goodness of fit

Type:: str

error_measures#

List of error measures to compute

Type:: list

is_trained#

true if model is trained

Type:: bool

x_train#

training inputs

Type:: np.array

y_train#

training outputs

Type:: np.array

visualization#

Surrogate visualization object.

Type:: SurrogateVisualization

build_approximation()[source]#: Build underlying approximation.

static compute_error(y_test, y_posterior_mean, measure)[source]#

Compute error for given a specific error measure.

Parameters:

y_test (ndarray) – Output values from testing data set
y_posterior_mean (ndarray) – Posterior mean values of the GP
measure (str) – Desired error metric

Returns:

float – Error based on desired metric

compute_error_measures(y_test, y_posterior_mean, measures)[source]#

Compute error measures.

Compute based on difference between predicted and actual values.

Parameters:

y_test (ndarray) – Output values from testing data set
y_posterior_mean (ndarray) – Posterior mean values of the GP
measures (list) – Dictionary with desired error measures

Returns:

dict – Dictionary with error measures and corresponding error values

static compute_nash_sutcliffe_efficiency(y_test, y_posterior_mean)[source]#

Compute Nash-Sutcliffe model efficiency.

\[NSE = 1-\frac{\sum_{i=1}^{N}(e_{i}-s_{i})^2}{\sum_{i=1}^{N}(e_{i}-\bar{e})^2}\]

Parameters:

y_test (ndarray) – Output values from testing data set
y_posterior_mean (ndarray) – Posterior mean values of the GP

Returns:

efficiency (float) – Nash-Sutcliffe model efficiency

cross_validate(x_train, y_train, folds)[source]#

Cross validation function which calls the regression approximation.

Parameters:

x_train (np.array) – Array of inputs
y_train (np.array) – Array of outputs
folds (int) – In how many subsets do we split for cv

Returns:

np.array – Array with predictions

eval_surrogate_accuracy(x_test, y_test, measures)[source]#

Evaluate the accuracy of the surrogate model based on test set.

Evaluate the accuracy of the surrogate model using the provided error metrics.

Parameters:

x_test (np.array) – Test inputs
y_test (np.array) – Test outputs
measures (list) – List with desired error metrics

Returns:

dict – Dictionary with proving error metrics

eval_surrogate_accuracy_cv(x_test, y_test, k_fold, measures)[source]#

Compute k-fold cross-validation error.

Parameters:

x_test (np.array) – Input array
y_test (np.array) – Output array
k_fold (int) – Split dataset in k_fold subsets for cv
measures (list) – List with desired error metrics

Returns:

dict – y with error measures and corresponding error values

evaluate(samples)[source]#

Evaluate model with current set of input samples.

Parameters:: samples (np.ndarray) – Input samples
Returns:: dict – Results corresponding to current set of input samples

abstract grad(samples, upstream_gradient)[source]#

Evaluate gradient of model w.r.t. current set of input samples.

Parameters:

samples (np.array) – Input samples
upstream_gradient (np.array) – Upstream gradient function evaluated at input samples \(\frac{\partial g}{\partial f}\)

Returns:

gradient (np.array) – Gradient w.r.t. current set of input samples \(\frac{\partial g}{\partial f} \frac{df}{dx}\)

abstract predict(x_test, support='y')[source]#: Predict.

abstract setup(x_train, y_train)[source]#

Setup surrogate model.

Parameters:

x_train (np.array) – training inputs
y_train (np.array) – training outputs

abstract train()[source]#: Train surrogate model.