queens.variational_distributions package#

Variational distributions.

Modules containing probability distributions for variational inference.

Submodules#

queens.variational_distributions.full_rank_normal module#

Full-Rank Normal Variational Distribution.

class FullRankNormal[source]#

Bases: Variational

Full-rank multivariate normal distribution.

Uses the parameterization (as in [1]) \(parameters=[\mu, \lambda]\), where \(\mu\) are the mean values and \(\lambda\) is an array containing the nonzero entries of the lower Cholesky decomposition of the covariance matrix \(L\): \(\lambda=[L_{00},L_{10},L_{11},L_{20},L_{21},L_{22}, ...]\). This allows the parameters \(\lambda\) to be unconstrained.

References

[1]: Kucukelbir, Alp, et al. “Automatic differentiation variational inference.”: The Journal of Machine Learning Research 18.1 (2017): 430-474.

n_parameters#: Number of parameters used in the parameterization.

__init__(dimension)[source]#

Initialize variational distribution.

Parameters:: dimension (NDims) – Dimension of the RV
Return type:: None

conduct_reparameterization(variational_parameters, n_samples)[source]#

Conduct a reparameterization.

Parameters:

variational_parameters (ndarray[tuple[NParams], dtype[floating]]) – Array with variational parameters
n_samples (NSamples) – Number of samples for current batch

Returns:

Actual samples from the variational distribution
Standard normal distributed samples used for the reparameterization

Return type:

tuple[ndarray[tuple[NSamples, NDims], dtype[floating]], ndarray[tuple[NSamples, NDims], dtype[floating]]]

construct_variational_parameters(mean, covariance)[source]#

Construct the variational parameters from mean and covariance.

Parameters:

mean (ndarray[tuple[NDims, Literal[1]], ~numpy.dtype[~numpy.floating]] | ~numpy.ndarray[tuple[~queens.variational_distributions._variational_distribution.NDims], ~numpy.dtype[~numpy.floating]]) – Mean values of the distribution
covariance (ndarray[tuple[NDims, NDims], dtype[floating]]) – Covariance matrix of the distribution

Returns:

Variational parameters

Return type:

ndarray[tuple[NParams], dtype[floating]]

draw(variational_parameters, n_draws)[source]#

Draw n_draw samples from the variational distribution.

Parameters:

variational_parameters (ndarray[tuple[NParams], dtype[floating]]) – Variational parameters
n_draws (NSamples) – Number of samples to draw

Returns:

Samples

Return type:

ndarray[tuple[NSamples, NDims], dtype[floating]]

export_dict(variational_parameters)[source]#

Create a dict of the distribution based on the given parameters.

Parameters:: variational_parameters (ndarray[tuple[NParams], dtype[floating]]) – Variational parameters
Returns:: Dictionary containing distribution information
Return type:: dict

fisher_information_matrix(variational_parameters)[source]#

Compute the Fisher information matrix analytically.

Parameters:: variational_parameters (ndarray[tuple[NParams], dtype[floating]]) – Variational parameters
Returns:: Fisher information matrix
Return type:: ndarray[tuple[NParams, NParams], dtype[floating]]

grad_params_logpdf(variational_parameters, x)[source]#

Log-PDF gradient w.r.t. to the variational parameters.

Evaluated at samples x. Also known as the score function.

Parameters:

variational_parameters (ndarray[tuple[NParams], dtype[floating]]) – Variational parameters
x (ndarray[tuple[NSamples, NDims], dtype[floating]]) – Row-wise samples

Returns:

Column-wise scores

Return type:

ndarray[tuple[NParams, NSamples], dtype[floating]]

grad_params_reparameterization(variational_parameters, standard_normal_sample_batch, upstream_gradient)[source]#

Calculate the gradient of the reparameterization.

Parameters:

variational_parameters (ndarray[tuple[NParams], dtype[floating]]) – Variational parameters
standard_normal_sample_batch (ndarray[tuple[NSamples, NDims], dtype[floating]]) – Standard normal distributed sample batch
upstream_gradient (ndarray[tuple[NSamples, NDims], dtype[floating]]) – Upstream gradient

Returns:

Gradient of the upstream function w.r.t. the variational parameters.

Return type:

ndarray[tuple[NSamples, NParams], dtype[floating]]

Note:

We assume that grad_reconstruct_params is a row-vector containing the partial derivatives of the reconstruction mapping of the actual distribution parameters w.r.t. the variational parameters.

The variable jacobi_parameters is the (n_parameters \(\times\) dim_sample) Jacobi matrix of the reparameterization w.r.t. the distribution parameters, with differentiating after the distribution parameters in different rows and different output dimensions of the sample per column.

grad_sample_logpdf(variational_parameters, sample_batch)[source]#

Computes the gradient of the log-PDF w.r.t. to the sample x.

Parameters:

variational_parameters (ndarray[tuple[NParams], dtype[floating]]) – Variational parameters
sample_batch (ndarray[tuple[NSamples, NDims], dtype[floating]]) – Row-wise samples

Returns:

Gradients of the log-pdf w.r.t. the sample *x*. The first dimension of the array – corresponds to the different samples. The second dimension to different dimensions within one sample.

Return type:

ndarray[tuple[NSamples, NDims], dtype[floating]]

initialize_variational_parameters(random=False)[source]#

Initialize variational parameters.

Default initialization:: \(\mu=0\) and \(L=diag(1)\) where \(\Sigma=LL^T\)
Random intialization:: \(\mu=Uniform(-0.1,0.1)\) \(L=diag(Uniform(0.9,1.1))\) where \(\Sigma=LL^T\)

Parameters:: random (bool) – If True, a random initialization is used. Otherwise the default is selected.
Returns:: Variational parameters
Return type:: ndarray[tuple[NParams], dtype[floating]]

logpdf(variational_parameters, x)[source]#

Log-PDF evaluated at the samples x.

Parameters:

variational_parameters (ndarray[tuple[NParams], dtype[floating]]) – Variational parameters
x (ndarray[tuple[NSamples, NDims], dtype[floating]]) – Row-wise samples

Returns:

Log-PDF values

Return type:

ndarray[tuple[NSamples], dtype[floating]]

pdf(variational_parameters, x)[source]#

PDF of evaluated at given samples x.

First computes the log-PDF, which is numerically more stable for exponential distributions.

Parameters:

variational_parameters (ndarray[tuple[NParams], dtype[floating]]) – Variational parameters
x (ndarray[tuple[NSamples, NDims], dtype[floating]]) – Row-wise samples

Returns:

Row vector of the PDF values

Return type:

ndarray[tuple[NSamples], dtype[floating]]

reconstruct_distribution_parameters(variational_parameters)[source]#

Reconstruct mean value and covariance.

Parameters:

variational_parameters (ndarray[tuple[NParams], dtype[floating]]) – Variational parameters

Returns:

Mean value of the distribution
Covariance of the distribution

Return type:

reconstruct_distribution_parameters_with_cholesky(variational_parameters)[source]#

Reconstruct mean value, covariance and its Cholesky decomposition.

Parameters:

variational_parameters (ndarray[tuple[NParams], dtype[floating]]) – Variational parameters

Returns:

Mean value of the distribution
Covariance of the distribution
Cholesky decomposition of the covariance matrix

Return type:

tuple[ndarray[tuple[NDims, Literal[1]], ~numpy.dtype[~numpy.floating]], ~numpy.ndarray[tuple[~queens.variational_distributions._variational_distribution.NDims, ~queens.variational_distributions._variational_distribution.NDims], ~numpy.dtype[~numpy.floating]], ~numpy.ndarray[tuple[~queens.variational_distributions._variational_distribution.NDims, ~queens.variational_distributions._variational_distribution.NDims], ~numpy.dtype[~numpy.floating]]]

total_grad_params_logpdf(variational_parameters, standard_normal_sample_batch)[source]#

Total log-PDF reparameterization gradient.

Total log-PDF reparameterization gradient w.r.t. the variational parameters.

Parameters:

variational_parameters (ndarray[tuple[NParams], dtype[floating]]) – Variational parameters
standard_normal_sample_batch (ndarray[tuple[NSamples, NDims], dtype[floating]]) – Standard normal distributed sample batch

Returns:

Total log-PDF reparameterization gradient

Return type:

ndarray[tuple[NSamples, NParams], dtype[floating]]

queens.variational_distributions.joint module#

Joint Variational Distribution.

class Joint[source]#

Bases: Variational, Generic[V]

Joint variational distribution class.

This distribution allows to join distributions in an independent fashion: \(q(\theta|\lambda)=\prod_{i=1}^{N}q_i(\theta_i | \lambda_i)\)

NOTE: \(q_i(\theta_i | \lambda_i)\) can be multivariate or of different families. Hence it is a generalization of the mean field distribution

distributions#: Variational distribution objects for the different independent distributions.

n_parameters#: Total number of parameters used in the parameterization.

distributions_n_parameters#: Number of parameters per distribution

distributions_dimension#: Number of dimension per distribution

__init__(distributions, dimension)[source]#

Initialize joint distribution.

Parameters:

distributions (list[V]) – Variational distribution objects for the different independent distributions.
dimension (NDims) – Dimension of the random variable

Return type:

None

construct_variational_parameters(distributions_parameters)[source]#

Construct the variational parameters from the distribution list.

Parameters:: distributions_parameters (list) – Parameters of the distributions
Returns:: Variational parameters
Return type:: ndarray[tuple[NParams], dtype[floating]]

draw(variational_parameters, n_draws)[source]#

Draw n_draw samples from the variational distribution.

Parameters:

variational_parameters (ndarray[tuple[NParams], dtype[floating]]) – Variational parameters
n_draws (NSamples) – Number of samples to draw

Returns:

Samples

Return type:

ndarray[tuple[NSamples, NDims], dtype[floating]]

export_dict(variational_parameters)[source]#

Create a dict of the distribution based on the given parameters.

Parameters:: variational_parameters (ndarray[tuple[NParams], dtype[floating]]) – Variational parameters
Returns:: Dictionary containing distribution information
Return type:: dict

fisher_information_matrix(variational_parameters)[source]#

Approximate the Fisher information matrix using Monte Carlo.

Parameters:: variational_parameters (ndarray[tuple[NParams], dtype[floating]]) – Variational parameters
Returns:: Fisher information matrix
Return type:: ndarray[tuple[NParams, NParams], dtype[floating]]

grad_params_logpdf(variational_parameters, x)[source]#

Log-PDF gradient w.r.t. the variational parameters.

Evaluated at samples x. Also known as the score function. Is a general implementation using the score functions of the components.

Parameters:

variational_parameters (ndarray[tuple[NParams], dtype[floating]]) – Variational parameters
x (ndarray[tuple[NSamples, NDims], dtype[floating]]) – Row-wise samples

Returns:

Column-wise scores

Return type:

ndarray[tuple[NParams, NSamples], dtype[floating]]

initialize_variational_parameters(random=False)[source]#

Initialize variational parameters.

The distribution initialization is handle by the component itself.

Parameters:: random (bool) – If True, a random initialization is used. Otherwise the default is selected
Returns:: Variational parameters
Return type:: ndarray[tuple[NParams], dtype[floating]]

logpdf(variational_parameters, x)[source]#

Log-PDF evaluated using the variational parameters at samples x.

Parameters:

variational_parameters (ndarray[tuple[NParams], dtype[floating]]) – Variational parameters
x (ndarray[tuple[NSamples, NDims], dtype[floating]]) – Row-wise samples

Returns:

Row vector of the Log-PDF values

Return type:

ndarray[tuple[NSamples], dtype[floating]]

pdf(variational_parameters, x)[source]#

Pdf evaluated using the variational parameters at given samples x.

Parameters:

variational_parameters (ndarray[tuple[NParams], dtype[floating]]) – Variational parameters
x (ndarray[tuple[NSamples, NDims], dtype[floating]]) – Row-wise samples

Returns:

Row vector of the PDF values

Return type:

ndarray[tuple[NSamples], dtype[floating]]

reconstruct_distribution_parameters(variational_parameters)[source]#

Reconstruct the parameters of distributions.

The list is nested, each entry correspond to the parameters of a distribution.

Parameters:: variational_parameters (ndarray[tuple[NParams], dtype[floating]]) – Variational parameters
Returns:: Distribution parameters of the components
Return type:: list[list[tuple[list | ndarray]]]

split_array_by_chunk_sizes(array, chunk_sizes)[source]#

Split up array by a list of chunk sizes.

Parameters:

array (ndarray) – Array to be split
chunk_sizes (_SupportsArray[dtype[Any]] | _NestedSequence[_SupportsArray[dtype[Any]]] | bool | int | float | complex | str | bytes | _NestedSequence[bool | int | float | complex | str | bytes]) – Chunk sizes

Returns:

Chunks of the array

Return type:

list

queens.variational_distributions.mean_field_normal module#

Mean-Field Normal Variational Distribution.

class MeanFieldNormal[source]#

Bases: Variational

Mean field multivariate normal distribution.

Uses the parameterization (as in [1]): \(parameters=[\mu, \lambda]\) where \(\mu\) are the mean values and \(\sigma^2=exp(2 \lambda)\) the variances allowing for \(\lambda\) to be unconstrained.

References

[1]: Kucukelbir, Alp, et al. “Automatic differentiation variational inference.”: The Journal of Machine Learning Research 18.1 (2017): 430-474.

n_parameters#: Number of parameters used in the parameterization.

__init__(dimension)[source]#

Initialize variational distribution.

Parameters:: dimension (NDims) – Dimension of random variable.
Return type:: None

conduct_reparameterization(variational_parameters, n_samples)[source]#

Conduct a reparameterization.

Parameters:

variational_parameters (ndarray[tuple[NParams], dtype[floating]]) – Array with variational parameters
n_samples (NSamples) – Number of samples for current batch

Returns:

Actual samples from the variational distribution
Standard normal distributed sample batch

Return type:

tuple[ndarray[tuple[NSamples, NDims], dtype[floating]], ndarray[tuple[NSamples, NDims], dtype[floating]]]

construct_variational_parameters(mean, covariance)[source]#

Construct the variational parameters from mean and covariance.

Parameters:

mean (ndarray[tuple[NDims, Literal[1]], ~numpy.dtype[~numpy.floating]] | ~numpy.ndarray[tuple[~queens.variational_distributions._variational_distribution.NDims], ~numpy.dtype[~numpy.floating]]) – Mean values of the distribution
covariance (ndarray[tuple[NDims, NDims], dtype[floating]]) – Covariance matrix of the distribution

Returns:

Variational parameters

Return type:

ndarray[tuple[NParams], dtype[floating]]

draw(variational_parameters, n_draws)[source]#

Draw n_draw samples from the variational distribution.

Parameters:

variational_parameters (ndarray[tuple[NParams], dtype[floating]]) – Variational parameters
n_draws (NSamples) – Number of samples to draw

Returns:

Samples

Return type:

ndarray[tuple[NSamples, NDims], dtype[floating]]

export_dict(variational_parameters)[source]#

Create a dict of the distribution based on the given parameters.

Parameters:: variational_parameters (ndarray[tuple[NParams], dtype[floating]]) – Variational parameters
Returns:: Dictionary containing distribution information
Return type:: dict

fisher_information_matrix(variational_parameters)[source]#

Compute the Fisher information matrix analytically.

Parameters:: variational_parameters (ndarray[tuple[NParams], dtype[floating]]) – Variational parameters
Returns:: Fisher information matrix
Return type:: ndarray[tuple[NParams, NParams], dtype[floating]]

grad_params_logpdf(variational_parameters, x)[source]#

Log-PDF gradient w.r.t. the variational parameters.

Evaluated at samples x. Also known as the score function.

Parameters:

variational_parameters (ndarray[tuple[NParams], dtype[floating]]) – Variational parameters
x (ndarray[tuple[NSamples, NDims], dtype[floating]]) – Row-wise samples

Returns:

Column-wise scores

Return type:

ndarray[tuple[NParams, NSamples], dtype[floating]]

grad_params_reparameterization(variational_parameters, standard_normal_sample_batch, upstream_gradient)[source]#

Calculate the gradient of the reparameterization.

Parameters:

variational_parameters (ndarray[tuple[NParams], dtype[floating]]) – Variational parameters
standard_normal_sample_batch (ndarray[tuple[NSamples, NDims], dtype[floating]]) – Standard normal distributed sample batch
upstream_gradient (ndarray[tuple[NSamples, NDims], dtype[floating]]) – Upstream gradient

Returns:

Gradient of the upstream function w.r.t. the variational parameters.

Return type:

ndarray[tuple[NSamples, NParams], dtype[floating]]

Note

We assume that grad_reconstruct_params is a row-vector containing the partial derivatives of the reconstruction mapping of the actual distribution parameters w.r.t. the variational parameters.

grad_sample_logpdf(variational_parameters, sample_batch)[source]#

Computes the gradient of the log-PDF w.r.t. to the sample x.

Parameters:

variational_parameters (ndarray[tuple[NParams], dtype[floating]]) – Variational parameters
sample_batch (ndarray[tuple[NSamples, NDims], dtype[floating]]) – Row-wise samples

Returns:

Gradients of the log-PDF w.r.t. the sample *x*. The first dimension of the array – corresponds to the different samples. The second dimension to different dimensions within one sample.

Return type:

ndarray[tuple[NSamples, NDims], dtype[floating]]

initialize_variational_parameters(random=False)[source]#

Initialize variational parameters.

Default initialization:: \(\mu=0\) and \(\sigma^2=1\)
Random intialization:: \(\mu=Uniform(-0.1,0.1)\) and \(\sigma^2=Uniform(0.9,1.1)\)

Parameters:: random (bool) – If True, a random initialization is used. Otherwise the default is selected
Returns:: Variational parameters
Return type:: ndarray[tuple[NParams], dtype[floating]]

logpdf(variational_parameters, x)[source]#

Log-PDF evaluated using the variational parameters at samples x.

Parameters:

variational_parameters (ndarray[tuple[NParams], dtype[floating]]) – Variational parameters
x (ndarray[tuple[NSamples, NDims], dtype[floating]]) – Row-wise samples

Returns:

Row vector of the Log-PDF values

Return type:

ndarray[tuple[NSamples], dtype[floating]]

pdf(variational_parameters, x)[source]#

PDF of the variational distribution evaluated at samples x.

First computes the log-PDF, which is numerically more stable for exponential distributions.

Parameters:

variational_parameters (ndarray[tuple[NParams], dtype[floating]]) – Variational parameters
x (ndarray[tuple[NSamples, NDims], dtype[floating]]) – Row-wise samples

Returns:

Row vector of the PDF values

Return type:

ndarray[tuple[NSamples], dtype[floating]]

reconstruct_distribution_parameters(variational_parameters)[source]#

Reconstruct mean and covariance from the variational parameters.

Parameters:

variational_parameters (ndarray[tuple[NParams], dtype[floating]]) – Variational parameters

Returns:

Mean value of the distribution
Covariance matrix of the distribution

Return type:

total_grad_params_logpdf(variational_parameters, standard_normal_sample_batch)[source]#

Total log-PDF reparameterization gradient.

Total log-PDF reparameterization gradient w.r.t. the variational parameters.

Parameters:

variational_parameters (ndarray[tuple[NParams], dtype[floating]]) – Variational parameters
standard_normal_sample_batch (ndarray[tuple[NSamples, NDims], dtype[floating]]) – Standard normal distributed sample batch

Returns:

Total log-PDF reparameterization gradient

Return type:

ndarray[tuple[NSamples, NParams], dtype[floating]]

queens.variational_distributions.mixture_model module#

Mixture Model Variational Distribution.

class MixtureModel[source]#

Bases: Variational, Generic[V]

Mixture model variational distribution class.

Every component is a member of the same distribution family. Uses the parameterization: \(parameters=[\lambda_0,\lambda_1,...,\lambda_{C},\lambda_{weights}]\) where \(C\) is the number of components, \(\\lambda_i\) are the variational parameters of the ith component and \(\\lambda_{weights}\) parameters such that the component weights are obtained by: \(weight_i=\frac{exp(\lambda_{weights,i})}{\sum_{j=1}^{C}exp(\lambda_{weights,j})}\)

This allows the weight parameters \(\lambda_{weights}\) to be unconstrained.

n_components#: Number of mixture components.

base_distribution#: Variational distribution object for the components.

n_parameters#: Number of parameters used in the parameterization.

__init__(base_distribution, dimension, n_components)[source]#

Initialize mixture model.

Parameters:

base_distribution (V) – Variational distribution object for the components
dimension (NDims) – Dimension of the random variable
n_components (NComponents) – Number of mixture components

Return type:

None

construct_variational_parameters(parameters_per_component, weights)[source]#

Construct the variational parameters from the probabilities.

Parameters:

parameters_per_component (list[Iterable[ndarray]]) – Distribution parameters per component
weights (ndarray[tuple[NComponents], dtype[floating]]) – Probabilities of the distribution

Returns:

Variational parameters

Return type:

ndarray[tuple[NParams], dtype[floating]]

draw(variational_parameters, n_draws)[source]#

Draw n_draw samples from the variational distribution.

Uses a two-step process:

From a multinomial distribution, based on the weights, select a component
Sample from the selected component

Parameters:

variational_parameters (ndarray[tuple[NParams], dtype[floating]]) – Variational parameters
n_draws (NSamples) – Number of samples to draw

Returns:

Samples

Return type:

ndarray[tuple[NSamples, NDims], dtype[floating]]

export_dict(variational_parameters)[source]#

Create a dict of the distribution based on the given parameters.

Parameters:: variational_parameters (ndarray[tuple[NParams], dtype[floating]]) – Variational parameters
Returns:: Dictionary containing distribution information
Return type:: dict

fisher_information_matrix(variational_parameters, n_samples=10000)[source]#

Approximate the Fisher information matrix using Monte Carlo.

Parameters:

variational_parameters (ndarray[tuple[NParams], dtype[floating]]) – Variational parameters
n_samples (int) – Number of samples for a MC FIM estimation

Returns:

Fisher information matrix

Return type:

ndarray[tuple[NParams, NParams], dtype[floating]]

grad_params_logpdf(variational_parameters, x)[source]#

Log-PDF gradient w.r.t. the variational parameters.

Evaluated at samples x. Also known as the score function. Is a general implementation using the score functions of the components.

Parameters:

variational_parameters (ndarray[tuple[NParams], dtype[floating]]) – Variational parameters
x (ndarray[tuple[NSamples, NDims], dtype[floating]]) – Row-wise samples

Returns:

Column-wise scores

Return type:

ndarray[tuple[NParams, NSamples], dtype[floating]]

initialize_variational_parameters(random=False)[source]#

Initialize variational parameters.

Default weights initialization:: \(w_i=\frac{1}{N_\text{sample space}}\)
Random weights intialization:: \(w_i=\frac{s}{N_\text{experiments}}\) where \(s\) is a sample of a multinomial distribution with \(N_\text{experiments}\)

The component initialization is handle by the component itself.

Parameters:: random (bool) – If True, a random initialization is used. Otherwise the default is selected
Returns:: Variational parameters
Return type:: ndarray[tuple[NParams], dtype[floating]]

logpdf(variational_parameters, x)[source]#

Log-PDF evaluated using the variational parameters at samples x.

Is a general implementation using the log-PDF function of the components. Uses the log-sum-exp trick [1] in order to reduce floating point issues.

References

[1] David M. Blei, Alp Kucukelbir & Jon D. McAuliffe (2017) Variational Inference: A Review for Statisticians, Journal of the American Statistical Association, 112:518

Parameters:

variational_parameters (ndarray[tuple[NParams], dtype[floating]]) – Variational parameters
x (ndarray[tuple[NSamples, NDims], dtype[floating]]) – Row-wise samples

Returns:

Row vector of the Log-PDF values

Return type:

ndarray[tuple[NSamples], dtype[floating]]

pdf(variational_parameters, x)[source]#

Pdf evaluated using the variational parameters at given samples x.

Parameters:

variational_parameters (ndarray[tuple[NParams], dtype[floating]]) – Variational parameters
x (ndarray[tuple[NSamples, NDims], dtype[floating]]) – Row-wise samples

Returns:

Row vector of the PDF values

Return type:

ndarray[tuple[NSamples], dtype[floating]]

reconstruct_distribution_parameters(variational_parameters)[source]#

Reconstruct the weights and parameters of the mixture components.

The list is nested, each entry correspond to the parameters of a component.

Parameters:

variational_parameters (ndarray[tuple[NParams], dtype[floating]]) – Variational parameters

Returns:

Distribution parameters of the components
Weights of the mixture

Return type:

tuple[list[tuple[list | ndarray]], ndarray[tuple[NComponents], dtype[floating]]]

queens.variational_distributions.particle module#

Particle Variational Distribution.

class Particle[source]#

Bases: Variational

Variational distribution for particle distributions.

The probabilities of the distribution are parameterized by softmax: \(p_i=p(\lambda_i)=\frac{\exp(\lambda_i)}{\sum_k exp(\lambda_k)}\)

particles_obj#: Particle distribution object

dimension#: Number of random variables

__init__(sample_space)[source]#

Initialize variational distribution.

Parameters:: sample_space (ndarray | Sequence[Sized]) – Sample space of the variational distribution
Return type:: None

construct_variational_parameters(probabilities, sample_space)[source]#

Construct the variational parameters from the probabilities.

Parameters:

probabilities (ndarray[tuple[NParams], dtype[floating]]) – Probabilities of the distribution
sample_space (ndarray | Sequence[Sized]) – Sample space of the distribution

Returns:

Variational parameters

Return type:

ndarray[tuple[NParams], dtype[floating]]

draw(variational_parameters, n_draws)[source]#

Draw n_draws samples from distribution.

Parameters:

variational_parameters (ndarray[tuple[NParams], dtype[floating]]) – Variational parameters of the distribution
n_draws (NSamples) – Number of samples

Returns:

Samples

Return type:

ndarray[tuple[NSamples, NDims], dtype[floating]]

export_dict(variational_parameters)[source]#

Create a dict of the distribution based on the given parameters.

Parameters:: variational_parameters (ndarray[tuple[NParams], dtype[floating]]) – Variational parameters
Returns:: Dictionary containing distribution information
Return type:: dict

fisher_information_matrix(variational_parameters)[source]#

Compute the Fisher information matrix.

For the given parameterization, the Fisher information yields: \(\text{FIM}_{ij}=\delta_{ij} p_i -p_i p_j\)

Parameters:: variational_parameters (ndarray[tuple[NParams], dtype[floating]]) – Variational parameters of the distribution
Returns:: Fisher information matrix
Return type:: ndarray[tuple[NParams, NParams], dtype[floating]]

grad_params_logpdf(variational_parameters, x)[source]#

Log-PDF gradient w.r.t. the variational parameters.

Evaluated at samples x. Also known as the score function.

For the given parameterization, the score function yields: \(\nabla_{\lambda_i}\ln p(\theta_j | \lambda)=\delta_{ij}-p_i\)

Parameters:

variational_parameters (ndarray[tuple[NParams], dtype[floating]]) – Variational parameters of the distribution
x (ndarray[tuple[NSamples, NDims], dtype[floating]]) – Locations at which to evaluate the distribution

Returns:

Score functions at the locations x

Return type:

ndarray[tuple[NParams, NSamples], dtype[floating]]

initialize_variational_parameters(random=False)[source]#

Initialize variational parameters.

Default initialization:: \(w_i=\frac{1}{N_\text{sample space}}\)
Random intialization:: \(w_i=\frac{s}{N_\text{experiments}}\) where \(s\) is a sample of a multinomial distribution with \(N_\text{experiments}\)

Parameters:: random (bool) – If True, a random initialization is used. Otherwise the default is selected
Returns:: Variational parameters
Return type:: ndarray[tuple[NParams], dtype[floating]]

logpdf(variational_parameters, x)[source]#

Evaluate the natural logarithm of the PDF.

Parameters:

variational_parameters (ndarray[tuple[NParams], dtype[floating]]) – Variational parameters of the distribution
x (ndarray[tuple[NSamples, NDims], dtype[floating]]) – Locations at which to evaluate the distribution

Returns:

Log-PDF values at the locations x

Return type:

ndarray[tuple[NSamples], dtype[floating]]

pdf(variational_parameters, x)[source]#

Evaluate the probability density function (PDF).

Parameters:

variational_parameters (ndarray[tuple[NParams], dtype[floating]]) – Variational parameters of the distribution
x (ndarray[tuple[NSamples, NDims], dtype[floating]]) – Locations at which to evaluate the distribution

Returns:

Row vector of the PDF values

Return type:

ndarray[tuple[NSamples], dtype[floating]]

reconstruct_distribution_parameters(variational_parameters)[source]#

Reconstruct probabilities from the variational parameters.

Parameters:

variational_parameters (ndarray[tuple[NParams], dtype[floating]]) – Variational parameters

Returns:

Probabilities of the distribution
Sample space of the distribution

Return type:

tuple[ndarray[tuple[NParams], dtype[floating]], ndarray]