Skip to content

Index

Modelling

# DynamicPPL.@modelMacro.

@model(expr[, warn = true])

Macro to specify a probabilistic model.

If warn is true, a warning is displayed if internal variable names are used in the model definition.

Examples

Model definition:

@model function model(x, y = 42)
    ...
end

To generate a Model, call model(xvalue) or model(xvalue, yvalue).

Samplers

# DynamicPPL.SamplerType.

Sampler{T}

Generic interface for implementing inference algorithms. An implementation of an algorithm should include the following:

  1. A type specifying the algorithm and its parameters, derived from InferenceAlgorithm
  2. A method of sample function that produces results of inference, which is where actual inference happens.

DynamicPPL translates models to chunks that call the modelling functions at specified points. The dispatch is based on the value of a sampler variable. To include a new inference algorithm implements the requirements mentioned above in a separate file, then include that file at the end of this one.

# Turing.Inference.GibbsType.

Gibbs(algs...)

Compositional MCMC interface. Gibbs sampling combines one or more sampling algorithms, each of which samples from a different set of variables in a model.

Example:

@model gibbs_example(x) = begin
    v1 ~ Normal(0,1)
    v2 ~ Categorical(5)
end

Use PG for a ‘v2’ variable, and use HMC for the ‘v1’ variable.

Note that v2 is discrete, so the PG sampler is more appropriate

than is HMC.

alg = Gibbs(HMC(0.2, 3, :v1), PG(20, :v2)) ```

Tips:

  • HMC and NUTS are fast samplers, and can throw off particle-based

methods like Particle Gibbs. You can increase the effectiveness of particle sampling by including more particles in the particle sampler.

source

# Turing.Inference.HMCType.

HMC(ϵ::Float64, n_leapfrog::Int)

Hamiltonian Monte Carlo sampler with static trajectory.

Arguments:

  • ϵ::Float64 : The leapfrog step size to use.
  • n_leapfrog::Int : The number of leapfrop steps to use.

Usage:

HMC(0.05, 10)

Tips:

  • If you are receiving gradient errors when using HMC, try reducing the leapfrog step size ϵ, e.g.
# Original step size
sample(gdemo([1.5, 2]), HMC(0.1, 10), 1000)

# Reduced step size
sample(gdemo([1.5, 2]), HMC(0.01, 10), 1000)

source

# Turing.Inference.HMCDAType.

HMCDA(n_adapts::Int, δ::Float64, λ::Float64; ϵ::Float64=0.0)

Hamiltonian Monte Carlo sampler with Dual Averaging algorithm.

Usage:

HMCDA(200, 0.65, 0.3)

Arguments:

  • n_adapts::Int : Numbers of samples to use for adaptation.
  • δ::Float64 : Target acceptance rate. 65% is often recommended.
  • λ::Float64 : Target leapfrop length.
  • ϵ::Float64=0.0 : Inital step size; 0 means automatically search by Turing.

For more information, please view the following paper (arXiv link):

  • Hoffman, Matthew D., and Andrew Gelman. “The No-U-turn sampler: adaptively setting path lengths in Hamiltonian Monte Carlo.” Journal of Machine Learning Research 15, no. 1 (2014): 1593-1623.

source

# Turing.Inference.ISType.

IS()

Importance sampling algorithm.

Note that this method is particle-based, and arrays of variables must be stored in a TArray object.

Usage:

IS()

Example:

# Define a simple Normal model with unknown mean and variance.
@model gdemo(x) = begin
    s ~ InverseGamma(2,3)
    m ~ Normal(0,sqrt.(s))
    x[1] ~ Normal(m, sqrt.(s))
    x[2] ~ Normal(m, sqrt.(s))
    return s, m
end

sample(gdemo([1.5, 2]), IS(), 1000)

source

# Turing.Inference.MHType.

MH(space...)

Construct a Metropolis-Hastings algorithm.

The arguments space can be

  • Blank (i.e. MH()), in which case MH defaults to using the prior for each parameter as the proposal distribution.
  • A set of one or more symbols to sample with MH in conjunction with Gibbs, i.e. Gibbs(MH(:m), PG(10, :s))
  • An iterable of pairs or tuples mapping a Symbol to a AdvancedMH.Proposal, Distribution, or Function that generates returns a conditional proposal distribution.
  • A covariance matrix to use as for mean-zero multivariate normal proposals.

Examples

The default MH will use propose samples from the prior distribution using AdvancedMH.StaticProposal.

@model function gdemo(x, y)
    s ~ InverseGamma(2,3)
    m ~ Normal(0, sqrt(s))
    x ~ Normal(m, sqrt(s))
    y ~ Normal(m, sqrt(s))
end

chain = sample(gdemo(1.5, 2.0), MH(), 1_000)
mean(chain)

Alternatively, you can specify particular parameters to sample if you want to combine sampling from multiple samplers:

@model function gdemo(x, y)
    s ~ InverseGamma(2,3)
    m ~ Normal(0, sqrt(s))
    x ~ Normal(m, sqrt(s))
    y ~ Normal(m, sqrt(s))
end

# Samples s with MH and m with PG
chain = sample(gdemo(1.5, 2.0), Gibbs(MH(:s), PG(10, :m)), 1_000)
mean(chain)

Using custom distributions defaults to using static MH:

@model function gdemo(x, y)
    s ~ InverseGamma(2,3)
    m ~ Normal(0, sqrt(s))
    x ~ Normal(m, sqrt(s))
    y ~ Normal(m, sqrt(s))
end

# Use a static proposal for s and random walk with proposal 
# standard deviation of 0.25 for m.
chain = sample(
    gdemo(1.5, 2.0), 
    MH(
        :s => InverseGamma(2, 3),
        :m => Normal(0, 1)
    ), 
    1_000
)
mean(chain)

Specifying explicit proposals using the AdvancedMH interface:

@model function gdemo(x, y)
    s ~ InverseGamma(2,3)
    m ~ Normal(0, sqrt(s))
    x ~ Normal(m, sqrt(s))
    y ~ Normal(m, sqrt(s))
end

# Use a static proposal for s and random walk with proposal 
# standard deviation of 0.25 for m.
chain = sample(
    gdemo(1.5, 2.0), 
    MH(
        :s => AdvancedMH.StaticProposal(InverseGamma(2,3)),
        :m => AdvancedMH.RandomWalkProposal(Normal(0, 0.25))
    ), 
    1_000
)
mean(chain)

Using a custom function to specify a conditional distribution:

@model function gdemo(x, y)
    s ~ InverseGamma(2,3)
    m ~ Normal(0, sqrt(s))
    x ~ Normal(m, sqrt(s))
    y ~ Normal(m, sqrt(s))
end

# Use a static proposal for s and and a conditional proposal for m,
# where the proposal is centered around the current sample.
chain = sample(
    gdemo(1.5, 2.0), 
    MH(
        :s => InverseGamma(2, 3),
        :m => x -> Normal(x, 1)
    ), 
    1_000
)
mean(chain)

Providing a covariance matrix will cause MH to perform random-walk sampling in the transformed space with proposals drawn from a multivariate normal distribution. The provided matrix must be positive semi-definite and square. Usage:

@model function gdemo(x, y)
    s ~ InverseGamma(2,3)
    m ~ Normal(0, sqrt(s))
    x ~ Normal(m, sqrt(s))
    y ~ Normal(m, sqrt(s))
end

# Providing a custom variance-covariance matrix
chain = sample(
    gdemo(1.5, 2.0), 
    MH(
        [0.25 0.05; 
         0.05 0.50]
    ), 
    1_000
)
mean(chain)

source

# Turing.Inference.NUTSType.

NUTS(n_adapts::Int, δ::Float64; max_depth::Int=5, Δ_max::Float64=1000.0, ϵ::Float64=0.0)

No-U-Turn Sampler (NUTS) sampler.

Usage:

NUTS()            # Use default NUTS configuration.
NUTS(1000, 0.65)  # Use 1000 adaption steps, and target accept ratio 0.65.

Arguments:

  • n_adapts::Int : The number of samples to use with adaptation.
  • δ::Float64 : Target acceptance rate for dual averaging.
  • max_depth::Int : Maximum doubling tree depth.
  • Δ_max::Float64 : Maximum divergence during doubling tree.
  • init_ϵ::Float64 : Inital step size; 0 means automatically searching using a heuristic procedure.

source

# Turing.Inference.PGType.

struct PG{space, R} <: Turing.Inference.ParticleInference

Particle Gibbs sampler.

Note that this method is particle-based, and arrays of variables must be stored in a TArray object.

Fields

  • nparticles::Int64

    Number of particles.

  • resampler::Any

    Resampling algorithm.

source

# Turing.Inference.SMCType.

struct SMC{space, R} <: Turing.Inference.ParticleInference

Sequential Monte Carlo sampler.

Fields

  • resampler::Any

source

Distributions

# Turing.FlatType.

Flat()

The flat distribution is the improper distribution of real numbers that has the improper probability density function

\[f(x) = 1.\]

source

# Turing.FlatPosType.

FlatPos(l::Real)

The positive flat distribution with real-valued parameter l is the improper distribution of real numbers that has the improper probability density function

\[f(x) = \begin{cases} 0 & \text{if } x \leq l, \\ 1 & \text{otherwise}. \end{cases}\]

source

# Turing.BinomialLogitType.

BinomialLogit(n, logitp)

The Binomial distribution with logit parameterization characterizes the number of successes in a sequence of independent trials.

It has two parameters: n, the number of trials, and logitp, the logit of the probability of success in an individual trial, with the distribution

\[P(X = k) = {n \choose k}{(\text{logistic}(logitp))}^k (1 - \text{logistic}(logitp))^{n-k}, \quad \text{ for } k = 0,1,2, \ldots, n.\]

See also: Binomial

source

!!! warning “Missing docstring.” Missing docstring for VecBinomialLogit. Check Documenter’s build log for details.

# Turing.OrderedLogisticType.

OrderedLogistic(η, c::AbstractVector)

The ordered logistic distribution with real-valued parameter η and cutpoints c has the probability mass function

\[P(X = k) = \begin{cases} 1 - \text{logistic}(\eta - c_1) & \text{if } k = 1, \\ \text{logistic}(\eta - c_{k-1}) - \text{logistic}(\eta - c_k) & \text{if } 1 < k < K, \\ \text{logistic}(\eta - c_{K-1}) & \text{if } k = K, \end{cases}\]

where K = length(c) + 1.

source

Data Structures

# Libtask.TArrayType.

TArray{T}(dims, ...)

Implementation of data structures that automatically perform copy-on-write after task copying.

If current*task is an existing key in s, then return s[current*task]. Otherwise, returns[current*task] = s[last*task].

Usage:

TArray(dim)

Example:

ta = TArray(4)              # init
for i in 1:4 ta[i] = i end  # assign
Array(ta)                   # convert to 4-element Array{Int64,1}: [1, 2, 3, 4]

Utilities

# Libtask.tzerosFunction.

 tzeros(dims, ...)

Construct a distributed array of zeros. Trailing arguments are the same as those accepted by TArray.

tzeros(dim)

Example:

tz = tzeros(4)              # construct
Array(tz)                   # convert to 4-element Array{Int64,1}: [0, 0, 0, 0]