Library

Modelling

# Turing.Core.@modelMacro.

@model(body)

Macro to specify a probabilistic model.

Example:

Model definition:

@model model_generator(x = default_x, y) = begin
    ...
end

Expanded model definition

# Allows passing arguments as kwargs
model_generator(; x = nothing, y = nothing)) = model_generator(x, y)
function model_generator(x = nothing, y = nothing)
    pvars, dvars = Turing.get_vars(Tuple{:x, :y}, (x = x, y = y))
    data = Turing.get_data(dvars, (x = x, y = y))
    defaults = Turing.get_default_values(dvars, (x = default_x, y = nothing))

    inner_function(sampler::Turing.AbstractSampler, model) = inner_function(model)
    function inner_function(model)
        return inner_function(Turing.VarInfo(), Turing.SampleFromPrior(), model)
    end
    function inner_function(vi::Turing.VarInfo, model)
        return inner_function(vi, Turing.SampleFromPrior(), model)
    end
    # Define the main inner function
    function inner_function(vi::Turing.VarInfo, sampler::Turing.AbstractSampler, model)
        local x
        if isdefined(model.data, :x)
            if model.data.x isa Type && (model.data.x <: AbstractFloat || model.data.x <: AbstractArray)
                x = Turing.Core.get_matching_type(sampler, vi, model.data.x)
            else
                x = model.data.x
            end
        else
            x = model_defaults.x
        end
        local y
        if isdefined(model.data, :y)
            if model.data.y isa Type && (model.data.y <: AbstractFloat || model.data.y <: AbstractArray)
                y = Turing.Core.get_matching_type(sampler, vi, model.data.y)
            else
                y = model.data.y
            end
        else
            y = model.defaults.y
        end

        vi.logp = zero(Real)
        ...
    end
    model = Turing.Model{pvars, dvars}(inner_function, data, defaults)
    return model
end

Generating a model: model_generator(x_value)::Model.

source

Samplers

# Turing.SamplerType.

Sampler{T}

Generic interface for implementing inference algorithms. An implementation of an algorithm should include the following:

  1. A type specifying the algorithm and its parameters, derived from InferenceAlgorithm
  2. A method of sample function that produces results of inference, which is where actual inference happens.

Turing translates models to chunks that call the modelling functions at specified points. The dispatch is based on the value of a sampler variable. To include a new inference algorithm implements the requirements mentioned above in a separate file, then include that file at the end of this one.

source

# Turing.Inference.GibbsType.

Gibbs(n_iters, algs...)

Compositional MCMC interface. Gibbs sampling combines one or more sampling algorithms, each of which samples from a different set of variables in a model.

Example:

@model gibbs_example(x) = begin
    v1 ~ Normal(0,1)
    v2 ~ Categorical(5)
        ...
end

# Use PG for a 'v2' variable, and use HMC for the 'v1' variable.
# Note that v2 is discrete, so the PG sampler is more appropriate
# than is HMC.
alg = Gibbs(1000, HMC(1, 0.2, 3, :v1), PG(20, 1, :v2))

Tips:

  • HMC and NUTS are fast samplers, and can throw off particle-based

methods like Particle Gibbs. You can increase the effectiveness of particle sampling by including more particles in the particle sampler.

source

# Turing.Inference.HMCType.

HMC(n_iters::Int, ϵ::Float64, n_leapfrog::Int)

Hamiltonian Monte Carlo sampler.

Arguments:

  • n_iters::Int : The number of samples to pull.
  • ϵ::Float64 : The leapfrog step size to use.
  • n_leapfrog::Int : The number of leapfrop steps to use.

Usage:

HMC(1000, 0.05, 10)

Tips:

  • If you are receiving gradient errors when using HMC, try reducing the

step_size parameter.

# Original step_size
sample(gdemo([1.5, 2]), HMC(1000, 0.1, 10))

# Reduced step_size.
sample(gdemo([1.5, 2]), HMC(1000, 0.01, 10))

source

# Turing.Inference.HMCDAType.

HMCDA(n_iters::Int, n_adapts::Int, δ::Float64, λ::Float64; init_ϵ::Float64=0.1)

Hamiltonian Monte Carlo sampler with Dual Averaging algorithm.

Usage:

HMCDA(1000, 200, 0.65, 0.3)

Arguments:

  • n_iters::Int : Number of samples to pull.
  • n_adapts::Int : Numbers of samples to use for adaptation.
  • δ::Float64 : Target acceptance rate. 65% is often recommended.
  • λ::Float64 : Target leapfrop length.
  • init_ϵ::Float64=0.1 : Inital step size; 0 means automatically search by Turing.

For more information, please view the following paper (arXiv link):

  • Hoffman, Matthew D., and Andrew Gelman. “The No-U-turn sampler: adaptively setting path lengths in Hamiltonian Monte Carlo.” Journal of Machine Learning Research 15, no. 1 (2014): 1593-1623.

source

# Turing.Inference.IPMCMCType.

IPMCMC(n_particles::Int, n_iters::Int, n_nodes::Int, n_csmc_nodes::Int)

Particle Gibbs sampler.

Note that this method is particle-based, and arrays of variables must be stored in a TArray object.

Usage:

IPMCMC(100, 100, 4, 2)

Arguments:

  • n_particles::Int : Number of particles to use.
  • n_iters::Int : Number of iterations to employ.
  • n_nodes::Int : The number of nodes running SMC and CSMC.
  • n_csmc_nodes::Int : The number of CSMC nodes.

A paper on this can be found [here](https://arxiv.org/abs/1602.05128).


<a target='_blank' href='https://github.com/TuringLang/Turing.jl/blob/2f2ed6a8618156465781ddc5b8b41fa26e8c9ec0/src/inference/AdvancedSMC.jl#L469-L492' class='documenter-source'>source</a><br>

### <a id='Turing.Inference.IS' href='#Turing.Inference.IS'>#</a> **`Turing.Inference.IS`** &mdash; *Type*.


```julia
IS(n_particles::Int)

Importance sampling algorithm.

Note that this method is particle-based, and arrays of variables must be stored in a TArray object.

Arguments:

  • n_particles is the number of particles to use.

Usage:

IS(1000)

Example:

# Define a simple Normal model with unknown mean and variance.
@model gdemo(x) = begin
    s ~ InverseGamma(2,3)
    m ~ Normal(0,sqrt.(s))
    x[1] ~ Normal(m, sqrt.(s))
    x[2] ~ Normal(m, sqrt.(s))
    return s, m
end

sample(gdemo([1.5, 2]), IS(1000))

source

# Turing.Inference.MHType.

MH(n_iters::Int)

Metropolis-Hastings sampler.

Usage:

MH(100, (:m, (x) -> Normal(x, 0.1)))

Example:

# Define a simple Normal model with unknown mean and variance.
@model gdemo(x) = begin
  s ~ InverseGamma(2,3)
  m ~ Normal(0,sqrt(s))
  x[1] ~ Normal(m, sqrt(s))
  x[2] ~ Normal(m, sqrt(s))
  return s, m
end

chn = sample(gdemo([1.5, 2]), MH(1000))

source

# Turing.Inference.NUTSType.

NUTS(n_iters::Int, n_adapts::Int, δ::Float64; max_depth::Int=5, Δ_max::Float64=1000.0, init_ϵ::Float64=0.1)

No-U-Turn Sampler (NUTS) sampler.

Usage:

NUTS(1000, 200, 0.6j_max)

Arguments:

  • n_iters::Int : The number of samples to pull.
  • n_adapts::Int : The number of samples to use with adapatation.
  • δ::Float64 : Target acceptance rate.
  • max_depth::Float64 : Maximum doubling tree depth.
  • Δ_max::Float64 : Maximum divergence during doubling tree.
  • init_ϵ::Float64 : Inital step size; 0 means automatically search by Turing.

source

# Turing.Inference.PGType.

PG(n_particles::Int, n_iters::Int)

Particle Gibbs sampler.

Note that this method is particle-based, and arrays of variables must be stored in a TArray object.

Usage:

PG(100, 100)

source

# Turing.Inference.PMMHType.

PMMH(n_iters::Int, smc_alg:::SMC, parameters_algs::Tuple{MH})

Particle independant Metropolis–Hastings and Particle marginal Metropolis–Hastings samplers.

Note that this method is particle-based, and arrays of variables must be stored in a TArray object.

Usage:

alg = PMMH(100, SMC(20, :v1), MH(1,:v2))
alg = PMMH(100, SMC(20, :v1), MH(1,(:v2, (x) -> Normal(x, 1))))

Arguments:

  • n_iters::Int : Number of iterations to run.
  • smc_alg:::SMC : An SMC algorithm to use.
  • parameters_algs::Tuple{MH} : An MH algorithm, which includes a

sample space specification.

source

# Turing.Inference.SGHMCType.

SGHMC(n_iters::Int, learning_rate::Float64, momentum_decay::Float64)

Stochastic Gradient Hamiltonian Monte Carlo sampler.

Usage:

SGHMC(1000, 0.01, 0.1)

Arguments:

  • n_iters::Int : Number of samples to pull.
  • learning_rate::Float64 : The learning rate.
  • momentum_decay::Float64 : Momentum decay variable.

source

# Turing.Inference.SGLDType.

SGLD(n_iters::Int, ϵ::Float64)

Stochastic Gradient Langevin Dynamics sampler.

Usage:

SGLD(1000, 0.5)

Arguments:

  • n_iters::Int : Number of samples to pull.
  • ϵ::Float64 : The scaling factor for the learing rate.

Reference:

Welling, M., & Teh, Y. W. (2011). Bayesian learning via stochastic gradient Langevin dynamics. In Proceedings of the 28th international conference on machine learning (ICML-11) (pp. 681-688).

source

# Turing.Inference.SMCType.

SMC(n_particles::Int)

Sequential Monte Carlo sampler.

Note that this method is particle-based, and arrays of variables must be stored in a TArray object.

Usage:

SMC(1000)

source

Data Structures

# Libtask.TArrayType.

TArray{T}(dims, ...)

Implementation of data structures that automatically perform copy-on-write after task copying.

If current*task is an existing key in s, then return s[current*task]. Otherwise, returns[current*task] = s[last*task].

Usage:

TArray(dim)

Example:

ta = TArray(4)              # init
for i in 1:4 ta[i] = i end  # assign
Array(ta)                   # convert to 4-element Array{Int64,1}: [1, 2, 3, 4]

Utilities

# Libtask.tzerosFunction.

 tzeros(dims, ...)

Construct a distributed array of zeros. Trailing arguments are the same as those accepted by TArray.

tzeros(dim)

Example:

tz = tzeros(4)              # construct
Array(tz)                   # convert to 4-element Array{Int64,1}: [0, 0, 0, 0]

Index