Title: | Conditional Random Fields |
---|---|
Description: | Implements modeling and computational tools for conditional random fields (CRF) model as well as other probabilistic undirected graphical models of discrete data with pairwise and unary potentials. |
Authors: | Ling-Yun Wu [aut, cre] |
Maintainer: | Ling-Yun Wu <[email protected]> |
License: | GPL (>= 2) |
Version: | 0.4-3 |
Built: | 2024-11-25 05:09:31 UTC |
Source: | https://github.com/wulingyun/crf |
Library of Conditional Random Fields model
CRF is R package for various computational tasks of conditional random fields as well as other probabilistic undirected graphical models of discrete data with pairwise and unary potentials. The decoding/inference/sampling tasks are implemented for general discrete undirected graphical models with pairwise potentials. The training task is less general, focusing on conditional random fields with log-linear potentials and a fixed structure. The code is written entirely in R and C++. The initial version is ported from UGM written by Mark Schmidt.
Decoding: Computing the most likely configuration
decode.exact
Exact decoding for small graphs with brute-force search
decode.chain
Exact decoding for chain-structured graphs with the Viterbi algorithm
decode.tree
Exact decoding for tree- and forest-structured graphs with max-product belief propagation
decode.conditional
Conditional decoding (takes another decoding method as input)
decode.cutset
Exact decoding for graphs with a small cutset using cutset conditioning
decode.junction
Exact decoding for low-treewidth graphs using junction trees
decode.sample
Approximate decoding using sampling (takes a sampling method as input)
decode.marginal
Approximate decoding using inference (takes an inference method as input)
decode.lbp
Approximate decoding using max-product loopy belief propagation
decode.trbp
Approximate decoding using max-product tree-reweighted belief propagtion
decode.greedy
Approximate decoding with greedy algorithm
decode.icm
Approximate decoding with the iterated conditional modes algorithm
decode.block
Approximate decoding with the block iterated conditional modes algorithm
decode.ilp
Exact decoding with an integer linear programming formulation and approximate using LP relaxation
Inference: Computing the partition function and marginal probabilities
infer.exact
Exact inference for small graphs with brute-force counting
infer.chain
Exact inference for chain-structured graphs with the forward-backward algorithm
infer.tree
Exact inference for tree- and forest-structured graphs with sum-product belief propagation
infer.conditional
Conditional inference (takes another inference method as input)
infer.cutset
Exact inference for graphs with a small cutset using cutset conditioning
infer.junction
Exact decoding for low-treewidth graphs using junction trees
infer.sample
Approximate inference using sampling (takes a sampling method as input)
infer.lbp
Approximate inference using sum-product loopy belief propagation
infer.trbp
Approximate inference using sum-product tree-reweighted belief propagation
Sampling: Generating samples from the distribution
sample.exact
Exact sampling for small graphs with brute-force inverse cumulative distribution
sample.chain
Exact sampling for chain-structured graphs with the forward-filter backward-sample algorithm
sample.tree
Exact sampling for tree- and forest-structured graphs with sum-product belief propagation and backward-sampling
sample.conditional
Conditional sampling (takes another sampling method as input)
sample.cutset
Exact sampling for graphs with a small cutset using cutset conditioning
sample.junction
Exact sampling for low-treewidth graphs using junction trees
sample.gibbs
Approximate sampling using a single-site Gibbs sampler
Training: Given data, computing the most likely estimates of the parameters
Tools: Tools for building and manipulating CRF data
make.crf
Generate CRF from the adjacent matrix
make.features
Make the data structure of CRF features
make.par
Make the data structure of CRF parameters
duplicate.crf
Duplicate an existing CRF
clamp.crf
Generate clamped CRF by fixing the states of some nodes
clamp.reset
Reset clamped CRF by changing the states of clamped nodes
sub.crf
Generate sub CRF by selecting some nodes
mrf.update
Update node and edge potentials of MRF model
crf.update
Update node and edge potentials of CRF model
Ling-Yun Wu [email protected]
J. Lafferty, A. McCallum, and F. Pereira. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In the proceedings of International Conference on Machine Learning (ICML), pp. 282-289, 2001.
Mark Schmidt. UGM: A Matlab toolbox for probabilistic undirected graphical models. http://www.cs.ubc.ca/~schmidtm/Software/UGM.html, 2007.
library(CRF) data(Small) decode.exact(Small$crf) infer.exact(Small$crf) sample.exact(Small$crf, 100)
library(CRF) data(Small) decode.exact(Small$crf) infer.exact(Small$crf) sample.exact(Small$crf, 100)
This data set gives a chain CRF example
data(Chain)
data(Chain)
A list containing two elements:
crf
The CRF
answer
A list of 4 elements:
decode
The most likely configuration
node.bel
The node belief
edge.bel
The edge belief
logZ
The logarithmic value of CRF normalization factor Z
Generate clamped CRF by fixing the states of some nodes
clamp.crf(crf, clamped)
clamp.crf(crf, clamped)
crf |
The CRF generated by |
clamped |
The vector of fixed states of nodes |
The function will generate a clamped CRF from a given CRF
by fixing the states of some nodes. The vector clamped
contains the desired state for each node while zero means the state is not
fixed. The node and edge potentials are updated to the conditional
potentials based on the clamped vector.
The function will return a new CRF with additional components:
original |
The original CRF. |
clamped |
The vector of fixed states of nodes. |
node.id |
The vector of the original node ids for nodes in the new CRF. |
node.map |
The vector of the new node ids for nodes in the original CRF. |
edge.id |
The vector of the original edge ids for edges in the new CRF. |
edge.map |
The vector of the new edge ids for edges in the original CRF. |
make.crf
, sub.crf
, clamp.reset
library(CRF) data(Small) crf <- clamp.crf(Small$crf, c(0, 0, 1, 1))
library(CRF) data(Small) crf <- clamp.crf(Small$crf, c(0, 0, 1, 1))
Reset clamped CRF by changing the states of clamped nodes
clamp.reset(crf, clamped)
clamp.reset(crf, clamped)
crf |
The clamped CRF generated by |
clamped |
The vector of fixed states of nodes |
The function will reset a clamped CRF by changing the states of fixed nodes.
The vector clamped
contains the desired state for each node
while zero means the state is not fixed. The node and edge potentials are
updated to the conditional potentials based on the clamped vector.
The function will return the same clamped CRF.
library(CRF) data(Small) crf <- clamp.crf(Small$crf, c(0, 0, 1, 1)) clamp.reset(crf, c(0,0,2,2))
library(CRF) data(Small) crf <- clamp.crf(Small$crf, c(0, 0, 1, 1)) clamp.reset(crf, c(0,0,2,2))
This data set gives a clique CRF example
data(Clique)
data(Clique)
A list containing two elements:
crf
The CRF
answer
A list of 4 elements:
decode
The most likely configuration
node.bel
The node belief
edge.bel
The edge belief
logZ
The logarithmic value of CRF normalization factor Z
Calculate the negative log likelihood of CRF model
crf.nll( par, crf, instances, node.fea = NULL, edge.fea = NULL, node.ext = NULL, edge.ext = NULL, infer.method = infer.chain, ... )
crf.nll( par, crf, instances, node.fea = NULL, edge.fea = NULL, node.ext = NULL, edge.ext = NULL, infer.method = infer.chain, ... )
par |
The parameter vector of CRF |
crf |
The CRF |
instances |
The training data matrix of CRF model |
node.fea |
The list of node features |
edge.fea |
The list of edge features |
node.ext |
The list of extended information of node features |
edge.ext |
The list of extended information of edge features |
infer.method |
The inference method used to compute the likelihood |
... |
Extra parameters need by the inference method |
This function calculates the negative log likelihood of CRF model as well as the gradient. This function is intended to be called by optimization algorithm in training process.
In the training data matrix instances
, each row is an instance and
each column corresponds a node in CRF.
The variables node.fea
, edge.fea
, node.ext
, edge.ext
are lists of length equal to the number of instances, and their elements are
defined as in crf.update
respectively.
This function will return the value of CRF negative log-likelihood.
Update node and edge potentials of CRF model
crf.update( crf, node.fea = NULL, edge.fea = NULL, node.ext = NULL, edge.ext = NULL )
crf.update( crf, node.fea = NULL, edge.fea = NULL, node.ext = NULL, edge.ext = NULL )
crf |
The CRF |
node.fea |
The node features matrix with dimension |
edge.fea |
The edge features matrix with dimension |
node.ext |
The extended information of node features |
edge.ext |
The extended information of edge features |
This function updates node.pot
and edge.pot
of CRF model by using
the current values of parameters and features.
There are two ways to model the relationship between parameters and features. The first one exploits the special structure of features to reduce the memory usage. However it may not suitable for all circumstances. The other one is more straighforward by explicitly specifying the coefficients of each parameter to calculate the potentials, and may use much more memory. Two approaches can be used together.
The first way uses the objects node.par
and edge.par
to define
the structure of features and provides the feature information in variables
node.fea
and edge.fea
. The second way directly provides the
feature information in variables node.ext
and edge.ext
without
any prior assumption on feature structure. node.ext
is a list and
each element has the same structure as node.pot
. edge.ext
is
a list and each element has the same structure as edge.pot
.
In detail, the node potential is updated as follows:
and the edge potential is updated as follows:
This function will directly modify the CRF and return the same CRF.
Computing the most likely configuration for CRF
decode.block( crf, blocks, decode.method = decode.tree, restart = 0, start = apply(crf$node.pot, 1, which.max), ... )
decode.block( crf, blocks, decode.method = decode.tree, restart = 0, start = apply(crf$node.pot, 1, which.max), ... )
crf |
The CRF |
blocks |
A list of vectors, each vector containing the nodes in a block |
decode.method |
The decoding method to solve the clamped CRF |
restart |
Non-negative integer to control how many restart iterations are repeated |
start |
An initial configuration, a good start will significantly reduce the seraching time |
... |
The parameters for |
Approximate decoding with the block iterated conditional modes algorithm
This function will return the most likely configuration, which is a vector of length crf$n.nodes
.
library(CRF) data(Small) d <- decode.block(Small$crf, list(c(1,3), c(2,4)))
library(CRF) data(Small) d <- decode.block(Small$crf, list(c(1,3), c(2,4)))
Computing the most likely configuration for CRF
decode.chain(crf)
decode.chain(crf)
crf |
The CRF |
Exact decoding for chain-structured graphs with the Viterbi algorithm.
This function will return the most likely configuration, which is a vector of length crf$n.nodes
.
library(CRF) data(Small) d <- decode.chain(Small$crf)
library(CRF) data(Small) d <- decode.chain(Small$crf)
Computing the most likely configuration for CRF
decode.conditional(crf, clamped, decode.method, ...)
decode.conditional(crf, clamped, decode.method, ...)
crf |
The CRF |
clamped |
The vector of fixed values for clamped nodes, 0 for unfixed nodes |
decode.method |
The decoding method to solve clamped CRF |
... |
The parameters for |
Conditional decoding (takes another decoding method as input)
This function will return the most likely configuration, which is a vector of length crf$n.nodes
.
library(CRF) data(Small) d <- decode.conditional(Small$crf, c(0,1,0,0), decode.exact)
library(CRF) data(Small) d <- decode.conditional(Small$crf, c(0,1,0,0), decode.exact)
Computing the most likely configuration for CRF
decode.cutset( crf, cutset, engine = "default", start = apply(crf$node.pot, 1, which.max) )
decode.cutset( crf, cutset, engine = "default", start = apply(crf$node.pot, 1, which.max) )
crf |
The CRF |
cutset |
A vector of nodes in the cutset |
engine |
The underlying engine for cutset decoding, possible values are "default", "none", "exact", "chain", and "tree". |
start |
An initial configuration, a good start will significantly reduce the seraching time |
Exact decoding for graphs with a small cutset using cutset conditioning
This function will return the most likely configuration, which is a vector of length crf$n.nodes
.
library(CRF) data(Small) d <- decode.cutset(Small$crf, c(2))
library(CRF) data(Small) d <- decode.cutset(Small$crf, c(2))
Computing the most likely configuration for CRF
decode.exact(crf)
decode.exact(crf)
crf |
The CRF |
Exact decoding for small graphs with brute-force search
This function will return the most likely configuration, which is a vector of length crf$n.nodes
.
library(CRF) data(Small) d <- decode.exact(Small$crf)
library(CRF) data(Small) d <- decode.exact(Small$crf)
Computing the most likely configuration for CRF
decode.greedy(crf, restart = 0, start = apply(crf$node.pot, 1, which.max))
decode.greedy(crf, restart = 0, start = apply(crf$node.pot, 1, which.max))
crf |
The CRF |
restart |
Non-negative integer to control how many restart iterations are repeated |
start |
An initial configuration, a good start will significantly reduce the seraching time |
Approximate decoding with greedy algorithm
This function will return the most likely configuration, which is a vector of length crf$n.nodes
.
library(CRF) data(Small) d <- decode.greedy(Small$crf)
library(CRF) data(Small) d <- decode.greedy(Small$crf)
Computing the most likely configuration for CRF
decode.icm(crf, restart = 0, start = apply(crf$node.pot, 1, which.max))
decode.icm(crf, restart = 0, start = apply(crf$node.pot, 1, which.max))
crf |
The CRF |
restart |
Non-negative integer to control how many restart iterations are repeated |
start |
An initial configuration, a good start will significantly reduce the seraching time |
Approximate decoding with the iterated conditional modes algorithm
This function will return the most likely configuration, which is a vector of length crf$n.nodes
.
library(CRF) data(Small) d <- decode.icm(Small$crf)
library(CRF) data(Small) d <- decode.icm(Small$crf)
Computing the most likely configuration for CRF
decode.ilp(crf, lp.rounding = FALSE)
decode.ilp(crf, lp.rounding = FALSE)
crf |
The CRF |
lp.rounding |
Boolean variable to indicate whether LP rounding is need. |
Exact decoding with an integer linear programming formulation and approximate using LP relaxation
This function will return the most likely configuration, which is a vector of length crf$n.nodes
.
## Not run: library(CRF) data(Small) d <- decode.ilp(Small$crf) ## End(Not run)
## Not run: library(CRF) data(Small) d <- decode.ilp(Small$crf) ## End(Not run)
Computing the most likely configuration for CRF
decode.junction(crf)
decode.junction(crf)
crf |
The CRF |
Exact decoding for low-treewidth graphs using junction trees
This function will return the most likely configuration, which is a vector of length crf$n.nodes
.
library(CRF) data(Small) d <- decode.junction(Small$crf)
library(CRF) data(Small) d <- decode.junction(Small$crf)
Computing the most likely configuration for CRF
decode.lbp(crf, max.iter = 10000, cutoff = 1e-04, verbose = 0)
decode.lbp(crf, max.iter = 10000, cutoff = 1e-04, verbose = 0)
crf |
The CRF |
max.iter |
The maximum allowed iterations of termination criteria |
cutoff |
The convergence cutoff of termination criteria |
verbose |
Non-negative integer to control the tracing informtion in algorithm |
Approximate decoding using max-product loopy belief propagation
This function will return the most likely configuration, which is a vector of length crf$n.nodes
.
library(CRF) data(Small) d <- decode.lbp(Small$crf)
library(CRF) data(Small) d <- decode.lbp(Small$crf)
Computing the most likely configuration for CRF
decode.marginal(crf, infer.method, ...)
decode.marginal(crf, infer.method, ...)
crf |
The CRF |
infer.method |
The inference method |
... |
The parameters for |
Approximate decoding using inference (takes an inference method as input)
This function will return the most likely configuration, which is a vector of length crf$n.nodes
.
library(CRF) data(Small) d <- decode.marginal(Small$crf, infer.exact)
library(CRF) data(Small) d <- decode.marginal(Small$crf, infer.exact)
Computing the most likely configuration for CRF
decode.rbp(crf, max.iter = 10000, cutoff = 1e-04, verbose = 0)
decode.rbp(crf, max.iter = 10000, cutoff = 1e-04, verbose = 0)
crf |
The CRF |
max.iter |
The maximum allowed iterations of termination criteria |
cutoff |
The convergence cutoff of termination criteria |
verbose |
Non-negative integer to control the tracing informtion in algorithm |
Approximate decoding using max-product residual belief propagation
This function will return the most likely configuration, which is a vector of length crf$n.nodes
.
library(CRF) data(Small) d <- decode.rbp(Small$crf)
library(CRF) data(Small) d <- decode.rbp(Small$crf)
Computing the most likely configuration for CRF
decode.sample(crf, sample.method, ...)
decode.sample(crf, sample.method, ...)
crf |
The CRF |
sample.method |
The sampling method |
... |
The parameters for |
Approximate decoding using sampling (takes a sampling method as input)
This function will return the most likely configuration, which is a vector of length crf$n.nodes
.
library(CRF) data(Small) d <- decode.sample(Small$crf, sample.exact, 10000)
library(CRF) data(Small) d <- decode.sample(Small$crf, sample.exact, 10000)
Computing the most likely configuration for CRF
decode.trbp(crf, max.iter = 10000, cutoff = 1e-04, verbose = 0)
decode.trbp(crf, max.iter = 10000, cutoff = 1e-04, verbose = 0)
crf |
The CRF |
max.iter |
The maximum allowed iterations of termination criteria |
cutoff |
The convergence cutoff of termination criteria |
verbose |
Non-negative integer to control the tracing informtion in algorithm |
Approximate decoding using max-product tree-reweighted belief propagtion
This function will return the most likely configuration, which is a vector of length crf$n.nodes
.
library(CRF) data(Small) d <- decode.trbp(Small$crf)
library(CRF) data(Small) d <- decode.trbp(Small$crf)
Computing the most likely configuration for CRF
decode.tree(crf)
decode.tree(crf)
crf |
The CRF |
Exact decoding for tree- and forest-structured graphs with max-product belief propagation
This function will return the most likely configuration, which is a vector of length crf$n.nodes
.
library(CRF) data(Small) d <- decode.tree(Small$crf)
library(CRF) data(Small) d <- decode.tree(Small$crf)
Duplicate an existing CRF
duplicate.crf(crf)
duplicate.crf(crf)
crf |
The existing CRF |
This function will duplicate an existing CRF. Since CRF is implemented as an environment, normal assignment will only copy the pointer instead of the real data. This function will generate a new CRF and really copy all data.
The function will return a new CRF with copied data
Calculate the logarithmic potential of a CRF with given configuration
get.logPotential(crf, configuration)
get.logPotential(crf, configuration)
crf |
The CRF |
configuration |
The vector of states of nodes |
The function will calculate the logarithmic potential of a CRF with given configuration, i.e., the assigned states of nodes in the CRF.
The function will return the log-potential of CRF with given configuration
Calculate the potential of a CRF with given configuration
get.potential(crf, configuration)
get.potential(crf, configuration)
crf |
The CRF |
configuration |
The vector of states of nodes |
The function will calculate the potential of a CRF with given configuration, i.e., the assigned states of nodes in the CRF.
The function will return the potential of CRF with given configuration
Computing the partition function and marginal probabilities
infer.chain(crf)
infer.chain(crf)
crf |
The CRF |
Exact inference for chain-structured graphs with the forward-backward algorithm
This function will return a list with components:
node.bel |
Node belief. It is a matrix with |
edge.bel |
Edge belief. It is a list of matrices. The size of list is |
logZ |
The logarithmic value of CRF normalization factor Z. |
library(CRF) data(Small) i <- infer.chain(Small$crf)
library(CRF) data(Small) i <- infer.chain(Small$crf)
Computing the partition function and marginal probabilities
infer.conditional(crf, clamped, infer.method, ...)
infer.conditional(crf, clamped, infer.method, ...)
crf |
The CRF |
clamped |
The vector of fixed values for clamped nodes, 0 for unfixed nodes |
infer.method |
The inference method to solve the clamped CRF |
... |
The parameters for |
Conditional inference (takes another inference method as input)
This function will return a list with components:
node.bel |
Node belief. It is a matrix with |
edge.bel |
Edge belief. It is a list of matrices. The size of list is |
logZ |
The logarithmic value of CRF normalization factor Z. |
library(CRF) data(Small) i <- infer.conditional(Small$crf, c(0,1,0,0), infer.exact)
library(CRF) data(Small) i <- infer.conditional(Small$crf, c(0,1,0,0), infer.exact)
Computing the partition function and marginal probabilities
infer.cutset(crf, cutset, engine = "default")
infer.cutset(crf, cutset, engine = "default")
crf |
The CRF |
cutset |
A vector of nodes in the cutset |
engine |
The underlying engine for cutset decoding, possible values are "default", "none", "exact", "chain", and "tree". |
Exact inference for graphs with a small cutset using cutset conditioning
This function will return a list with components:
node.bel |
Node belief. It is a matrix with |
edge.bel |
Edge belief. It is a list of matrices. The size of list is |
logZ |
The logarithmic value of CRF normalization factor Z. |
library(CRF) data(Small) i <- infer.cutset(Small$crf, c(2))
library(CRF) data(Small) i <- infer.cutset(Small$crf, c(2))
Computing the partition function and marginal probabilities
infer.exact(crf)
infer.exact(crf)
crf |
The CRF |
Exact inference for small graphs with brute-force counting
This function will return a list with components:
node.bel |
Node belief. It is a matrix with |
edge.bel |
Edge belief. It is a list of matrices. The size of list is |
logZ |
The logarithmic value of CRF normalization factor Z. |
library(CRF) data(Small) i <- infer.exact(Small$crf)
library(CRF) data(Small) i <- infer.exact(Small$crf)
Computing the partition function and marginal probabilities
infer.junction(crf)
infer.junction(crf)
crf |
The CRF |
Exact decoding for low-treewidth graphs using junction trees
This function will return a list with components:
node.bel |
Node belief. It is a matrix with |
edge.bel |
Edge belief. It is a list of matrices. The size of list is |
logZ |
The logarithmic value of CRF normalization factor Z. |
library(CRF) data(Small) i <- infer.junction(Small$crf)
library(CRF) data(Small) i <- infer.junction(Small$crf)
Computing the partition function and marginal probabilities
infer.lbp(crf, max.iter = 10000, cutoff = 1e-04, verbose = 0, maximize = FALSE)
infer.lbp(crf, max.iter = 10000, cutoff = 1e-04, verbose = 0, maximize = FALSE)
crf |
The CRF |
max.iter |
The maximum allowed iterations of termination criteria |
cutoff |
The convergence cutoff of termination criteria |
verbose |
Non-negative integer to control the tracing informtion in algorithm |
maximize |
Logical variable to indicate using max-product instead of sum-product |
Approximate inference using sum-product loopy belief propagation
This function will return a list with components:
node.bel |
Node belief. It is a matrix with |
edge.bel |
Edge belief. It is a list of matrices. The size of list is |
logZ |
The logarithmic value of CRF normalization factor Z. |
library(CRF) data(Small) i <- infer.lbp(Small$crf)
library(CRF) data(Small) i <- infer.lbp(Small$crf)
Computing the partition function and marginal probabilities
infer.rbp(crf, max.iter = 10000, cutoff = 1e-04, verbose = 0, maximize = FALSE)
infer.rbp(crf, max.iter = 10000, cutoff = 1e-04, verbose = 0, maximize = FALSE)
crf |
The CRF |
max.iter |
The maximum allowed iterations of termination criteria |
cutoff |
The convergence cutoff of termination criteria |
verbose |
Non-negative integer to control the tracing informtion in algorithm |
maximize |
Logical variable to indicate using max-product instead of sum-product |
Approximate inference using sum-product residual belief propagation
This function will return a list with components:
node.bel |
Node belief. It is a matrix with |
edge.bel |
Edge belief. It is a list of matrices. The size of list is |
logZ |
The logarithmic value of CRF normalization factor Z. |
library(CRF) data(Small) i <- infer.rbp(Small$crf)
library(CRF) data(Small) i <- infer.rbp(Small$crf)
Computing the partition function and marginal probabilities
infer.sample(crf, sample.method, ...)
infer.sample(crf, sample.method, ...)
crf |
The CRF |
sample.method |
The sampling method |
... |
The parameters for |
Approximate inference using sampling (takes a sampling method as input)
This function will return a list with components:
node.bel |
Node belief. It is a matrix with |
edge.bel |
Edge belief. It is a list of matrices. The size of list is |
logZ |
The logarithmic value of CRF normalization factor Z. |
library(CRF) data(Small) i <- infer.sample(Small$crf, sample.exact, 10000)
library(CRF) data(Small) i <- infer.sample(Small$crf, sample.exact, 10000)
Computing the partition function and marginal probabilities
infer.trbp( crf, max.iter = 10000, cutoff = 1e-04, verbose = 0, maximize = FALSE )
infer.trbp( crf, max.iter = 10000, cutoff = 1e-04, verbose = 0, maximize = FALSE )
crf |
The CRF |
max.iter |
The maximum allowed iterations of termination criteria |
cutoff |
The convergence cutoff of termination criteria |
verbose |
Non-negative integer to control the tracing informtion in algorithm |
maximize |
Logical variable to indicate using max-product instead of sum-product |
Approximate inference using sum-product tree-reweighted belief propagation
This function will return a list with components:
node.bel |
Node belief. It is a matrix with |
edge.bel |
Edge belief. It is a list of matrices. The size of list is |
logZ |
The logarithmic value of CRF normalization factor Z. |
library(CRF) data(Small) i <- infer.trbp(Small$crf)
library(CRF) data(Small) i <- infer.trbp(Small$crf)
Computing the partition function and marginal probabilities
infer.tree(crf)
infer.tree(crf)
crf |
The CRF |
Exact inference for tree- and forest-structured graphs with sum-product belief propagation
This function will return a list with components:
node.bel |
Node belief. It is a matrix with |
edge.bel |
Edge belief. It is a list of matrices. The size of list is |
logZ |
The logarithmic value of CRF normalization factor Z. |
library(CRF) data(Small) i <- infer.tree(Small$crf)
library(CRF) data(Small) i <- infer.tree(Small$crf)
This data set gives a loop CRF example
data(Loop)
data(Loop)
A list containing two elements:
crf
The CRF
answer
A list of 4 elements:
decode
The most likely configuration
node.bel
The node belief
edge.bel
The edge belief
logZ
The logarithmic value of CRF normalization factor Z
Generate CRF from the adjacent matrix
make.crf(adj.matrix = NULL, n.states = 2, n.nodes = 2)
make.crf(adj.matrix = NULL, n.states = 2, n.nodes = 2)
adj.matrix |
The adjacent matrix of CRF network. |
n.states |
The state numbers of nodes. |
n.nodes |
The number of nodes, which is only used to generate linear chain CRF when |
The function will generate an empty CRF from a given adjacent
matrix. If the length of nstates
is less than n.nodes
, it will
be used repeatly. All node and edge potentials are initilized as 1.
Since the CRF data are often very huge, CRF is implemented as an environment.
The assignment of environments will only copy the addresses instead of real data,
therefore the variables using normal assignment will refer to the exactly same CRF.
For complete duplication of the data, please use duplicate.crf
.
The function will return a new CRF, which is an environment with components:
n.nodes |
The number of nodes. |
n.edges |
The number of edges. |
n.states |
The number of states for each node. It is a vector of length |
max.state |
The maximum number of states. It is equal to |
edges |
The node pair of each edge. It is a matrix with 2 columns and |
n.adj |
The number of adjacent nodes for each node. It is a vector of length |
adj.nodes |
The list of adjacent nodes for each
node. It is a list of length |
adj.edges |
The list of adjacent edges for each node. It is similiar to |
node.pot |
The node potentials. It is a matrix with dimmension |
edge.pot |
The edge potentials. It is a list of |
duplicate.crf
, clamp.crf
, sub.crf
library(CRF) nNodes <- 4 nStates <- 2 adj <- matrix(0, nrow=nNodes, ncol=nNodes) for (i in 1:(nNodes-1)) { adj[i,i+1] <- 1 adj[i+1,i] <- 1 } crf <- make.crf(adj, nStates) crf$node.pot[1,] <- c(1, 3) crf$node.pot[2,] <- c(9, 1) crf$node.pot[3,] <- c(1, 3) crf$node.pot[4,] <- c(9, 1) for (i in 1:crf$n.edges) { crf$edge.pot[[i]][1,] <- c(2, 1) crf$edge.pot[[i]][2,] <- c(1, 2) }
library(CRF) nNodes <- 4 nStates <- 2 adj <- matrix(0, nrow=nNodes, ncol=nNodes) for (i in 1:(nNodes-1)) { adj[i,i+1] <- 1 adj[i+1,i] <- 1 } crf <- make.crf(adj, nStates) crf$node.pot[1,] <- c(1, 3) crf$node.pot[2,] <- c(9, 1) crf$node.pot[3,] <- c(1, 3) crf$node.pot[4,] <- c(9, 1) for (i in 1:crf$n.edges) { crf$edge.pot[[i]][1,] <- c(2, 1) crf$edge.pot[[i]][2,] <- c(1, 2) }
Make the data structure of CRF features
make.features(crf, n.nf = 1, n.ef = 1)
make.features(crf, n.nf = 1, n.ef = 1)
crf |
The CRF |
n.nf |
The number of node features |
n.ef |
The number of edge features |
This function makes the data structure of features need for modeling and training CRF.
The parameters n.nf
and n.ef
specify the number of node and edge features,
respectively.
The objects node.par
and edge.par
define the corresponding
parameters used with each feature. node.par
is a 3-dimensional arrays,
and element node.par[n,i,f]
is the index of parameter associated with the
corresponding node potential node.pot[n,i]
and node feature f
.
edge.par
is a list of 3-dimensional arrays, and element
edge.par[[e]][i,j,f]
is the index of parameter associated with the
corresponding edge potential edge.pot[[e]][i,j]
and edge feature f
.
The value 0 is used to indicate the corresponding node or edge potential
does not depend on that feature.
For detail of calculation of node and edge potentials from features and parameters,
please see crf.update
.
This function will directly modify the CRF and return the same CRF.
crf.update
, make.par
, make.crf
Make the data structure of CRF parameters
make.par(crf, n.par = 1)
make.par(crf, n.par = 1)
crf |
The CRF |
n.par |
The number of parameters |
This function makes the data structure of parameters need for modeling and training CRF.
The parameters are stored in par
, which is a numeric vector of length n.par
.
This function will directly modify the CRF and return the same CRF.
crf.update
, make.features
, make.crf
Calculate the negative log-likelihood of MRF model
mrf.nll(par, crf, instances, infer.method = infer.chain, ...)
mrf.nll(par, crf, instances, infer.method = infer.chain, ...)
par |
The parameter vector of CRF |
crf |
The CRF |
instances |
The training data matrix of MRF model |
infer.method |
The inference method used to compute the likelihood |
... |
Extra parameters need by the inference method |
This function calculates the negative log-likelihood of MRF model as well as
the gradient. This function is intended to be called by optimization algorithm
in training process. Before calling this function, the MRF sufficient
statistics must be calculated and stored in object par.stat
of CRF.
In the training data matrix instances
, each row is an instance and
each column corresponds a node in CRF.
This function will return the value of MRF negative log-likilihood.
mrf.stat
, mrf.update
, train.mrf
Calculate the sufficient statistics of MRF model
mrf.stat(crf, instances)
mrf.stat(crf, instances)
crf |
The CRF |
instances |
The training data matrix of MRF model |
This function calculates the sufficient statistics of MRF model. This function
much be called before the first calling to mrf.nll
.
In the training data matrix instances
, each row is an instance and
each column corresponds a node in CRF.
This function will return the value of MRF sufficient statistics.
Update node and edge potentials of MRF model
mrf.update(crf)
mrf.update(crf)
crf |
The CRF |
The function updates node.pot
and edge.pot
of MRF model.
This function will directly modify the CRF and return the same CRF.
This data set gives an example of rain data used to train CRF and MRF models
data(Rain)
data(Rain)
A list containing two elements:
rain
A matrix of 28 columns containing raining data (1: rain, 2: sunny).
Each row is an instance of 28 days for one month.
months
A vector containing the months of each instance.
Mark Schmidt. UGM: A Matlab toolbox for probabilistic undirected graphical models. http://www.cs.ubc.ca/~schmidtm/Software/UGM.html, 2007.
Generating samples from the distribution
sample.chain(crf, size)
sample.chain(crf, size)
crf |
The CRF |
size |
The sample size |
Exact sampling for chain-structured graphs with the forward-filter backward-sample algorithm
This function will return a matrix with size
rows and crf$n.nodes
columns,
in which each row is a sampled configuration.
library(CRF) data(Small) s <- sample.chain(Small$crf, 100)
library(CRF) data(Small) s <- sample.chain(Small$crf, 100)
Generating samples from the distribution
sample.conditional(crf, size, clamped, sample.method, ...)
sample.conditional(crf, size, clamped, sample.method, ...)
crf |
The CRF |
size |
The sample size |
clamped |
The vector of fixed values for clamped nodes, 0 for unfixed nodes |
sample.method |
The sampling method to solve the clamped CRF |
... |
The parameters for |
Conditional sampling (takes another sampling method as input)
This function will return a matrix with size
rows and crf$n.nodes
columns,
in which each row is a sampled configuration.
library(CRF) data(Small) s <- sample.conditional(Small$crf, 100, c(0,1,0,0), sample.exact)
library(CRF) data(Small) s <- sample.conditional(Small$crf, 100, c(0,1,0,0), sample.exact)
Generating samples from the distribution
sample.cutset(crf, size, cutset, engine = "default")
sample.cutset(crf, size, cutset, engine = "default")
crf |
The CRF |
size |
The sample size |
cutset |
A vector of nodes in the cutset |
engine |
The underlying engine for cutset sampling, possible values are "default", "none", "exact", "chain", and "tree". |
Exact sampling for graphs with a small cutset using cutset conditioning
This function will return a matrix with size
rows and crf$n.nodes
columns,
in which each row is a sampled configuration.
library(CRF) data(Small) s <- sample.cutset(Small$crf, 100, c(2))
library(CRF) data(Small) s <- sample.cutset(Small$crf, 100, c(2))
Generating samples from the distribution
sample.exact(crf, size)
sample.exact(crf, size)
crf |
The CRF |
size |
The sample size |
Exact sampling for small graphs with brute-force inverse cumulative distribution
This function will return a matrix with size
rows and crf$n.nodes
columns,
in which each row is a sampled configuration.
library(CRF) data(Small) s <- sample.exact(Small$crf, 100)
library(CRF) data(Small) s <- sample.exact(Small$crf, 100)
Generating samples from the distribution
sample.gibbs( crf, size, burn.in = 1000, start = apply(crf$node.pot, 1, which.max) )
sample.gibbs( crf, size, burn.in = 1000, start = apply(crf$node.pot, 1, which.max) )
crf |
The CRF |
size |
The sample size |
burn.in |
The number of samples at the beginning that will be discarded |
start |
An initial configuration |
Approximate sampling using a single-site Gibbs sampler
This function will return a matrix with size
rows and crf$n.nodes
columns,
in which each row is a sampled configuration.
library(CRF) data(Small) s <- sample.gibbs(Small$crf, 100)
library(CRF) data(Small) s <- sample.gibbs(Small$crf, 100)
Generating samples from the distribution
sample.junction(crf, size)
sample.junction(crf, size)
crf |
The CRF |
size |
The sample size |
Exact sampling for low-treewidth graphs using junction trees
This function will return a matrix with size
rows and crf$n.nodes
columns,
in which each row is a sampled configuration.
library(CRF) data(Small) s <- sample.junction(Small$crf, 100)
library(CRF) data(Small) s <- sample.junction(Small$crf, 100)
Generating samples from the distribution
sample.tree(crf, size)
sample.tree(crf, size)
crf |
The CRF |
size |
The sample size |
Exact sampling for tree- and forest-structured graphs with sum-product belief propagation and backward-sampling
This function will return a matrix with size
rows and crf$n.nodes
columns,
in which each row is a sampled configuration.
library(CRF) data(Small) s <- sample.tree(Small$crf, 100)
library(CRF) data(Small) s <- sample.tree(Small$crf, 100)
This data set gives a small CRF example
data(Small)
data(Small)
A list containing two elements:
crf
The CRF
answer
A list of 4 elements:
decode
The most likely configuration
node.bel
The node belief
edge.bel
The edge belief
logZ
The logarithmic value of CRF normalization factor Z
Generate sub CRF by selecting some nodes
sub.crf(crf, subset)
sub.crf(crf, subset)
crf |
The CRF generated by |
subset |
The vector of selected node ids |
The function will generate a new CRF from a given CRF
by selecting some nodes. The vector subset
contains the
node ids selected to generate the new CRF. Unlike
clamp.crf
, the potentials of remainning nodes and edges are
untouched.
The function will return a new CRF with additional components:
original |
The original CRF data. |
node.id |
The vector of the original node ids for nodes in the new CRF. |
node.map |
The vector of the new node ids for nodes in the original CRF. |
edge.id |
The vector of the original edge ids for edges in the new CRF. |
edge.map |
The vector of the new edge ids for edges in the original CRF. |
library(CRF) data(Small) crf <- sub.crf(Small$crf, c(2, 3))
library(CRF) data(Small) crf <- sub.crf(Small$crf, c(2, 3))
Train the CRF model to estimate the parameters
train.crf( crf, instances, node.fea = NULL, edge.fea = NULL, node.ext = NULL, edge.ext = NULL, nll = crf.nll, infer.method = infer.chain, ..., trace = 0 )
train.crf( crf, instances, node.fea = NULL, edge.fea = NULL, node.ext = NULL, edge.ext = NULL, nll = crf.nll, infer.method = infer.chain, ..., trace = 0 )
crf |
The CRF |
instances |
The training data matrix of CRF model |
node.fea |
The list of node features |
edge.fea |
The list of edge features |
node.ext |
The list of extended information of node features |
edge.ext |
The list of extended information of edge features |
nll |
The function to calculate negative log likelihood |
infer.method |
The inference method used to compute the likelihood |
... |
Extra parameters need by the inference method |
trace |
Non-negative integer to control the tracing informtion of the optimization process |
This function train the CRF model.
In the training data matrix instances
, each row is an instance and
each column corresponds a node in CRF.
The variables node.fea
, edge.fea
, node.ext
, edge.ext
are lists of length equal to the number of instances, and their elements are
defined as in crf.update
respectively.
This function will directly modify the CRF and return the same CRF.
Train the MRF model to estimate the parameters
train.mrf( crf, instances, nll = mrf.nll, infer.method = infer.chain, ..., trace = 0 )
train.mrf( crf, instances, nll = mrf.nll, infer.method = infer.chain, ..., trace = 0 )
crf |
The CRF |
instances |
The training data matrix of CRF model |
nll |
The function to calculate negative log likelihood |
infer.method |
The inference method used to compute the likelihood |
... |
Extra parameters need by the inference method |
trace |
Non-negative integer to control the tracing informtion of the optimization process |
This function trains the Markov Random Fields (MRF) model, which is a simple variant of CRF model.
In the training data matrix instances
, each row is an instance and
each column corresponds a node in CRF.
This function will directly modify the CRF and return the same CRF.
mrf.update
, mrf.stat
, mrf.nll
, make.crf
This data set gives a tree CRF example
data(Tree)
data(Tree)
A list containing two elements:
crf
The CRF
answer
A list of 4 elements:
decode
The most likely configuration
node.bel
The node belief
edge.bel
The edge belief
logZ
The logarithmic value of CRF normalization factor Z