A directed Graphical model. Encodes a Causality structure (work of Judea Pearls. Book: Causality: Models, Reasoning, and Inference)
Graphical representation
They are represented via DAGs
Joint probability distribution
The joint prob distribution given by the chain rule of Bayesian networks: product of the conditional probability distributions (CPDs) of each of the nodes
Properties of Bayesian nets
When are two Bayesian networks equivalent (they represent the same class of Probability distributions with the same independencies)?: Two DAGs are equivalent if and only if they have the same skeleton and the same v-structures.
Flow of probabilistic influence
When can one variable (X) influence another variable (Y)? More precisely, what this means is that when conditioning on X, the probability distribution of Y changes.
Two variables can influence each other if they are connected by an active trail, which is a path (ignoring direction) through the network, such that there are no "v-structures" (which are causal structures like X -> W <- Y, for which Y is marginally independent on X, i.e. independent when marginalizing over W).
But given that we have evidence (a set of the random variables Z, with fixed values, which we condition over), the flow of influence changes, and we now have:
The main effect here is the "activation" of v-structures, known as Explaining away
Independences
general procedure called d-separation (which stands for directed separation) can answer whether an independence assertion must hold in any distribution consistent with the graph G. However, note that other independencies may hold in some distributions consistent withG; these are due to flukes in the particular choice of parameters of the network (and this is why they hold in some of the distributions). d-separation occurs when there is no flow of influence as defined above
Independence of random variables – vid
Independence and factors, I-maps
Causality in Bayes nets
The arrows need not represent causal structure, but if they do the models tend to be simpler.
Types of Bayesian networks
Template models
Ways of representing graphical models that have a lot of internal shared structure (repeated variables and topologies), like events that occur over time, or relation types found over and over in a graph.. See vid
- Temporal models for temporal processes.
- Object-relational models
- Directed: Plate models
- Undirected
Structured CPDs
CPDs which extra structure, which are useful for models with lots of variables.
Context-specific independence
- Tree-structured CPD, like Decision trees
Fuzzy-OR structure (see here), is useful when an event can be caused by several possible causes. We model this using an intermediate variable, which determines if the cause actually causes the effect (it is always 0 if the cause is inactive, and 1 with a certain probability if the cause is active). The effect occurs, if either of the Zs is 1.
Context-specific independence: If the effect has not happened, the causes are then independent.
More examples of Bayesian nets
- Naive Bayes models