Introduction to Bayesian Networks
Bayesian Networks (BNs) are graphical models that allow you to depict the cause and effect relationships among factors within a system. In a BN, these cause and effect relationships are quanitified using probabilities.
The word 'Bayesian' comes from the surname of Reverend Thomas Bayes (Figure 1), who was the inventor of the probability theorem on which Bayesian Networks are based - Bayes Theorem of Conditional Probability.
Figure 1: Thomas Bayes (1702 - 1761)
Bayesian Networks have two basic parts: a qualitative part and a quantitative part. The qualitative part is called an influence diagram and is made up of three components: nodes, states and links. The quantitative part consists of probabilities, which are stored within Conditional Probability Tables (CPTs) that sit behind each node within a BN.
To explain further, take the BN in Figure 2. It has two nodes, Weather and Forecast, each with two states. There is a link from Weather to Forecast, which represents the logic that the weather influences the weather forecast. In Bayesian networks, a node is called a child node if it sits at the affect side of a link and a parent node if it sits at the cause side of a link. Hence, in Figure 2, Forecast is a child node and Weather is a parent node. In complex networks, a node can be both a child and and parent.
Behind the Forecast node there is a Conditional Probability Table (CPT) that stores the probability of a Sunny or Cloudy Forecast for a Rainy day and the probability of a Sunny or Cloudy Forecast for a Fine day (No Rain) (see Table 1). Note that for each row in the CPT, the probabilities add to 100%. There is also a probability table behind the Weather node, but because this node does not have any parents (it is parentless), its probability table simply stores the probability of a rainy or fine (no rain) day (see Table 2).
 e
Figure 2: Simple BN
Table 1: CPT for the Forecast node in Figure 2 (probabilities are shown as percentages).
| Weather |
Sunny |
Cloudy |
| Rain |
0 |
100 |
| No Rain |
40 |
60 |
Table 2: Probability table for the Weather node in Figure 2
The probabilities stored in a BN, along with Bayes Theorem, are used to make predictions. Lets look at an example. Say we want to know the probability of rain given that there has been a cloudy forecast. We can use the BN in Figure 2 to answer this question by selecting Cloudy in the Forecast node (see Figure 3). The BN uses Bayes Theorem to update the probabilities in the Weather node to give us the answer: If the forecast is cloudy then there is a 62.5% chance that their will be rain.
Figure 3: BN with scenario inserted (Cloudy selected for Forecast)
Lets see how this answer is calculated using Bayes Theorem:
Bayes Theorem states the following:
P (B|A) = [P (A|B) × P (B)] / P (A)
which in plain English means that the probability of B occurring when A occurs, P (B|A), is equal to the probability of A occurring when B occurs, P (A|B), multiplied by the probability of B occurring, P (B), and then divided by the probability of A occurring, P (A).
We want to know the probability of rain given that there has been a cloudy forecast. In other words, we want to know
P(Rain|Cloudy). So to solve the problem using Bayes Theorem, we simply need to substitute Rain for B and Cloudy for A:
P (Rain|Cloudy) = [P (Cloudy|Rain) × P (Rain)] / P (Cloudy)
Using the probabilities in tables 1 and 2 above, we know that P (Cloudy|Rain) = 1 (or 100%) and P (Rain) = 0.5 (or 50%). Tables 1 and 2 do not give us the probability of a cloudy forecast, P (Cloudy), however it is not too difficult to work this out. The probability of a cloudy forecast is simply the probability of a cloudy forecast for a rainy day, multiplied by the probability of a rainy day; plus the probability of a cloudy forecast for a fine day (no rain), multiplied by the probability of a fine day (no rain). In other words:
P (Cloudy) = [P(Cloudy|Rain) × P(Rain)] + [P(Cloudy|No Rain) × P(No Rain)]
Therefore, P (Cloudy) = [1 x 0.5] + [0.6 x 0.5] = 0.8
So, putting the probailities into Bayes Theorom gives us:
P (Rain|Cloudy) = [1 × 0.5] / 0.8 = 0.625 or 62.5%
BNs have been used within a wide variety of discipines as a predictive and diagnistic tool due to their flexibility and ability to accomodate uncertainty (because they use probabilities). These discipines include:
- Aerospace
- Agriculture
- Crime Risk Assessment
- Computer Science and Information Technology
- Defence and Intelligence
- Engineering
- Environmental and Natural Resource Management
- Finance and Insurance
- Forensic Science
- Medicine and Biotechnology
- Mining and Exploration
- Marketing
- Robotics and Atificial Intelligence
- Terrorism Risk Assessment
To learn more on Bayesian Networks and to see example of their applications, please visit our online tutorials section.
|