Influence diagrams are an alternative decision analysis representation. They attempt to compactly represent the relationships (influences) between uncertain events, decisions, and outcomes.

• Often much more compact than decision trees
• Highlight influence relationships that can be meaningful and important, especially conditional independence.
• Often easier to build than trees, because information is used in more natural forms (e.g. sensitivity and specificity, prevalence, etc.)

• More difficult to solve (and get decision recommendations from) than trees
• Less commonly used, less familiar to readers, less software available.

There are two approaches to solving influence diagrams. The first involves converting the diagram to a tree. The DATA 3.0 software package, for example, does this.

The second approach is to use the following algorithm:

1. Remove all nodes (except the value node) that don’t point to any other nodes. They don’t matter to the problem any more.
2. If there is a decision node pointing into the value node, and all other nodes pointing to the value node also point to that decision node, you now have all of the information needed to make the decision. Make the decision and remove the decision node. Whenever you remove any node, anything pointing to that node now points to wherever the removed node used to point. Go back to step 1.
3. If there is a chance node that points only into the value node, the uncertainty it represents is only revealed after all decisions are made. Remove this node by averaging (changing the table associated with the value node to no longer refer to the chance node, but to refer to nodes that point to the chance node). Go back to step 2.
4. Find a chance node that points into the value node and not into any decision node. This represents an uncertainty that will also be revealed only after all decision are made, but we must reverse the direction of the arcs pointing from this node to other chance nodes in order to continue. Reversing an arc is essentially applying Bayes’ Theorem -- we go from knowing p(Test+|Disease+) (sensitivity) information to p(Disease+|Test+) (positive value of predictive test) information. When this is done, the chance node will only point to the value node (and will get removed by step 3), but go back to step 2, just in case.

Because the evaluation algorithm is so ugly and counterintuitive (compared to folding back a tree, anyway), it’s almost always either done by computer or skipped in favor of converting to a tree and solving the tree.

Markov models compactly represent situations in which there is an ongoing risk of a patient moving from one state of health to another. We assume that there are a set of possible health states, and specify the probability per unit of time that a patient in a given state will "transition" to each possible state. These transition probabilities can depend on the current time (for example, the chance of death increases with time due to aging, independent of health). We also need to know the utilities of the states. Utilities may also be a function of the time at which they’re entered (if, for example, utilities for health states are discounted, bad health sooner is worse than bad health later.) What we do assume, however, is that we have no memory -- how we came to this state doesn’t matter, only when we came.

Markov models are often represented using two figures:

1. A state diagram, which shows the possible states as nodes and arrows indicating possible transitions between states (for example, it’s generally not possible to transition out of the "death" state).
2. A transition-probability matrix that shows the probability of transitioning from one state to another.

(Actually, you can show all of the information using either a diagram or a matrix, but both are often employed because they each make certain kinds of questions and operations easier. This should sound familiar.)

To evaluate a Markov model, we can imagine a hypothetical patient who begins at some state, and then we can follow that patient until death. Each year of life, the patient gains the utility associated with the state s/he’s in (and possibly the time, if utility is discounted). Each year of life, the patient has a given probability of transitioning to a new state. When the patient’s dead, we examine his/her accumulated utility. If we repeat this simulation for a few thousand patients, we can get a pretty good idea of the total expected utility associated with a life beginning at the initial state. (We can also measure variance, confidence intervals, etc.) This approach to evaluating Markov models is called "monte carlo simulation".

Another way to think about this is to imagine a few thousand patients, and use the transition probabilities to apportion them into groups that transition into different states, adding up the total utility for each patient in each group. This is called "cohort simulation". You don’t get variance measures this way, though.

If you have two different cohorts of patients with different utilities, you run the simulation separately for each group. If utilities depend on past history, you can also create separate states associated with each past history.

If the transition probabilities don’t change with time, you can get an exact solution, without simulation, using matrix algebra.

The article also discusses Markov-cycle trees, a way to represent the information that’s more typically available clinically (e.g., the chance of death following surgery due to infection, rather than the overall chance of death following surgery.) These are like recursive decision trees, with only chance nodes.

The Kattan et al. article shows an example of a Markov model. (Discuss here, with focus on the Markov model).

Next week is Spring break. No class meeting.

The following week we get into cost-effectiveness analysis in depth. The computer exercise is not yet available, but keep checking the web page next week for it.