As we’ve said before, Bayes’ Theorem provides a normative method for updating one’s beliefs with new information. However, Bayes’ theorem becomes hopeless when there are many, possibly correlated, cues to a judgment. In these situations, we turn to other models: regression, probabilistic mental models, and neural networks.

Multiple regression is a statistic procedure in which we try to express the variable we want to predict (which I’ll call the "outcome" variable for short) as a function of the cues we know. We look for the function that either minimizes the (squared) error -- the difference between what we predict and the actual outcome, or maximizes the probability that the outcome would have resulted given the cues.

Most multiple regression is multiple linear regression. We assume that the function that relates the cues to the outcome is a linear function. That is, we predict the outcome by adding up some multiple of each cue. For example, we might predict a judgment of how attractive a college candidate is, on a scale of 0-200 (roughly) by using SAT*0.1 + GPA*25 - 60. as our equation. This says that increasing your GPA by 1 point will increase your attractiveness by 25 points, while increasing your SAT by 100 points will increase your attractiveness by 10 points. This suggests that GPA is 250 times more important than SAT scores, but this isn’t true because GPA has much less room to vary than SAT scores.

One typically feeds cue and outcome information for a number of cases into a computer and the computer generates the coefficients for each cue.

The predictive value of a positive test is a function of the prevalence and test characteristics. According to Bayes’ Theorem, it’s:

If we instead tried to do a multiple regression, we’d get an equation like this:

This looks quite different, and will be systematically biased. On the other hand, it’ll be close, especially when we’re in the realm of multiple correlated tests. Linear equations usually capture most of the important relationships.

A major finding with regards to judgments made by these linear models is that they’re almost always better than judgments made by people! The models predict the outcomes better than an intuitive judgment by a human judge because they (1) derive more appropriate weights for each cue (the regression coefficients) and then (2) more consistently apply these weights. Human judges are often swayed from consistency by irrelevant individual features of the case they’re judging. If we can’t model the outcomes themselves, but we can only model the judge’s judgments (called bootstrapping or policy capturing), we still gain the benefit of (2).

A second major finding is that most of the benefit of linear models in practice depends on having the right cues and giving them the right sign (i.e., better MCAT predicts better success in medical school, not worse) -- the actual coefficients are less important. Robyn Dawes has shown that in some cases, unit-weighted models (where all coefficients are 1 or -1) or even random-weighted models (where coefficients are determined randomly, apart from sign) can outperform human judges. (In these models, the cues are standardized -- divided by their standard deviations -- so they’re all on a similar scale).

Gigerenzer and Goldstein propose an alternative account of quantitative judgment, based on simple "fast and frugal" heuristics, also called probabilistic mental models. They argue first, that people don’t use regression intuitively (because they don’t have access to all of the information and computing power needed to do so), and second, that people don’t need to (because cognition has adapted to the structure of cues in the environment). In their studies, they find that their heuristics can perform as well as regression with a fraction of the memory and computational costs. Is this how people really make decisions? Is it appropriate?

Neural networks are another approach to making judgments based on multiple, possibly redundant or interacting, cues. A neural net consists of a set of nodes ("neurons") that are connected to one another. Each connection has a weight. When a node is activated, it transmits activation to other nodes to which it’s connected, weighted by the connection weight. If a node receives enough activation in total, it activates.

The most common form of neural network, a so-called "feed-forward hidden-layer network" is usually structured in 3 layers. The input layer is a set of nodes that are activated by the data in the case. For example, you might have input nodes representing the presence or absence of symptoms, chemistry levels, etc. The input layer is connected to a "hidden layer" of nodes, and the hidden layer is connected to an output layer. The nodes in the output layer represent the prediction of the network. For example, there might be a single output node whose level of activation represents the predicted life expectancy of the patient, or many output nodes representing different outcome variables.

A neural network works by pattern matching. It must be trained by presenting a set of cases to the input layer, and then providing feedback about how closely the output layer predicts the actual outcomes. The weights of the connections are adjusted to make the prediction better (to minimize error in prediction) -- the most common algorithm for doing this is called "back propagation". The training set is repeated until the network performs sufficiently well. Then, the network is validated with an independent set of data (with outcomes) to see how well it has generalized the training set. If it performs well enough, the network can be used as a decision aid -- data with unknown outcomes can be fed in and the predicted outcome used to make clinical decisions.

In one sense, neural networks are a very fancy form of nonlinear multiple regression. Because they are often so complex, some people are uncomfortable with them -- it’s difficult to explain how the network performs so well, because its knowledge is distributed among the connections. On the other hand, they do perform very well. A very useful tutorial article by Penny and Frost (Med Dec Making 1996; 16:386-398) lists 25 neural network studies and compares the network’s accuracy with that of the best other method. In 5 of the cases, the other method was more accurate. In the vast majority, however, the neural net was more accurate, sometimes much more accurate.