Research on posttraumatic stress disorder (PTSD) over the last twenty years has identified a large number of factors, from multiple domains, that increase the risk of development of PTSD, given exposure to a traumatic stressor. However, none of the factors, alone, predict PTSD with sufficient accuracy to facilitate important clinical decisions, such as: Should this rape survivor seen in the emergency department be referred for diagnostic assessment of PTSD? Is this person seen in a psychologist’s office one week after a disaster at high enough risk of chronic PTSD to justify early intervention?
Clinical prediction rules are statistical tools that “quantify the individual contributions that various components of the history, physical examination and basic laboratory results make toward diagnosis, prognosis, or likely response to treatment in an individual patient”(McGinn et al., 2000, p. 80). They help clinicians to integrate information to improve decisions when information is complex, when individual risk factors do not predict diagnosis or prognosis with sufficient accuracy to facilitate treatment decisions or when diverse data must be incorporated into a decision about diagnosis, prognosis or treatment. Although prediction rules are used widely in clinical medicine, few prediction rules have been developed or used in the mental health field. In recognition of the potential utility of prediction rules in the field of traumatic stress, the National Institute of Mental Health recently released a Request for Applications to stimulate development of prediction rules for PTSD (Network(s) for Developing PTSD Risk Assessment Tools, 2008). A rule that predicts which survivors of acute trauma are likely to develop chronic PTSD may be invaluable for early intervention. A rule that predicts treatment response might be helpful in tailoring treatments for trauma survivors with PTSD.
There are four steps involved in creating and testing a prediction rule. The first, and most important, step is to carefully define the purpose of the rule. Some rules are designed to identify persons who are at high risk of having a disease or disorder and should therefore undergo further diagnostic testing. A good example of this type of rule is the one developed by Wells and colleagues for diagnosis of pulmonary embolism (blood clot in the lungs; Wells et al., 1998). Based on the combination of seven elements from history and physical exam, none of which are individually useful for diagnosis, patients who have a high probability of pulmonary embolism can be identified and sent for definitive evaluation using diagnostic imaging. Other rules are designed to identify persons at low risk of disease or disorder who can safely forego the expense and/or risk of additional diagnostic testing. This is an important distinction because a rule designed to detect disease should have a high positive predictive value while a rule designed to detect non-disease should have a high negative predictive value (i.e., a high probability of no disease when the rule predicts no disease). A good example of this type of rule is the Ottawa ankle rule, which identifies, among persons with ankle sprains, those who have such a low risk of ankle fracture that they do not need to have x-rays (Stiell et al., 1993).
The second step is derivation of the rule. Derivation involves development of a mathematical model that incorporates individual elements from history, physical, lab, or imaging studies to predict the outcome (diagnosis, prognosis, or response to treatment). Typically, the investigator starts with a large pool of potential predictors and then determines which of those factors predict the outcome of interest in a large sample of people. A variety of mathematical techniques, including regression modeling, neural networks and signal detection theory, have been used to identify the important predictors. After the model (i.e., the list of items that predict the outcome with a high degree of accuracy) has been developed, it needs to be simplified to make it easy for clinicians to use. The easiest way to do that is to assign points to each item, based on the weight of the items in the model. The score on the predictive rule is then simply the sum of the points.
The third step is validation of the rule. Validation involves testing the rule in a sample different from the one used to derive the rule to see whether the model predicts the outcome as well as it did in the original sample. This is done to make sure that the factors included in the model were not included by chance, or because of idiosyncrasies of the original sample.
The fourth step, impact analysis, is testing to see whether implementation of the rule, in real life clinical practice, has the expected effects. Typically, impact analyses are done with randomized trials, in which outcomes are assessed in clients whose clinicians are randomized to use or not use the rule. For example, suppose a rule were developed to detect, in an emergency department setting, the small percentage of people likely to develop chronic PTSD, from among the many who have PTSD symptoms in the acute aftermath of trauma. An impact analysis would then assess whether use of the rule successfully identifies those who are truly at high risk of chronic PTSD, and, most importantly, whether use of the rule leads to improved outcomes (through provision of early intervention to those identified as being at risk).
Prediction rules, when used to supplement, not supplant, clinical judgment can help busy clinicians make important decisions. Development of prediction rules in the field of traumatic stress may be able to facilitate the promise of early intervention and reduce the burden of chronic PTSD.
McGinn, T.G., Guyatt, G.H., Wyer, P.C., Naylor, C.D., Stiell, I.G., & Richardson, W.S. (2000). Users’ guides to the medical literature. XXII: How to use article about clinical prediction rules. Journal of the American Medical Association, 284, 79-84.
Network(s) for developing PTSD risk assessment tools. (2008, April 7). Retrieved September 1, 2008, from http://grants.nih.gov/grants/guide/rfa-files/RFA-MH-09-060.html
Stiell, I.G., Greenberg, G.H., McKnight, R.D., Nair, R.C., McDowell, I., Reardon, M., et al. (1993). Decision rules for the use of radiography in acute ankle injuries. Refinement and prospective validation. Journal of the American Medical Association, 269, 1127-1132.
Wells, P.S., Ginsberg, J.S., Anderson, D.R., Kearon, C., Gent, M., Turpie, A.G., et al. (1998). Use of a clinical model for safe management of patients with suspected pulmonary embolism. Annals of Internal Medicine,1998; 129, 997–1005.