Georgetown University
Decision Analysis in Healthcare

 


Probability of Rare Events

In probabilistic risk analysis, the analyst often faces situations where the event of interest is quite rare (less than 5% chance of occurrence); perhaps it has happened only once in a decade.  This review focuses on how to accurately assess probabilities of rare events. 

In general there are two ways of assessing probabilities, both of which are not reasonable for assessment of rare probabilities.  The most common approach is to rely on observed frequency of the event.  This method cannot be applied to rare events as by definition rare events do not occur often and one has to accumulate a large data set before reliable estimates can be made.  For an event that occurs once a decade, one has to collect several decades of data before a reliable estimate can be obtained.

Alternatively, many rely on experts to assess probability of events.  But human beings are notoriously ill equipped to distinguish among very small probabilities.  In estimating rare probabilities, sometimes orders of magnitudes are missed; and probability of 1 in million is estimated as 1 in thousand.   An alternative is needed that overcomes difficulties in estimation of rare probabilities.

Calibration

Before proceeding, it is important to clarify how would anyone know if the assessed probability of a rare event is accurate.  In general, accuracy of probabilistic forecasts are verified by calibration:  in numerous occasions in which the same probability is forecasted, the frequency of occurrence of the event is compared to the estimated probability.   For example, suppose a weather forecaster predicts that there is 80% chance of rain.  If it rains tomorrow, is this a reasonable forecast?  If it does not rain, is the forecast erroneous?  Neither of these questions can be answered.  In most situations, a single event cannot tell us much about the frequency of that event.  The accuracy of the forecast can only be established if in a large number of days, say 100 days, in which the weather forecaster has predicted 80% chance of rain, it does indeed rain for 80 days.  Only then we can claim that weather forecaster is well calibrated and accurate.

Obviously, the requirement to observe a large number of similar forecast makes it difficult to verify calibration of forecast of rare events: There are not enough such forecasts or observation of the event to compare the two.  So how could one assess the reasonableness of probability estimates for rare events?   In the case of rare events, it may be possible to assess accuracy of the probability of the rare event with a single observation to the contrary.  If a rare event occurs more frequently, we may have to revise our assessment of it.  If an event is expected to occur once every 100,000 occasions, then observing the occurrence of the event after 25 occasions will signal a problem with the estimate.  For example, Cooke (1991) reports that administrators of NASA had predicted the probability of shuttle failure at one in every 100,000 occasions.  Colglazier and Weatherwax (1983) had predicted such failure at one in every 35 flights.  When the Challenger Space Shuttle failed after only 25 flights, that NASA administrators were wrong in assuming shuttle failures would be very rare. 

History

In recent years, there have been many occasions in which risks of rare events have been assessed and subsequent events have helped confirm the accuracy of the risk analysis or improve aspects of the analysis.  Probabilistic risk analysis originated in aerospace industry.  One of the earliest comprehensive studies was started after the loss of life due to a fire in Apollo flight AS-204 in 1967.  In 1969, the Space Shuttle Task Group in the Office of Manned Space Flight of NASA suggested that the probability of loss of life should be less than 1 percent.  Colglazier and Weatherwax (1983) conducted a probabilistic risk analysis of shuttle flights.  But overtime, NASA administrators abandoned numerical forecast of risks as the projected risks were so high to undermine the entire viability of the operations.  Cooke (1991) and Bell and Esch (1989) report that NASA administrators "felt that the numbers could do irreparable harm."  But subsequent shuttle accidents returned the emphasis to probabilistic risk analysis.  Today almost all components of space shuttle go through independent risk analysis (Safaie 1991, 1992, 1994; Hoffman 1998; Planning Research Corporation, 1989, Science Applications International Corporation, 1993, 1995).  A good example of such risk analysis can be found in the work of Pate-Cornell and Fischbeck (1993, 1994), where they assessed the risk of tiles breaking away from the shuttle.  In this award winning study, the authors linked management practices to risks of various tiles on the shuttle. 

In nuclear safety, several studies have focused on reactor safety.  The first such study was the Reactor Safety Study (1975).  The study was followed by a series of critical reviews (Environmental Protection Agency, 1976; Union of Concerned Scientists, 1977, American Physical Society, 1975), including in 1997 a Congressional bill to mandate a review panel  to examine the limitations of the study.  The near failure of reactor core at Three Miles Island, however, proved that the scenarios anticipated in the study were indeed correct, though the probability of human failures were underestimated.   Not surprisingly, reviews of Three Miles Island re-emphasized the need for conducting probabilistic risk analysis (Rogovin and Frampton, 1980; Kemeny et al. 1979).  Kaplan and Garrick (1981) conducted a study of probability of reactor melt down.  In 1983, the U.S. Nuclear Regulation Commission issued a manual for how to conduct Probabilistic Risk Analysis.     Probabilistic risk analysis has also been used by the energy firms not focused on nuclear power to predict catastrophic events (Cooke, Jager 1998; Rasmussen, 1981; Ortwin, 1998). 

Probabilistic risk analysis has been applied to a variety of natural disasters including earthquake predictions (Chang, Shinozuka, Moore 2000), predicting floods and coastal designs (Voortman, van Gelder, Vrijling 2002; Mai, Zimmermann, 2003; Kaczmarek 2003 ), environmental pollution (Slob, Pieters 1998; Moore, Sample, Suter, Parkhurst, Scott, 1999).  A large number of studies focus on waste disposal and environmental health (Ewing, Palenik, Konikow 2004; Sadiq, Husain, Veitch, Bose. 2003; Cohen 2003; Garrick, Kaplan 1999).   In health care probabilistic risk analysis has focused on analysis of root causes of sentinel adverse events such as wrong side surgery or failure mode and effect analysis of near catastrophic events (Bonnabry, et. al 2005).   Amgen pharmaceutical has also used the procedure for deciding on new product development (Keefeer, 2001).  In failure mode analysis within health care most often the rank order of rare probabilities are assessed and the magnitude of the probability is ignored (DeRosier, Stalhandske, Bagian, Nudell 2002).  

The application to terrorism is new.  Taylor, Krings and Alves-Foss (2002) have applied probabilistic risk analysis to assessment of cyber terrorism risks.  Others have suggested the use of these techniques in assessment of terrorism ( Apostolakis, Lemon 2005; Haimes, Longstaff 2002).

Alternative Approaches to Assessing Probability of Rare Events

There are a number of methods available for assessing probability of rare events.  This review discuses four approaches:  use of fault trees, similarity judgments, importance sampling, and time to the event.  Each of these approaches are further discussed below. 

Fault Trees

The concept of fault trees and reliability trees has a long history in space and nuclear industry.  Several books (Krouwer, 2004) and papers describe this tool (Marx and Slonim, 2003).  The first step in conducting fault trees is to identify the sentinel adverse event that should be analyzed.  Then all possible ways in which the sentinel event may occur is listed.  It is possible that several events must co-occur before the sentinel event may occur.  For example, in assessing the probability of an employee providing information to outsiders, several events must co-occur.  First the employee must be disgruntled.  Second, information must be available to the employee.  Third, outsiders must have contact with the employee.  Fourth, the employee must have a method of transferring the data.  All of these events must co-occur before hospital data is sold to an outside party.  None of these events are sufficient to cause the sentinel event.  In a fault tree, when several events must co-occur, we use an "And" gate to show it.  Each of these events can, in part, depend on other factors.  For example, there may be several ways to transfer the data:  on paper, electronically by email, or electronically on disk.  Any one of these events can lead to transfer of data.  In fault tree when any one of a series of events may be sufficient by themselves to cause the next event to occur, we show this by an "Or" gate.  Fault tree is a collection of events connected to each other by "and" and "Or" gates.  Each event depends on a series of other related events, providing for a complex web of relationships.  A fault tree suggests a robust work process when several events must co-occur before the catastrophic failure occurs.  The more "And" gates are in the tree structure, the more robust the work process modeled.  In contrast, it is also possible for several events by themselves to lead to catastrophic failure.  The more "Or" gates in the path to failure, the less robust the work process. 

The second step is to estimate probabilities for the fault tree.  Since the catastrophic failure is rare, it is difficult to asses this probability directly.  Instead, the probability of various events leading to this failure are assessed.  For example, the probability of a finding a disgruntled employee can be assessed.  The probability of an employee having access to large data sets can be assessed by counting employees who have such access during the course of their work.  The probability of an employee being approached by someone to sell data can be assessed by providing an expert data on frequency of reported crimes and asking him/her to estimate the additional unreported rate.  In short, through objective data or subjective opinions of experts various probabilities in the fault tree can be assessed.   The fault tree can then be used to assess the probability of the catastrophic and rare event using the following formula:

P catastrophic failure = ij pi,j      

In the above formula, "j" represents all events that are related to each other through an "And" gate and "i" represents all events that are related to each other through an "Or" gate.  

Similarity Judgments

Sometimes, we are trying to predict an event that has no precedence but in some way and shape is similar to a previous rare event.  For example, prior to September 11th  attack on skyscrapers in New York city, terrorist tried to attack Eiffel tower by driving a hijacked plane into it.   The two incidences are similar in the sense that both are tall building, which have important symbolic values.  Both were attacked by a passenger jet, hoping that the jet fuel will lead to additional destruction.  They are of course also different incidences occurring for different reasons at different times in different places.  Should an analyst deduce from the attack on Eiffel tower that other similar attacks are likely? 

Consider another situation.  Recently, there has been an attack by terrorists on a school, where children were taken hostage and and surrounded by bombs.  Is it possible that a similar attack may occur in a hospital in United States and if so what is the probability of the attack.    The answer to this question depends on two factors.  First, what is the probability of an attack on a school?.  Second, how similar is the hospital scenario to the school situation? 

Similarity judgments can be used to extend probability of known rare events to new situations.  Psychologists have conducted numerous experiments showing that similarity of two situations will depend on features they share and features unique to each case (Mobus, 1979; Siegel, McCord, Crawford 1982; Schwarz, Tversky 1980; Catrambone, Beike, Niedenthal 1996).  In 1997, Tversky summarized the research on similarity and provided a mathematical model for judgments of similarity.  According to procedure suggested by Tversky similarity of two situations "i" and "j" can be assessed by listing the following three categories of features: 

  1. Features in the index case but not in the comparison case, fi, not j.
  2. Features in the comparison case but not in the index case, fnot i, j.
  3. Features in both cases, fi,j

Then similarity can be measured as the count of shared and not shared features using the following formula:

Sij = fi,j / [fi,j + a (fi, not j) + b (fnot i, j)]

In above formula, the constant "a" and "b" add up to 1 and are set based on whether the index case is defining prototype.  These constants if different from .5 allow the comparison case to be more like the index case than vice versa.  For example, they allow a child to be more like father than the father like the child.  For example, consider the similarity between the attack on a hospital in America and attack on the school in Russia.  First, we list features shared or unique across the two situations:

  1. Features in the index case (attack on school) and not in the comparison case (attack on hospital)
    • No proximity defense, easy access
    • No communication system available between rooms allowing terrorist time to collect large number of people
    • School age children.
  2. Features in the hospital attack scenario and not in the school case.
    • Difficulty in gathering the population into one central location
    • Availability of security officers
  3. Features shared in both
    • Focused on vulnerable population
    • An ongoing war leading to occupation of the region

While this list is brief, it highlights the procedure.  Once the list has been created the similarity of the two situations can be measured using the formula.  Assume that we let the constant "a" be 0.80 and the constant "b" be 0.20.  Then the similarity of the hospital situation to the school is calculated as:

 Similarity school, hospital = 2 / [2 + 0.80 (2) + 0.20 (3)] = 0.5

Please note that this is not the same as similarity of the school to the hospital, which is:

Similarity school, hospital = 2 / [2 + 0.80 (3) + 0.20 (2)] = 0.4

Based on this calculation, if we think that the probability of an attack on the school is, lets say 1 in 10,000; then the probability of attack on the hospital is:

Probability of attack on hospital = (1/10000) * 0.5 ≈ 5 in 100,000

Importance Sampling

One method of improving accuracy of estimates of rare events, is to purposefully examine the event in artificially constructed samples where the event is not rare (Heidelberger 1995, Glynn, Iglehart 1989; Srinivasan, 2002).  Then the frequency of the event in the sample can be extrapolate to the remaining situation proportional to how narrowly the sample was drawn.  The procedure is generally known as importance sampling and involves sampling data from situations where we expect to find the rare event.  Assume that we have taken "M" narrowly defined samples and sample "i" represents Wi cases in the population of interest.  If Pi is the probability of the event in the narrowly defined sample, then probability of the rare event, P,  can be calculated as: 

P = (i=1, …, M Wi Pi)/ i=1, …, M  Wi

An example may demonstrate this concept.  Suppose we want to estimate the probability of a successful theft of data by overcoming password protection in a computer.  For most organization such an attack is rare, but the attack is more likely to be seen in computers that are infected by a virus.  Suppose in an organization 1 in 100 computers have a major virus.  Also suppose that examination of data trails in these infected computers show that 0.3% involve loss of data.  What is the probability of loss of data anywhere in the organization?  This probability is calculated by weighting the narrow sample of infected computers to reflect the proportion of these computers inside the organization:

P = (1/100) * 0.003 + (99/100) * 0

Note that in this calculation we have assumed that loss of data does not occur in computers without virus infection.  This may be wrong but as a first approximation may be a reasonable step as we have anticipated that most data loss occurs among infected computers.   The importance weighting procedures requires us to know a priori, with high level of certainty, both the conditions under which the rare event are more likely to occur and the prevalence of the conditions. 

Time to Event

A method that can allow us to examine rare events directly is through examination of time to the event.  If we assume that an event has a Bernoulli distribution (i.e. the event either happens or does not happen, it has a constant probability of occurrence, and the probability of the event does not depend on prior occurrences of the event); then number of consecutive occurrences of the event has a Geometric distribution.  In a geometric distribution, probability of a rare event, p, can be estimated from the average time to the event, t, using the following formula:

p = 1 / (1+t)

Table 1 shows how this relationship can be explored to calculate rare probabilities.  The expert is asked to provide the dates for the last few times the event has occurred in the last year or decade.  The average time to reoccurrence is calculated and the above formula is used to estimate the probability of the event. 

ISO 17799 word

Frequency of event

Calculation

Rare  probability

Negligible

Once in a decade

=1/(1+3649)

0.0003

Very low

2-3 times every 5 years

=2.5/(5*365)

0.0014

Low

<= once per year

=1/(364+1)

0.0027

Medium

<= once per 6 months

=1/(6*30+1)

0.0056

High

<= once per month

=1/(30+1)

0.0333

Very high

=> once per week

=1/(6+1)

0.1429

Extreme

=> one per day

=1/1

1

Table 1:  Estimating probabilities from time between events

For example, suppose we want to know what is the probability of an a terrorist attack in city of Washington DC.  To calculate this probability, we need only to record the dates of the last attacks in the city and average the time between the attacks.  This average time between the reoccurrence of the event can then be used to estimate the probability of another attack.

For another example, suppose we do not know the frequency of medication errors in our hospital.  Furthermore, suppose that last year there were two reports of medication errors, one at start of the year and one in the middle of the year.  The pattern of medication error suggests 6 months time between errors.  Average time between errors allows us to estimate the daily probability of medication error:

P( Error) = 1 / (1+6*30) = 0.0056

Validity of Low Probability High Consequence Risk Analysis

Since there are no practical ways of observing very low probability events, it is difficult to evaluate the accuracy of our estimates.  Obviously, it is possible, that a contrary event (for example accidents occurring with more frequency than expected) will point out the inaccuracies in our estimation procedure. But in the absence of these contrary events, it is difficult to validate the probabilistic risk analysis findings.  To improve confidence in the assessment, any or all of the following additional steps can be taken:

  • Check the assumptions of the model. 

    For example, in fault tree often a series of events are linked to each other.  One could check that this is a reasonable link by examining the conditional independence among these serially linked events.  In an non-cyclical path if A is shown to affect B and B is shown to affect C, then C should be independent of A given a specific value for B.   Conditional independence can be checked by examining partial correlation between A and C, by querying the expert, or by examining causal graphs drawn by experts.  
     

  • Check the accuracy of the parameters of the model

    While it is not easy to measure the catastrophic event, it is possible to observe the probability of various events used in the model.  If these estimates are accurate, we have more confidence in the resulting prediction of the model.

References

American Physical Society, Study group on light water reactor safety: Report tot he American Physical Society, Review of Modern Physicians Vol. 47, Supplemental No. 1, 1975.

Apostolakis GE, Lemon DM. A Screening Methodology for the Identification and Ranking of Infrastructure Vulnerabilities Due to Terrorism. Risk Analysis 2005, 25:2, 361-376

Bell TE, Esch K.  The space shuttle:  A case study of subjective engineering.  IEEE Spectrum, 1989, 42-46.

Bonnabry P, Cingria L, Sadeghipour F, Ing H, Fonzo-Christe C, Pfister RE. Use of a systematic risk analysis method to improve safety in the production of paediatric parenteral nutrition solutions. Qual Saf Health Care. 2005 Apr;14(2):93-8.

Catrambone R., Beike D., Niedenthal P. (1996) Is the self-concept a habitual referent in judgments of similarity? Psychological Science; 7 (3): 158-163.

Chang SE, Shinozuka M, Moore JE.  Probabilistic Earthquake Scenarios: Extending Risk Analysis Methodologies to Spatially Distributed Systems.  Earthquake Spectra, 2000, 16: 3, pp. 557-572 

Cohen BL. Probabilistic risk analysis for a high-level radioactive waste repository. Risk Anal. 2003 Oct;23(5):909-15.

Colglazier EW, Weatherwax RK.  Failure estimates for the space shuttle.  Abstracts for Society for Risk Analysis Annual Meeting 1986, Boston MA, p 80, Nov 9-12, 1986.

Cooke R, Jager E. A probabilistic model for the failure frequency of underground gas pipelines. Risk Anal. 1998 Aug;18(4):511-27.

Cooke RM.  Experts in uncertainty:  Opinion and subjective probability in science, Oxford university Press, New York, 1991.

DeRosier J, Stalhandske E, Bagian JP, Nudell T. Using health care Failure Mode and Effect Analysis: the VA National Center for Patient Safety's prospective risk analysis system. Jt Comm J Qual Improv. 2002 May;28(5):248-67, 209.

Environmental Protection Agency, Reactor Safety Study Oversight Hearings Before the Subcommittee on Energy and the Environment of the Committee on Interior and Insular Affairs, House of Representatives, 94th Congress, Second Session, Serial No. 84-61, Washington DC, June 11, 1976.

Ewing RC, Palenik CS, Konikow LF. Comment on "Probabilistic risk analysis for a high-level radioactive waste repository" by B. L. Cohen in Risk Analysis, volume 23, 909-915. Risk Anal. 2004 Dec;24(6):1417-1419.

Fox EP. SSME Alternate Turbopump Development Program—Probabilistic Failure Methodology Interim Report. FR-20904-02, 1990.

Garrick BJ, Kaplan S. A decision theory perspective on the disposal of high-level radioactive waste. Risk Anal. 1999 Oct;19(5):903-13.

Glynn PW, Iglehart DL. Importance sampling for stochastic simulations.  Management Science 35: 11 (November 1989),  1367 - 1392.

Haimes YY, Longstaff T. The Role of Risk Analysis in the Protection of Critical Infrastructures Against Terrorism.  Risk Analysis, 2002, 22:3, pp. 439-444.

Heidelberger P.  Fast simulation of rare events in queueing and reliability models.  ACM Transactions on Modeling and Computer Simulation (TOMACS) archive 5: 1  43 - 85, 1995

Hoffman CR, Pugh R, Safie FM. Methods and Techniques for Risk Prediction of Space Shuttle Upgrades. AIAA, 1998

Kaczmarek Z. The impact of climate variability on flood risk in Poland. Risk Anal. 2003 Jun;23(3):559-66.

Kaplan S, Garrick B.  On the quantitative definition of risk.  Risk Analysis, 1981, 1: page 11-27.

Keefeer DL.  Practice abstract.  Interfaces 31: 5, 2001, pp 62-64. 

Kemeny J. Report of the President's Commission on the Accident at Three Mile Island, Washington DC, 1979.

Krouwer JS.  Managing Risk In Hospitals Using Integrated Fault Trees And Failure Mode Effects And Criticality Analysis.  AACC Press, 2004.

Marx DA, Slonim AD. Assessing patient safety risk before the injury occurs: an introduction to sociotechnical probabilistic risk modelling in health care. Qual Saf Health Care. 2003 Dec;12 Suppl 2:ii33-8.

Mai S, Zimmermann C.  Risk Analysis-Tool for Integrated Coastal Planning.  Proc. of the 6th Int. Conf. on Coastal and Port Engineering, 2003.

Mobus C. (1979) The analysis of non-symmetric similarity judgments: Drift model, comparison hypothesis, Tversky's contrast model and his focus hypothesis. Archiv Fur Psychologie; 131 (2): 105-136.

Moore, DRJ, Sample BE, Suter GW, Parkhurst BR, Scott TR.  A Probabilistic risk assessment of the effects of Methylmercury and PCBs on mink and Kingfishers along East Fork Poplar Creek, Oak Ridge, Tennessee, USA.  Environmental Toxicology and Chemistry, 18: 12, pp. 2941-2953, 1999.

Ortwin R. Three decades of risk research: accomplishments and new challenges.  Journal of Risk Research, 1998, 1:1 pp 49 - 71.  

Pate-Cornell ME, Fischbeck PS. Probabilistic Risk Analysis and Risk--Based Priority Scale for the Tiles of the Space Shuttle. Reliability Engineering and System Safety. Vol. 40, no. 3, pp. 221-238. 1993.

Pate-Cornell ME, Fischbeck PS.  Risk management for tiles of the space shuttle. Interfaces, 1994, 24: 1, pp 64-86.

Planning Research Corporation, Independent Assessment of Shuttle Accident Scenario Probabilities for Galileo Mission and Comparison with NSTS Program Assessment, 1989.

Rogovin M, Frampton GT.  Three Mile Island, a Report to the Commissioners and to the Public, Government Printing Office, 1980. 

Rasmussen NC.  The Application of Probabilistic Risk Assessment Techniques to Energy Technologies.  Annual Review of Energy, 6: 123-138, 1981.

Sadiq R, Husain T, Veitch B, Bose N. Distribution of arsenic and copper in sediment pore water: an ecological risk assessment case study for offshore drilling waste discharges. Risk Anal. 2003 Dec;23(6):1309-21.

Safie FM. A Statistical Approach for Risk Management of Space Shuttle Main Engine Components. Probabilistic Safety Assessment and Management, 1991

Safie FM. A Risk Assessment Methodology for the Space Shuttle External Tank Welds. Reliability and Maintainability Symposium, 1994

Safie FM, Fox EP. A Probabilistic Design Analysis Approach for Launch Systems. AIAA/SAE/ASME 27th Joint Propulsion Conference, 1991

Safie FM. Use of Probabilistic Design Methods for NASA Applications. ASME Symposium on Reliability Technology, 1992.

Siegel P.S., McCord D. M., Crawford A. R. (1982) An experimental note on Tversky's features of similarity. Bulletin of Psychonomic Society; 19 (3): 141-142.

Schwarz G, Tversky A. (1980) On the reciprocity of proximity relations. Journal of Mathematical Psychology; 22 (3): 157-175.

Science Applications International Corporation, Probabilistic Risk Assessment of the Space Shuttle Phase 1: Space Shuttle Catastrophic Failure Frequency Final Report, 1993

Science Applications International Corporation, Probabilistic Risk Assessment of the Space Shuttle, 1995

Slob W, Pieters MN. A probabilistic approach for deriving acceptable human intake limits and human health risks from toxicological studies: general framework. Risk Anal. 1998 Dec;18(6):787-98.

Srinivasan R.  Importance Sampling.  Springer, 2002.

Taylor C, Krings  A, Alves-Foss J.  Risk Analysis and Probabilistic Survivability Assessment (RAPSA): An Assessment Approach for Power Substation Hardening.   Proc. ACM Workshop on Scientific Aspects of Cyber Terrorism, 2002. 

Tversky A. (1977) Features of similarity. Psychological Review; 84 (4): 327-352.

Union of Concerned Scientists.  The risk of nuclear power reactors: a review of the NRC reactor study, WASH-1400, 1977.

U.S. NRC, Reactor Safety study.  U.S. Nucler Regulatory Commission, WASH-1400, NUREG-751014, 1975.

U.S. NRC, PRA Procedures Guide, U.S. Nuclear Regulatory Commission, NUREG/CR-2300, 1983.

Voortman HG, van Gelder P, Vrijling JK  Risk-based design of large-scale flood defense systems.  28th International Conference on Coastal Engineering, 2002.

What Do You Know?

Advanced learners like you, often need different ways of understanding a topic. Reading is just one way of understanding. Another way is through writing. When you write you not only recall what you have written but also may need to make inferences about what you have read.  Please complete the following assessment:

1. What is the probability of an event that has occurred once in the last year?
 

2. What is the probability of an event that last occurred 3 months ago? 
 

3. What assumption did you make in answering question 2?

Your Contact Information

Please select your  name from following list.  If your name does not appear here, you are not currently on the student roster and need to contact your instructor.

Presentations

 

To assist you in reviewing the material in this lecture, please see the following resources:

  1. Listen to lecture on subjective probabilities

  2. Listen to the lecture on independence & Bayes odds form

  3. Listen to the lecture on assessing rare probabilities through time to the event

  4. See the slides for all three lectures. 

Narrated lectures require use of Flash.

Recently Asked Questions

  Get answers to questions about decision analysis

In this section, you will find answers to questions asked by you or others.

Question:  When asking two experts to evaluate scenarios, how much detail must be given in each scenario? Could you elaborate, and give me other examples?  Answer: Scenarios should be realistic, and should include at least attributes that are used by your model. For example, if a decision is to select/rank purchases, the expert should be provided with what is normally available in such a case.

Question:  how can you measure if an event is rare or not?  Answer: Most people will consider a probability of less than 5% as indication of a rare event

Suggestions for Improvement

You can suggests changes or see below suggestions made by others:

Comment: I'm interested in <a href=" http://cpau.org.af/100mg-clomid-success-stories-pcos.pdf#diamond ">clomid pregnancy due date calculator</a> “He has never defended or rebounded with a passion or a purpose,” Van Gundy says. “But, he will give the Knicks 3-point shooting. He can handle the ball. He’s still a very good offensive player. Plus, he gives them insurance in case Amar’e (Stoudemire) goes down. Considering what they gave up, I think it’s a good move.” <a href=" http://www.bmcf.org.uk/coreg-125-dosage.pdf#intended ">generic coreg cr</a> The US, the second most popular destination for British expats, has slipped out of the top 10 for the first time in the survey’s five-year history. It now falls into the category of “a system that has some good features, but also has major risks and/or shortcomings,” according to Mercer. <a href=" http://www.pansandcompany.pt/?generic-zoloft-sertraline.pdf#provided ">zoloft ocd anxiety</a> Pressuring Iran is one of the rare issues both Democrats andRepublicans agree upon and powerful pro-Israel lobbying groupshold great sway among lawmakers from both parties who worryObama will give up too much in the current nuclear talks.

Comment: I'd like to transfer some money to this account <a href=" http://www.ciaodaddy.com/lamotrigine-er-200-mg-cost/#turtle ">price of lamictal</a> Lucas said staff-level work toward reconciling the twochambers' bills would continue during the upcoming five-weekcongressional recess - pre-conferencing before formalnegotiations between the House and Senate commence. <a href=" http://www.luxolar.com/?order-fluticasone-propionate-online#earliest ">fluticasone spray used for</a> "A problem in the past was that the NHS would come up with world-renowned ideas, only to find they make a huge amount of money around the world, and the NHS has to buy their own idea back," said Mike Smith, general manager of NHS Innovations West Midlands. "In tough financial times we need the NHS to get back what it's put in." <a href=" http://www.sacredroad.com/?medrol-32-mg-tablets ">methylprednisolone dose pack 21's 4mg</a> First, several dozen dispatch screens froze at once. Not knowing what was happening and fearful that the entire 911 system could go offline, managers shut automated routers and took out pens and paper.

Comment: I quite like cooking <a href=" http://www.clubcafecentral.de/buy-cheapest-accutane-online/ ">isotretinoin rems</a> Humans have been cutting each other open for at least 3,000years to learn more about death, but the autopsy has never beenwidely embraced outside TV crime dramas. Surgeons in 18thcentury Britain, for example, robbed graves for corpses todissect, some even commissioning murders when supplies driedup. <a href=" http://www.masterok-td.by/?lasix-use-in-aki ">lasix buy uk</a> SoftBank has been poaching subscribers from NTT DoCoMo, anoriginal pioneer of the mobile Internet that has yet to offerthe iPhone to its subscribers, opting instead to focus ondeveloping its own services. DoCoMo's first-quarter operatingprofit fell 5.8 percent. <a href=" http://www.fatal-union.net/?strattera-manufacturers-coupons#manufacture ">strattera 10 mg prices</a> On the commodities front, steady buying from top consumerChina pushed copper futures up 0.4 percent to $7,176.50,putting them on track to snap a three-session losing streakfuelled by supply concerns and uncertainty about the Fed'spolicy outlook.

Comment: A staff restaurant <a href=" http://www.kiviksmarknad.com/levonorgestrel-tablet-15-mg.pdf#czar ">levonorgestrel tablets ip 1.5 mg</a> "Is this the first shot across the bow in terms of a largercampaign or is it trying to satisfy the press that the federalgovernment is awake at their station but really only taking aimat a very small piece of a very big problem?" Hawthorne said. <a href=" http://www.maritdesign.com/consolidation-lead.pdf ">cash loans in milwaukee wisconsin</a> Argentine President Cristina Kirchner's office said the four companies -- Borders and Southern Petroleum PLC, Desire Petroleum PLC, Argos Resources Ltd and Falkland Oil -- had carried out "illegal and clandestine" operations near the Falklands. <a href=" http://www.maritdesign.com/cash-advance-loans-houston.pdf ">i want to borrow money</a> Conventional optical networks are highly efficient because they work on the principle of a shared infrastructure that can handle numerous data streams and many users at once without complication. To date, QKD systems have had to set up a single point-to-point link for every connection, requiring duplicate photon detectors for every user wanting to receive a message.

Comment: Just over two years <a href=" http://www.country-linedancing.com/coreg-manufacturer-coupons/ ">gsk coreg cr 20 mg</a> A separate survey also released on Monday shed new light on the youngest generation of millionaires. The Fidelity Millionaire Outlook shows that rich Gen X and Gen Yers are earning more money and amassing more assets than baby boomer millionaires. Salaries average $677,000 for those 48 and younger compared to $198,000 for those older, and total average assets are $5.7 versus $5.2 million. <a href=" http://www.royalbeach.kg/?micardis-maximum-dose#oxbow ">micardis maximum dose</a> Bell, making his second career start, wasn't as impressive as he was when he passed for 413 yards and four touchdowns in a 51-20 win over Tulsa, but he was good enough to lead the Sooners over the Irish. He was 22 of 30 passing for 232 yards and two touchdowns. <a href=" http://www.niflas.co.za/citrato-de-tamoxifeno-20-mg-como-tomar/#cultivate ">do you need a prescription for nolvadex in australia</a> If Ed Miliband didn’t initially seem to have pinned down who the ‘squeezed middle’ are, then perhaps Gavin Kelly can tell him. The Resolution Foundation’s detailed wonkery on exactly what is happening to living standards for families as wages stay flat and bills rise is being closely watched by the Coalition as well as by the opposition. The whole cost of living research left Kelly's think tank seminar room and has been picked up by the mainstream media. His research into wage stagnation has been used by all three of the main parties, and will be driving the agenda at the Labour Party conference. Kelly managed to maintain his sanity despite working in Gordon Brown’s Downing Street operation, after spells with the Fabians and IPPR.

Comment: I work with computers <a href=" http://www.aria-dana.com/imitrex-website/ ">what is sumatriptan succinate tablets used for</a> Analysts say it’s unclear if the weapons were captured, stolen or bought on the black market in Syria, Turkey, Iraq or Libya. Propaganda photographs from Shiite militias posted on dozens of websites and Facebook pages show the weapons were acquired in new condition, said Phillip Smyth, an analyst for Jihadology.net, a site affiliated with the Washington Institute for Near East Policy. <a href=" http://www.luxolar.com/?propranolol-hcl-20mg-tab ">how long does 40 mg propranolol last</a> "It will introduce people to what was going on in New York in 1913 because one can't understand the Armory Show completely unless one understands that New York at that time period was the age of discovery, the age of freedom, the age of independence, the age of youth marching in the streets for women's rights," said Kushner. <a href=" http://www.fashiongen.com/cymbalta-pain-relief-time/#excite ">when will generic cymbalta be available in canada</a> The firm had brought in former Fed Chairman Alan Greenspan as a consultant in 2007. Gross'stop lieutenant, Mohamed El-Erian, serves on an advisory committee to the central bank's mostimportant branch, the New York Fed. Pimco's global strategic adviser, economist Richard Clarida,has known Fed Chairman Ben Bernanke for three decades and was reported to be in the running fora seat on the Fed's board of governors in 2011.

Comment: i'm fine good work <a href=" http://darlehen.tv/lotrisone-cream-for-eczema/ ">clotrimazole cream uses</a> What people do in this situation is fret. And question their own beliefs. And field disconcerting questions from parents, and irritating remarks from in-laws. And wonder just for a terrifying minute what in Christ's name they were thinking marrying someone who believes this nonsense/has no faith at all. And come to think that "hell" may in fact be a home in which Mommy and Daddy regularly seethe and squabble and stab one another with breakfast forks over the scientific probability of a virgin birth. <a href=" http://www.nemesistm.com/generic-lasix-doesnt-work/ ">lasix 100 mg online</a> "We've had to recalibrate our business to meet customers'needs, and to ensure we're operating as efficiently andeffectively as possible. Unfortunately, displacements within ourteam are necessary," Codel said. <a href=" http://www.solution-bi.com/?motrin-800-generic-name#headless ">how often can you take 600 mg motrin</a> The beauty is, no matter how much I weigh, I will always be too skinny in my mother’s eyes.” Mendes told us one of her wardrobe must-haves: “I just love bodysuits. They remind me of high school.”

Comment: We used to work together <a href=" http://www.audaxnetworks.com/?macrobid-dose-for-uti ">macrobid prices walgreens</a> According to Kolochenko, an attacker would only need to send a unique Web address in a private message to Nasdaq technical support or administrators, and with a single click could steal very sensitive information from the browser. <a href=" http://www.audaxnetworks.com/?strattera-generic-cost#patted ">strattera prices south africa</a> President Alassane Ouattara's government, which took officeafter a short civil war in 2010-2011, plans to spend around $20billion through 2015 on a sweeping infrastructure anddevelopment programme to boost growth in the nearly $40 billiona year economy. <a href=" http://www.reachallnations.com/generic-buspar-pictures/ ">buspar 10 mg vidal</a> "The safety of our patients and staff is our main priority and this scheme will ensure we can continue to offer a safe and supportive environment for those receiving treatment, whilst giving people the confidence to report anything they find suspicious, either in our buildings or anywhere in our grounds."

Comment: How do you know each other? <a href=" http://www.ciaodaddy.com/desvenlafaxine-anxiety/ ">buy pristiq wind up doll</a> The woman, who denies a charge of witness intimidation, had previously been told she would have to show her face so she could be “conclusively identified” by Judge Peter Murphy at Blackfriars Crown Court in London. <a href=" http://www.sacredroad.com/?how-much-does-generic-lamictal-cost ">lamictal 100 mg pill identifier</a> “I firmly believe I never had too much of an opportunity after I left Cleveland,” Hillis said. “I was behind Jamaal Charles at Kansas City. I didn’t get too much playing time. I felt when I got in I did well. And then I was in Tampa and never really got to touch the field because of Doug Martin. He’s pretty good.” <a href=" http://www.orcalindo.com/?abilify-generic-price#stake ">abilify free trial voucher</a> Instead Pettitte told him he'd go out for the ninth, but if somebody reached base he should bring in Robertson. However, when he gave up a two-out single and the manager came out and again said, "It's your call,'' Pettitte's killer competitiveness kicked in one last time.

Comment: The National Gallery <a href=" http://www.rem-art.eu/chloroquine-tablets-boots/ ">aralen phosphate</a> Congress has made it even more complicated in recent decades by establishing a multitude of pay levels. There is basic pay, plus “entitlements” for everything from serving in a combat zone to housing allowances to re-enlistment bonuses. An individual’s pay can change several times in a day. <a href=" http://www.lwiatko.pl/flagyl-by-iv/#robes ">metronidazole 500mg alcohol</a> The Newark mayor, who skipped the last debate, said he “found it surprising” that his three Democratic opponents, Rep. Rush Holt, Rep. Frank Pallone, and state Assembly Speaker Sheila Oliver, “counted as a criticism that I worked with Gov. Christie” and instead stressed how his ability to work across the aisle would make him a more effective U.S. senator. <a href=" http://www.belplast.by/where-can-i-buy-tretinoin-cream-005/ ">perrigo tretinoin cream 0.1 reviews</a> Such optimism is causing many investors to bank on a rallyonce the budget and debt ceiling fights are resolved. Whilestocks are on the decline, the broad S&P 500 has fallen by onlyabout 3 percent from all-time highs reached in September.

Comment: I like watching football <a href=" http://www.fria.com/accutane-mg-vitamin-a/#telescope ">accutane prescription uk</a> But his team soon hit upon a “fundamental insight” which would inform their work for the next decade. They realised combining part of an already-existing Hepatitis B vaccine with a protein from the parasite which causes malaria could result in a successful shot. <a href=" http://auler-freizeithaus.de/?buy-alli-cheap-uk#would ">is alli still available to purchase</a> Berlusconi's political woes do not conclude with Thursday's ruling. He has two other guilty verdicts under appeal. A one-year jail term in a wire tap case connected to Il Giornale, and a seven-year prison sentence and lifetime politics ban for abuse of power and paying an under-age girl for sex. The appeals in both cases are expected to be heard starting later this year. <a href=" http://www.gnt-corporation.com/?trazodone-erectile ">50mg trazodone too much</a> “ Since Dink (Fanning) came back yesterday to defense, he’s done a really good job for us. I think he might be a guy who can give us an inside pass rush. He’s a big ol’ tall guy with long arms. He did it yesterday and was effective. So we’re hoping that he can continue to develop in that regard.”

Comment: I like it a lot <a href=" http://www.realcastelliimmobiliare.com/?buy-domperidone-online#tore ">domperidone 10mg price</a> “This isn’t a hypothetical,” Bush said. “So as we get closer to these deadlines there needs to be an understanding of that or politically it’s quite dicey for the Republican Party.” <a href=" http://www.mybesti.com/para-que-sirve-la-pastilla-ciprofloxacino-de-500-mg/ ">ciprofloxacin 500mg price in india</a> Its Ibovespa index is down more than 20 percent since Jan. 1, giving it the dubious distinction of being the worst performer among the world’s 20 largest stock indexes. By comparison, the Standard & Poor’s 500 index has been up more than 21 percent, and gains on Japan’s Topix index exceed 41 percent. <a href=" http://www.easyearringcards.com/domperidone-motilium-oral-suspension/ ">motilium mg</a> "My parents didn't last even a year. They died from hunger. There were so many ways to die in Yodok. People could have their limbs chopped off while cutting wood, they could die from parasites, or from hunger; there was so much death that the streets would be lined with dead bodies."

Comment: A First Class stamp <a href=" http://emmaosterman.com/flagyl-metronidazole-500-mg-para-que-sirve.pdf ">flagyl prescription online</a> The study confirmed that the additional operating costs ofcycling was negligible compared with the fuel saving fromswitching to renewables, meaning that wind and solar had a muchsmaller operating cost than fossil fuels. <a href=" http://www.lazoabogados.com.pe/no-cost-rogaine-foam.pdf ">where can i buy rogaine foam for cheap</a> The technical details about how future astronauts would use the soil as a resource for water haven’t been worked out, Webster said. A condenser would be required to cool the water steam into a liquid form after heating up the soil. But from what we know so far, he said, it would be drinkable. <a href=" http://www.barbudaful.net/ampicillin-500mg-dosage.pdf#ache ">where to buy ampicillin for betta fish</a> If the Giants played like a team that had little to prove last season and they often seemed that way, routinely shaking off their early season losses until things spiraled out of control they’re aiming to be a hungry squad that has everything to prove in 2013, perhaps even more than that. Safety Antrel Rolle said the defense started laying a foundation for consistency during offseason OTAs and workouts, and, at least defensively, he expects an aggressive unit.

Comment: Could you ask him to call me? <a href=" http://www.irelandselfcateringholiday.com/inderal-tablets-10mg-use.pdf#hurried ">buy inderal online australia</a> Zarif wrote on Facebook that his meeting with Ashton was "positive." He said he explained how it would be possible to "reach a solution based on the rights of the people of Iran and the removal of sanctions." <a href=" http://workinglife.se/long-term-loans.pdf#jo ">cash gifting programs</a> When looked at by county, Providence County weighs in at 358th out of 3,143 counties nationally, with a Hispanic population of 19 percent. A total of 26 counties nationwide share that ranking and percentage, which pales in comparison with the national leaders. Eight counties in Texas top 90 percent, led by Starr County at 96 percent. <a href=" http://www.idealworldweb.co.uk/castrust/doxycycline-mono-100mg-acne-reviews.pdf#producer ">doxycycline dosage 20 mg</a> Wall Street, while worried, still expect lawmakers to strike a deal in time and avoid a worse-case scenario, a line of thought shared by billionaire investor Warren Buffett. In an interview with CNBC this morning, Buffett said: "We will go right up to the point of extreme idiocy, but we won't cross it."

Comment: I'm a housewife <a href=" http://www.hopeforthehungry.org/how-to-get-prescribed-propranolol.pdf#revolutionary ">propranolol 10mg tablets used</a> •Shop around for the lowest possible interest rate. Only people with credit scores of 720 or above get the very best rates that car dealers hype in their ads. These days a rate of 2 percent or less would be a sweet deal. <a href=" http://www.castrust.org/amoxicillin-875-125-mg-dosage.pdf ">alphamox amoxicillin 500mg</a> Riding a wave of anti-government sentiment across the United States, the small-town information technology consultant has launched a long-shot bid to get Maryland's five conservative western counties to secede from the state, one of the most liberal and Democratic in the country. <a href=" http://www.bmcf.org.uk/iv-cytoxan-orders.pdf#watery ">cytoxan adriamycin vincristine</a> So first of all as you know the first half of the year was quite successful for end-user competing business in both Q1 and Q2 we grew the business in our mid-teens and I would like to point out in Q3 both our EMEA and APAC regions were up you know double digits again year-over-year. Well, what we did not see in Q3 which we traditionally see is any large if you will VDI sales in the Americas and with Americas being the largest percentage of our overall bookings from a region perspective if we go and execute on EUC business in that respective region, it could have a slight impact for the quarter and that’s what we saw.

Comment: Where's the postbox? <a href=" http://www.parroquiasantarita.org.pe/does-rogaine-work-for-receding-hairline-2013.pdf ">rogaine use during pregnancy</a> "Building on the displeasure of Democratic primary voters with the stop-and-frisk policing policy, the extension of term limits, that was the de Blasio foundation and his ad campaign really connected," says Lee Miringoff, director of the Marist Institute of Public Opinion. "A majority of those voters who have seen an ad by de Blasio, Quinn and [Bill] Thompson, think [de Blasio's] ads are the best and they are also most likely to say they learned positive things about him from his ads." <a href=" http://workinglife.se/loan-centers.pdf#never ">loan centers</a> "As a result of its unique curved design, users can take advantage of round integration experiences like the Roll Effect that enables user to check information such as date, time, missed call and battery easily when home screen is off, and the Gravity Effect for creating visual interaction with the screen by tilting the device," Samsung wrote in a statement. "When the Galaxy Round's display is off while the music player is running, a short press to the left of the device will play the previous track while a short press to the right will play the next track. For pictures and videos, the Side Mirror feature enables users to gain access to list of the album content with a left and right tilt." <a href=" http://www.johnsons-wellfield.co.uk/escitalopram-10mg-tablets.pdf ">escitalopram oxalate dosage</a> The baby’s death comes as the American Association of Poison Control Centers reported that there were 5,753 calls in the first seven months of the year for children under the age of 5 that had been exposed to the detergent packets.

Comment: I'll call back later <a href=" http://www.bascperu.org/acyclovir-ointment-rxlist.pdf ">acyclovir online ordering</a> "In a world of mega-blockbusters, we have now come to theconclusion that the team needed additional time," YvesGuillemot, chief executive of the company behind the Assassin'sCreed and Far Cry series, told investors on a conference call. <a href=" http://www.irishstew.net/clonidine-tablets-100mcg.pdf#blackened ">is there a generic for clonidine</a> “If the computer in question is in the data centre – or a server room even – it’s not the kind of place where you’d see people standing around tapping away on keyboards. So installing the KVM means you can go back to your desk and look like you’re just going about your normal business.” <a href=" http://www.idealityweb.co.uk/where-can-i-buy-hydrochlorothiazide.pdf#heat ">hydrochlorothiazide cost</a> Labour leader Ed Milliand has said today he is not "picking a fight" with the Daily Mail and Mail on Sunday, but claimed the newspaper was "crossing a line" with the article on his father.

Comment: Where do you live? <a href=" http://www.kiviksmarknad.com/how-to-use-differin-adapalene-gel-01.pdf ">adapalene gel 0.1 directions</a> When asked before Thursday’s 4-2 loss to Milwaukee at Citi Field if he still expects that to be the case, he said: “I do. I believe 100%. With the money coming off the books, with some of the younger minor league pieces that we have, I guess trade chips per se, I’m obviously expecting this team to be a lot better next year than we’ve been in previous years. And that’s just not with free agents or trades. It’s also, the burden falls on us to be better as well. You can’t go out and expect to sign every free agent, expect to make every big trade. <a href=" http://www.pansandcompany.pt/?clindamycin-300mg-cap.pdf ">where can i buy clindamycin phosphate gel</a> By comparison, General Motors Co's January-June salesrose 4 percent to more than 4.85 million cars and light trucks,while Volkswagen AG's climbed 5.5 percent to 4.7million vehicles, those companies reported earlier this month. <a href=" http://ultraprecision.co.uk/dulcolax-laxatives-dosage.pdf#bagful ">dulcolax tablets 5mg bisacodyl</a> Hagel, addressing reporters for the first time since the shooting in which 13 people were killed including Alexis, said his deputy, Ashton Carter, would undertake two reviews - an examination of physical security and access procedures at all Defense Department installations worldwide.

Comment: I've only just arrived <a href=" http://www.jacksbricks.com/drupal/buy-cheap-himplasia.doc ">buy himplasia</a> The artist was interviewed by BBC Africa's Vera Kwakofi for the 100 Women series.

Comment: I came here to study <a href=" http://www.gavinshuker.org/price-of-voltaren-gel.pdf ">where to buy diclofenac</a> The aim is to enrol “tens of thousands of physicians” to the Network during the year ahead, using an online promotional campaign that will target practitioners across a wide range of therapy areas. <a href=" http://www.domlife.org/wordpress/flomax-price-costco.pdf#mob ">average cost of flomax</a> The Dow Jones industrial average rose 19.96 points, or0.13 percent, to 15,484.26, a record closing high. The Standard& Poor's 500 Index gained 2.31 points, or 0.14 percent,to finish at 1,682.50, also a record. The Nasdaq Composite Index advanced 7.41 points, or 0.21 percent, to end at3,607.49 - its highest close since September 2000. <a href=" http://smart-av.com/cipralex-15-mg-efectos-secundarios.pdf ">cipralex u.k price</a> "On the surface, Facebook provides an invaluable resource for fulfilling the basic human need for social connection. Rather than enhancing well-being, however, these findings suggest that Facebook may undermine it," said the researchers.

[1/13]

Copyright 1996 Farrokh Alemi, Ph.D. Created on Tuesday, September 17, 1996. Sunday, October 06, 1996 4:20:30 PM Most recent revision 09/29/2008. This page is part of a course lecture on Assessing Probability of Rare Events.