|
![]() |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Probabilistic Root Cause AnalysisBy Farrokh Alemi, Ph.D. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
SummaryRoot Cause and Failure Mode Analyses are commonly performed in hospitals to understand factors that contribute to errors and mistakes. Despite the effort that healthcare professionals are putting into creating these analyses, few models of root causes are validated or used to predict future occurrences of adverse events. We review the literature on Causal Networks and Bayesian Probability models and show how these tools can be used to improve Root Cause Analysis. In particular, we show that more in-depth insight can be gained by (1) testing the proposed root causes for structural assumptions of independence (root causes should be conditionally independent of sentinel events given direct causes) and by verifying that the root cause model implies probabilities that are comparable to those reported in the literature or known through experience of the health care organization. We show how both assumptions and conclusions of Root Cause Analysis can be verified against observed data. IntroductionRoot Cause Analysis, according to the Joint Commission on Accreditation of Health Care Organizations is a "process for identifying the basic or causal factors that underlie variation in performance, including the occurrence or possible occurrence of a sentinel event." Sentinel events include medication errors, patients' suicide, procedure complications, wrong site surgery, treatment delay, restraint death, elopement death, assault or rape, transfusion death, and infant abduction. Direct causes bring about the sentinel event without any other intervening event. Most direct causes are physically proximate to the sentinel event. The effect of root causes on sentinel events are always through some direct cause. Because of accreditation requirements and due to renewed interest in patient safety, many hospitals and clinics are actively conducting Root Cause Analyses. When a sentinel event occurs, most employees are focused on the direct causes that have led to the event. For example, many will claim that the cause of medication error is a failure to check label against the patient's armband. But this is just the direct cause. To get to the real reasons, one should ask why did the clinician not check the label against the armband. The purpose of Root Cause analysis is to go beyond direct and somewhat apparent causes and figure out the underlying reasons for the event. The objective is to force one to think harder about the source of the problem. It is possible that the label was not checked against the armband because the label was missing. Furthermore it is also possible that the label was missing because the computer was not printing. Then, the root cause is computer malfunction and the direct cause is the failure to check the label against the armband. Exhorting employees to check the armband against the label is a waste of time, if there is no label to check in the first place. A focus on direct causes may prevent the sentinel event for a while, but sooner or later the root cause will lead to a sentinel event. Inattention to root causes promotes palliative solutions that do not work in the long run. The value of root cause analysis lies in identifying the true, underlying causes. An investigation that dos not do this is at best a waste of time and resources, and at worst can exacerbate the problems it was intended to fix. But how do we know if our speculation about the causes of an event are correct? To make the situation worse, almost all who conduct Root Cause analyses become overconfident about the accuracy of their own insights. No matter how poorly an analysis is carried out, since there is no way of proving a person wrong, people persist in their own fallacies. Some are even incredulous about the possibility that their imagined causal influences could be wrong. They insist on the correctness of their insights because "it is obvious." Unfortunately, it is not clear why a complex problem, which has led to a sentinel event, which has not been corrected for years, which has been left unaddressed by hundreds of smart people should have such an obvious solution. After all, if the solution was so obvious why was it not adopted earlier? Search for obvious solutions contradicts the elusiveness of correcting for sentinel events. If a sound and reliable method existed for checking the accuracy and consistency of Root Cause Analysis, then employees might correct their misperceptions and not be so overconfident. One way to check on accuracy of Root Cause Analysis is to examine time to next sentinel event. Unfortunately, because sentinel events are rare, one has to wait a long time to see rare events occur again, even if no changes were made. Thus, the organization may have little solace by marking time as long periods of time are no sign of success and the event may reoccur any day. An alternative needs to be found to check the accuracy and consistency of Root Cause Analysis without having to wait for the next sentinel incidence. Simple methods for checking the accuracy of a Root Cause analysis have not been available to date. This paper suggests a method for doing so. As before, clinicians propose a set of causes. But now several additional steps are taken. First, probabilities are used to quantify the relationship between causes and effect. Then, the laws of probability and causal diagrams are examined to see if the suggested causes are consistent with the clinician's other beliefs and with existing objective data. Through a cycle of testing model assumptions and conclusions against observed data, one improves the accuracy of the analysis and gains new insights into the causes of the sentinel event. Bayesian NetworksWe use a set of techniques that have been developed for analysis of Bayesian Causal Networks to validate Root Cause Analysis. A Bayesian Causal Network is a mathematical model of the cause and effect relationships and the way these relationships lead to observed statistical patterns of an event. It consists of a set of nodes, typically pictured as ovals, connected by directed arcs. Each node represents a mutually exclusive and collectively exhaustive set of possible events. For example, Figure 1 shows a Bayesian network with two nodes. The node "Armband legibility?" has three possible values, exactly one of which must occur and no two can coincide. These states are "No armband," "Poor" and "Good." The other node "Armband checked?" has two possible values, "Ok" and "Not ok." A node with two possible values is called a binary node. Binary nodes are common in root cause analysis.
Figure 1: A simple Bayesian Causal Model with a Local Probability Table for "Armband checked?" A Bayesian Network is cyclical directed graph, meaning that you cannot start from a node and follow the arcs and arrive back to where you started. In a Bayesian Network, the relationship between any three nodes can be expressed in one of the following three ways: serial, diverging or converging structures. Each of these graph structures can be verified through tests of conditional independence and are further explained through examples below. The relationship between “Armband legible?” and “Armband Checked?” in Figure 1 is a direct causal relationship. Bayesian networks can also represent indirect causal relationships, through the concept of conditional independence, as shown in Figure 2. In this example, the root cause “Understaffing” is an indirect cause of the sentinel event. There is no direct arc from the root cause to the sentinel event. This means that the action of the root cause on the sentinel event is indirect, operating through an intermediate cause. That is, the direct cause of a medication error is a fatigued nurse. The root cause, “Understaffing,” is conditionally independent of the sentinel event given the intermediate cause. This means that if we intervene in any given instance to relieve a fatigued nurse, we can break the link from the root cause to the sentinel event, thus reducing the probability of the sentinel event to its nominal level. However, this solution is a palliative one and will not produce a long-term solution unless the root cause is addressed. Figure 2 illustrates a serial graph structure. In these structures the sentinel event is independent of the root cause given the known value for direct cause .
Figure 2: Serial Example of Direct and Root Cause of Medication Error Another type of conditional independence occurs when a cause gives rise independently to two different effects, as depicted in Figure 3. This type of graph structure is known as diverging. In this example, “High blood pressure” and “Diabetes” are conditionally independent given the value of “Weight gain,” but are correlated due to the influence of the common cause. That is, the two effects typically either co-occur (when the common cause is present) or are both absent (when the common cause is absent). This type of conditional independence relationship is quite useful for diagnosing the presence of root causes that can lead to multiple independent effects that each influence different sentinel events. For example, understaffing might lead to several different intermediate causes, each of which could be a precursor of different sentinel events. If several of these precursor events were to be observed, one could infer that the understaffing problem was sufficiently severe to impact patient care. Proactive remediation could then be initiated prior to the occurrence of serious adverse medical outcomes.
Figures 2 and 3 illustrate serial and diverging causal structures, respectively. As we have seen, a serial structure represents the action of an indirect causal relationship, and a diverging structure represents multiple independent effects of a single cause. In both these cases, the two terminal nodes are conditionally independent of each other given the middle node. A different kind of causal structure, the converging structure, is shown in Figure 4. A converging structure occurs when two different causes can produce a single effect, as when either a fatigued nurse or a missing armband can cause a medication error. Notice that in this case, the terminal nodes are not conditionally independent given the middle node. For example, if the sentinel event is known to occur, and we learn that the armband was present, this will increase our probability that the nurse was unacceptably fatigued. Likewise, if we find that the armband was missing, this will reduce the likelihood that the problem was due to fatigue.
Data can be used, if available, to validate the graph structure of a causal Bayesian network. As we noted above, when a connection is serial or diverging, the terminal nodes are conditionally independent given the intermediate node. This relationship can be evaluated by holding the value of the intermediate node fixed and examining whether the terminal nodes are correlated. In the example of Figure 3, we would compute the correlation of “High blood pressure” and “Diabetes” for all cases in which there was weight gain. We would then calculate the correlation for all cases in which there was no weight gain. If both correlations were statistically indistinguishable from zero, we could conclude that the conditional independence relationships were satisfied. Similar tests can be used to evaluate conditional independence relationships for a diverging structure. In general, in a Bayesian network a node is conditionally independent of all its non-descendents given its parents. This general condition implies a set of correlations that should be equal to zero if the structure is correct. Although it is tedious to verify all these relationships by hand, it is straightforward to automate the verification process, and computer programs have been written to accomplish the task. Given the structure of an acyclic causal graph, one can read off the assumed conditional independencies. Conditional independencies are identified through examining serial or diverging graphs in causal model so that removal of the condition would sever the directional flow from the cause to the effect. Often, a complicated Root Cause Analysis can be broken into smaller components containing serial and diverging structures. If these structures are observed and if removal of the condition in these structure would sever the link between the other two nodes, then a conditional dependencies has been identified. Careful examination of conditional independence relationships is an important element of specifying and validating a Bayesian Network for root cause analysis. Validation of Conditional IndependenceOnce conditional independencies have been identified, the assumptions can be verified by examining data or by querying experts. If data is available, in a serial structure, the correlations between root cause and sentinel event should equal to the correlation between root cause and direct cause times the correlation between direct cause and sentinel event. R root cause, sentinel event = R root cause, direct cause * R direct cause, sentinel event
In a diverging structure, a similar relationship should hold. In particular, correlation between the two effects should be equal to the multiplication of the correlation between the cause and each effect: R effect1, effect2 = R cause, effect1 * R cause, effect2
If data are not available, the analyst can ask the investigative team to verify assumptions of conditional independence based on their intuitions. For example in the serial structure in Figure 2, if we know that the nurse was fatigued, would information about staffing add much to our estimate of the probability of medication error. Another way to ask the same questions is to ask if understaffing affects medication errors only through creating a fatigued nurse. In this method, the exclusivity of the mechanism of change is checked. Still another way of verifying conditional independence is through asking for estimates of various probabilities. One might ask:
If conditional independence is violated, then the serial or diverging structures in the graph are incorrect. If these conditions are met, then the causal graph is correct. Lets look at slightly more complicated set of causes. Figure 4 shows four proposed causes for medication error: understaffing, fatigued nurse, vague communication, and similar medication bottles. Two root causes (understaffing and vague communications) are shown to precede the direct cause "fatigued nurse." Removing the node "fatigued nurse" would stop the flow from these two root causes to the medication error. Therefore, a conditional independence is assumed. This assumption can be verified either through data or through experts judgments. Let us assume that if we know that the nurse is fatigued, understaffing adds no additional information to probability of medication error. So this independence is verified. But suppose that even when the nurse is not fatigued, vague communications may lead to medication errors. Therefore, the assumption of conditional independence of vague communication and medication error is not met.
Figure 4: Four possible causes of medication error & their relations Therefore, the causal network needs to be modified. Further exploration may indicate that vague communications, similar bottles, and fatigued nurse directly affect medication errors. This example shows how verifying conditional independence could help us revise root cause analysis. Predictions from Root CausesThe causal model behind the root cause analysis can be used to predict the probability of the sentinel event and this probability can then be compared to the intuitions of the investigative team. The probability of sentinel event can be calculated from each of the direct causes and the probability of direct causes can be calculated from their root causes. p (Sentinel event, Various causes) = To calculate, the probability of sentinel event, S, given a set of different unobserved (CU) and observed causes (Ci), we can use the following formula:
The above formula requires one to carefully track numerous probabilities. Because these calculations are tedious, investigative teams can use widely available software, e.g. Netica (TM), to simplify the calculations. Download► An example can demonstrate how such calculations are made using the software . Suppose that Figure 5 shows root causes for wrong side surgery in a hospital. First, note that the root causes listed are poor MD training, and over staffing as it contributes to fatigued nurse. These are the root causes because they are independent of sentinel event given the various direct causes. The direct causes listed are nurse marking the patient wrong, surgeon not following the markings and patient providing wrong information. These are direct causes because an arc connects them to the sentinel event.
Figure 5: Root causes for wrong side surgery Given the Root Cause Analysis in Figure 5, the next step is to estimate the probability of the various cause and effects. These probabilities are obtained by asking the expert to assess the conditional probabilities implied in the graph. Each node is conditioned on its direct causes. For example, to estimate the probability of having a fatigued nurse, the facilitator needs to ask the investigative team the following two questions:
Obviously, estimates of probabilities from experts is subjective and therefore may be unreliable. But if experts are provided with tools (calculators, paper, pencils), brief training in concept of conditional probabilities, available objective data (e.g. JCAHO's reports on prevalence of various causes), and if experts are allowed to discuss their different estimates, then experts' estimates are sufficiently accurate and reliable to provide a useful model. These probabilities may not be accurate to the last digit, but can provide for a test of consistency. Suppose that through interviewing experts or through analysis of data of the hospital, the investigative team has estimated the following probabilities:
Using these estimates we can calculate the probability of wrong side surgeries when no information about any cause are present as 0.07 (see Figure 6 for an example analysis using Netica (TM) software available through Norsys). Does this seem reasonable to the investigative team? If the probability is higher by an order of magnitude from what the investigative team expected, then perhaps important constraints that prevent wrong side surgeries have been left out of the analysis. If it is too low by an order of magnitude, then an important cause or mechanism by which wrong side surgeries occur might have been missed. If it is in the ball park but not exactly what we expected, then perhaps the estimated probabilities might be wrong. In any case, when there is no correspondence between the probability of the sentinel event and the investigative team's intuition, it is time to re-think the analysis and its parameters.
Figure 6: Application of Netica Software to Root Cause Analysis in Figure 5 Other probabilities can also be calculated and compared to the experts' intuitions. Suppose on a particular unit in a particular day, we find the nurse was fatigued but the clinician was well trained and the patient provided accurate information. Given the above estimates and the root cause in Figure 5, the probability of wrong side surgery on this day is calculated as: 0.03 (See Figure 7). If this corresponds to the investigative team's expectation, then the analysis is consistent and we can proceed. If not, we need to examine why not and look for adjustments that would fit the model predictions to experienced historical rates.
Figure 7: Probability of wrong side surgery when the
patient has provided Reverse PredictionsThe Bayesian network can also be used to calculate the probability of observing a cause given an effect has occurred. This is the reverse of how most people think about causes and effects. Most people start with a cause and want to predict the probability of the effect. Bayesian probability models allow us to do the reverse. One can start with known sentinel events and ask about the prevalence of a particular cause among them. Since causes are not as rare as sentinel events, this procedure allows us to check on the adequacy of the analysis without having to wait a long time for reoccurrence of the sentinel event. To make matters easier, the Joint Commission on Accreditation of Healthcare Organizations publishes prevalence of categories of causes among sentential events. These data can be used to examine the consistency of the Root Cause Analysis. Large discrepancy between observed prevalence of causes among sentinel events and assumed prevalence of causes in the investigative team's model suggest errors in assignments of probabilities as well as possible missed cause or constraint.
Figure 8: JCAHO's Report on Various Categories of Causes for Sentinel Events There are a number of software systems that allow the calculation of reverse probabilities in a Root Cause Analysis. Using Netica software we calculated the prevalence of over staffing in our model of wrong side surgeries. We started by setting the probability of wrong side surgery to 100%. Then we allowed the software to calculate the prevalence of over staffing.
The software calculated that overstaffing is present in 44% of wrong side surgeries. Is this a reasonable estimate? In contrast, JCAHO reports staffing levels to be a cause of sentinel event less than 20% of the time (see Figure 8). Obviously, there are many reasons for a health care organization to differ from other aggregate data reported by JCAHO. But JCAHO's data can be used as a rough benchmark. The two probabilities differ considerably. These differences suggest the need to think again through the analysis. Summary of Proposed Method for Root Cause analysisSentinel events can be reduced if health care organization create a blame-free environment, conduct Root Cause Analysis and take concrete actions on the basis of the analysis. To conduct root cause analysis we propose the following steps:
DiscussionInvestigative teams often rely on their own intuitions for listing the root causes of a sentinel event. They rarely check the validity of their analysis. In this paper, we have shown how Bayesian networks can be applied to root cause analysis to test the validity and/or consistency of the analysis. Real analysis should be a careful examination of facts and not a cover for wishful speculation. By creating a Bayesian Network and estimating the probabilities of various events, one can scrutinize assumptions made in root cause analysis. In particular, one can check to see if important root causes have been missed, if the analysis is focused on root causes or direct causes, if frequency of sentinel event corresponds to expectations and experienced rates, if prevalence of causes of sentinel events correspond to known rates, and if assumptions of dependence or independence are wrong. These are not exact ways of checking the accuracy of the analysis. But these methods allow us to check the intuition of investigative teams and help them think through the implication of their analysis. Examples of Root Cause Analysis
What Do You Know?
1. When A causes B, B causes C, and there are no other causal relationships, what implication do
these relationships have for the conditional probabilities? Bi-weekly ProjectAdvanced learners like you, often need different ways of understanding a topic. Reading is just one way of understanding. Another way is through writing. When you write you not only recall what you have written but also may need to make inferences about what you have read.
Submit narrated slieds to your instructor. See examples of previous projects. Note that these are only examples and should not be used as templates for your project. There may be errors in these presentations! You must do everything yourself and examples are ONLY to show you how a project may look like. Rape on campus► Medication error► Prescription errors► Analysis of falls (in Arabic)► Slides and Narrated SlidesThese audio/visual resources may be of further assistance to you in understanding this topic:
Narrated slides and videos require Flash. Download► More
Recently Asked QuestionsAsk questions and we will try to answer it on this page. Following are answers to recent questions asked: Question: How do you really classify a cause as a direct cause as opposed to an indirect cause? Can there be instances where an indirect cause can be an direct cause as well depending how one interprets the analysis? Answer: Here lies the art of the analysis. A direct cause shows a mechanism that affects the target event in close physical proximity and with few steps, cigarettes are a direct cause of lung cancer because we see how smoke affects the lung. An indirect cause is something that does not have a physical link to the target event. Advertisement for cigarettes directed at children may be considered an indirect cause, as it affects the uptake of smoking but it does not affect the lung itself directly. Asked on 7/11/2008 12:40:59 AM and answered on 7/11/2008 9:38:08 AM. Question: When I try and use the Netica software, I set up the nodes correctly, however, when I try and enter the probabilities, it does not give me true and false, it just gives me and area to enter the number of trues, how do I get it set up so that it has the true and false columns? Answer: Are you sure that you have selected nodes that are event nodes with uncertain outcomes. You have probably selected a different node. Asked on 4/16/2008 11:05:48 PM and answered on 4/17/2008 12:58:31 PM. Question: What is your advice in distinguishing the root causes from direct causes? Is there a situation where the root and direct cause are the same variable? Answer: This is a really difficult question to answer. We have provided mathematical tests of root versus direct cause through conditional independence; but I am not sure that this is the whole picture. There is a lot more that meets the eye. I am not sure the field knows how to do this, except the obvious answer of repeatedly asking why. Asked on 4/14/2008 8:48:26 PM and answered on 4/15/2008 11:56:09 PM. Question: In the online survey, are we supposed to see the results of the information we provide? All I see is this as a response: Database Results Wizard Error The operation failed. If this continues, please contact your server administrator. Answer: I am not sure where this error is occurring. I was not able to find an online survey on this page. Please give me a few more clues about the location of the online survey. Asked on 4/13/2008 11:44:15 PM and answered on 4/14/2008 7:05:05 AM. Question: You mentioned Netica Software for Root Cause Analysis. My questions are: 1. what is the most popular software for root cause analysis? 2. How much does it cost? 3. Is it used a lot in the healthcare industry. Thanks. Answer: See earlier answer Asked on 7/12/2006 12:17:28 PM and answered on 7/18/2006 4:35:57 AM. Question: You mentioned Netica Software for Root Cause Anlaysis. My questions are: 1. what is the popular software root cause anlysis? 2.How much does it cost? 3.Is it used a lot in the healthcare industry. Thanks. Answer: To date, there is no specific software that has been developed for root cause analysis that has the functionality we need (e.g. check of conditional independence). There is a book on failure mode analysis in health care that is sold with an accompanying software which is an add on to Excel. Asked on 7/12/2006 12:14:54 PM and answered on 7/18/2006 4:35:37 AM. Question: Dr Alemi, what would be a simple way to identify the direct cause from the root cause? Answer: A direct cause often has a physical link to the event of interest. An indirect cause does not. Direct causes can be established by asking questions on what happened, when it happened and where it happened. An indirect cause is more likely to be a more systemic and removed from immediate events surrounding the adverse event. Asked on 4/3/2006 12:47:21 AM and answered on 4/7/2006 1:34:54 PM. Question: Do we really have to complete this survey as part of the assignments? Answer: I am not sure what you are asking? Is this in relation to root cause analysis? Asked on 10/15/2005 3:42:51 PM and answered on 10/17/2005 8:57:03 PM. Question: The link to the JCAHO materials does not seem to be working. Answer: Thanks I will look into it. Asked on 7/19/2005 6:53:57 AM and answered on 7/19/2005 7:42:25 PM. Question: Do we have to recognize the relationship between the root cause, direct cause, and incidence in the assignment? Answer: Yes, it is important to do so. You can do so by checking the independence of ideas expressed in the causal flow chart Asked on 4/17/2005 5:37:48 PM and answered on 4/21/2005 7:53:30 PM. Question: Is there a limit to how many root causes there can be? Answer: No and yes. There is no theoretical limit but experience with people shows that most people list at most 3-5 causes for a phenomena. Asked on 4/7/2005 9:36:50 PM and answered on 4/8/2005 12:17:37 PM. Question: Is there a stronger relationship between the root cause and the direct cause than the direct cause and the incident? Answer: I have hypothesized that direct causes will have a stronger relationship with the target event than indirect (root) causes. We are actually collecting the data to see if that is the case. Asked on 4/2/2005 11:01:53 PM and answered on 4/4/2005 10:32:39 PM. Question: Just have to use this method more in order to be clear on the relationships of marginal and conditional probabilities on sentinel events. Answer: Yes, practice makes perfect Asked on 4/17/2004 10:07:37 PM and answered on 5/20/2004 7:37:09 PM. Question: Will see after doing assignment Answer: Yes, of course. Asked on 3/30/2004 11:26:04 PM and answered on 3/31/2004 11:01:09 PM. Question: Just need to do hands on. The lecture and Jee's demonstration using the computer software was excellent. Answer: Yes of course Asked on 3/30/2004 9:06:44 PM and answered on 8/27/2004 12:51:18 PM. Question: Just need to do hands on. The lecture and Jee's demonstration using the computer software was excellent. Answer: Yes you do need to do it yourself Asked on 3/30/2004 9:06:22 PM and answered on 8/27/2004 12:51:10 PM. Question: How to validate the the probabilistic root cause analysis? Answer: You can valdiate the probabilistic root cause analysis by comparing its predictions or its assumptions to observed data. For example, you can see the prevalence of causes among sentinel events and compare this to JCAHO's reports. Asked on 3/30/2004 9:04:41 PM and answered on 3/30/2004 10:54:58 PM. Question: For now it seems that fair, I will see how well I can do the assignment. Answer: Yes, I agree Asked on 3/30/2004 9:04:32 PM and answered on 3/30/2004 10:52:58 PM. Question: Truth, I get it when I am in class however, I don't get it later at home. The lectures have been interesting and understandable until I get home. Answer: Perhaps we need to do more examples in class. We will do additional examples next time. Asked on 3/30/2004 9:04:32 PM and answered on 3/30/2004 10:53:33 PM. Question: I don't know until I actually attempt to apply this in the homework assignment Answer: Yes, I agree Asked on 3/30/2004 9:03:51 PM and answered on 3/30/2004 10:52:07 PM. Suggested Changes
You can
suggests changes or review below suggestions made by others: Suggestion: I'm about to run out of credit <a href=" http://www.purevolume.com/buymalegradxta ">Order Malegra Dxt Online</a> unnecessarily large files. If needed, additional RXportfolio storage space is available, but may incur a Comment made on 5/19/2013 1:35:27 PM. Suggestion: An accountancy practice <a href=" http://buyplendileb.metroblog.com/ ">buy plendil in us without prescription</a> If a prior approval/authorization number different from the one entered in field 12 applies Comment made on 5/18/2013 7:22:35 PM. Suggestion: A company car <a href=" http://cheapplavixcanadapharoko.metroblog.com/ ">plavix generic availability</a> the learning activity worksheet to record what you have learned during your Comment made on 5/18/2013 7:22:31 PM. Suggestion: Could you send me an application form? <a href=" http://w11.zetaboards.com/aubinyqeceo/ ">cordarone 200 mg tablet</a> in duration, determined by the number of claims selected for review, and the accessibility of the supporting documentation Comment made on 5/18/2013 6:59:32 PM. Suggestion: Another year <a href=" http://www.purevolume.com/buyharidraumu ">Order Haridra Online</a> assignments. The following set of responsibilities has been adopted by the Faculty Council: To inform students at the beginning of each course and at other appropriate times that the Honor Code, which prohibits Comment made on 5/15/2013 7:14:50 AM. Suggestion: I've got a very weak signal <a href=" http://www.purevolume.com/levlentabletsiel ">Levlen Tablets</a> call New York State Department of Health at 1-518-486-3209. Comment made on 5/14/2013 8:58:37 AM. Suggestion: Three years <a href=" http://accutanecostwithinsuranceumoc.blogs.sapo.pt/ ">Buy Accutane Online Canada</a> and epidemiologic studies) scientific knowledge Comment made on 5/11/2013 3:30:48 PM. Suggestion: Will I have to work on Saturdays? <a href=" http://www.sergiomoran.net23.net/user/view.php?id=760 ">porno young 16 year</a> She's damn moving it... Comment made on 5/7/2013 5:10:05 AM. Suggestion: Sorry, I'm busy at the moment <a href=" http://acugatike.ozzfans.net/ ">lola non nude photo</a> i should get to fuck him instead of this ugly bitch Comment made on 5/7/2013 12:58:43 AM. Suggestion: A law firm <a href=" http://naniakuo.ozzfans.net/ ">preteen tiny lolitas pics</a> nice tight teen, nice small tits Comment made on 5/5/2013 2:17:52 PM. Suggestion: What do you like doing in your spare time? <a href=" http://mizukoi.heteml.jp/iten/user/view.php?id=9151 ">loli pedo cp</a> FUCKING LOVE IT!!!!!!! Comment made on 5/1/2013 9:25:07 AM. Suggestion: I've been made redundant <a href=" http://www.compromisoempresarial.net/cursos/user/view.php?id=4282 ">sexy top list bbs</a> Damn, she's definitely DTF Comment made on 4/26/2013 3:17:39 PM. Suggestion: What do you do for a living? <a href=" http://xxx.everywebspace.com/xhamstermovies/ ">xhamstermovies </a> wow that was nice! <a href=" http://xxx.everywebspace.com/bustybeauties/ ">bustybeauties </a> Now i want a sex girl slave, too!!! Comment made on 4/26/2013 2:11:55 PM. Suggestion: Thanks funny site <a href=" http://courses.homeopaths.in/user/view.php?id=6866 ">virgin funlumpkins no titts</a> Naja, typischer Brazil Bi Porno. Hab besseres gesehen und besseres erlebt ;) Comment made on 4/26/2013 12:26:12 PM. Suggestion: Can you hear me OK? <a href=" http://www1.dva.jp/user/view.php?id=2724 ">teen virgin gallerie</a> i only fuck white pussy Comment made on 4/26/2013 2:27:20 AM. Suggestion: I stay at home and look after the children <a href=" http://www.krupornsawan.com/user/view.php?id=2902 ">russian bikini bride</a> miley cyrus ! Comment made on 4/25/2013 8:04:28 AM. Suggestion: Do you have any exams coming up? <a href=" http://php.educanet2.ch/epsonvi/mottaz/user/view.php?id=5199 ">pre teens ls</a> your guys r sick. Comment made on 4/19/2013 10:28:42 AM. Suggestion: I've come to collect a parcel <a href=" http://www.zoji.com/1230750 ">ukrainian ls lolita pics</a> she had one white ass lmao Comment made on 4/18/2013 3:24:02 PM. Suggestion: Will I have to work on Saturdays? <a href=" http://www.pinellasanimalfoundation.org ">prozac alternative</a> MaineCare program and have a valid NDC in the State's drug file. Comment made on 4/16/2013 11:52:39 AM. Suggestion: I love the theatre <a href=" http://www.yhfchat.com/geyfobicusic ">bbs kds lolita top</a> i think that girl is from Colombia Comment made on 4/16/2013 11:29:44 AM. Copyright © 2004 Farrokh Alemi, Ph.D. Created on 3/24/2004. Most recent revision 09/29/2008. This page is part of the course on Decision Analysis, the lecture on Root Cause Analysis & Causal Networks. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||