Probability Charts

Objectives

87 people have rated this page an average of 4.15 out of 5 stars.  See their  suggestions and  questions.  Rate this page.  P-chart lecture Risk adjusted P-chart lecture Calculating expected deviations for risk adjusted p-charts (SWF file) Creating risk adjusted P-chart  Video► SWF► Excel 2003► Objectives Why chart? Which chart is right? Assumptions P-chart Risk adjusted P-chart Presentations Example Analyze  Data More Recently asked questions Suggested changes See table of Student t-values Recently asked:  how to measure by severity index.do every disease has different value  See answer to this and other questions.

In this section, we show you how to analyze the data from  improvement efforts involving frequency of events.  We assume that you have  collected data about a key indicator over several weeks and that you need to  analyze the data.  We will teach you how to analyze the data and we will  even provide you with templates that you can use to quickly complete your tasks.  

The objectives of this session are:

• Chart data using P-chart.

• Chart data using risk-adjusted P-charts.

• Decide if the changes you have introduced have led to real  improvements.

Why chart?

There are two reasons why to construct a  control chart.

1. To discipline intuitions. Data on human judgment show  that we, meaning all of us including you, have a tendency to attribute  system improvements to our own effort and skill and system failure to chance  events. In essence, we tend to fool ourselves. Control charts help see  through these gimmicks. It helps us see if the improvement is real or we  have just been lucky.

   An example may help you understand my point. In an air force,  a program was put in place to motivate more accurate bombing of targets  during practice run. The pilot who most accurately bombed a target site, was  given a $2,000 bonus for that month. After a year of continuing the program,  we found an unusual relationship. Every pilot who received a reward did  worse the month after. How is this possible? Rewards are supposed to  encourage and not discourage positive behavior. Why would pilots who did  well before do worse now just because they received a reward? The  explanation is simple. Each month, the pilot who won did so not because  he/she was better than the others but because he/she was lucky. We were  rewarding luck; thus, it was difficult to repeat the performance next month.  Control chart helps us focus on patterns of changes and go beyond a focus on  luck.

   In a field like medicine, over time a poor outcome occurs  the natural tendency is to think of it as poor care. But such rash judgments  are misleading. In an uncertain field such as medicine, from time to time  there will be unusual outcomes. If you focus on these occasional outcomes,  you would be punishing good clinicians whose efforts have failed by chance.  Instead, focus on patterns of good or bad outcomes. Then you know that the  outcomes are associated with the skills of the people and the underlying  features of the process and not just chance events. Control charts help you  see if there is a pattern in the data and move you away from making  judgments about quality through case by case review.
    

2. To tell a story. P-charts display the change over time.  These charts tell how the system was performing before and after we changed  it. They are testimonials to the success of our improvement efforts. Telling  these types of stories helps the organization to:
    

   • celebrate small scale successes, an important step in  keeping the momentum for continuous improvement.

   • communicate to others not part of the cross functional  team. Such communications help pave the way for eventual organization  wide implementation of the change.

   You can of course present the data without plotting it and  showing it. But without charts and plots, people not be able to see the  data. Numbers makes people understand the data but plots and charts,  especially those drawn over time, make people connect a story with the data,  they end up feeling what they have understood. For many people seeing the  data is believing it. When these charts are posted publicly in the  organization, control charts prepare the organization for change. They  transfer and explain the experience of one unit of the organization to other  units.

Which chart is right?

Figure 1:  P-chart is best when there are multiple observations  & outcomes are dichotomous and not rare

When tracking data over time, you have a number of options.   You could use a P-chart, designed specifically to track mortality or adverse  health events over time.   You could use an X-bar chart designed for  tracking health status and satisfaction surveys of a group of patients over  time.  You could use a moving average chart to help you construct control  chart for an individual patient's data over time.  This section helps you  decide which of these various charts are appropriate for your application.   If you do not have a specific application in mind or if you wish to learn more  about each of the various different charts, skip this section.  In the  following, we ask you 4-7 questions and based on your answers advise you which  chart is right for the application that you have in mind.

Have you collected observations  over different time periods?

• Yes►

• No►

Assumptions

When analyzing adverse health outcomes,  such as mortality, a useful method of analysis is P-chart. In the P-chart we  assume the following:

• The outcome is dichotomous, mutually exclusive and exhaustive events. Dichotomous means that there are only two outcomes. Mutually exclusive means that these two outcomes cannot both occur. Mutually exhaustive means than one of these two outcomes must happen. Thus p-chart may be considered appropriate for analysis of mortality rates if we agree that there are only two outcomes of interest (alive and dead) and that it is not possible to be both alive and dead, or to be in a state other than alive or dead.
   

• The outcome is measured over time to track improvements in the process.
   

• The observations over time are independent. Meaning that the probability of adverse outcomes for one patient does not affect the adverse outcome of the other patient. This is not always true. In infectious diseases, one patient affects another. When infection breaks in a hospital ward, the use of P-chart to analyze the outcomes of the process is inappropriate.
   

• Risk adjustment of the P-chart requires the availability of accurate predictions of risk faced by each patient in our sample. These predictions can be made based on a number of commercially available severity indices or based on the clinician's opinions. In either case, the quality of the analysis depends on the quality and the availability of these predictions.
   

• The analysis assumes that the sample of patients examined represent the population of patients treated at the specific time period.

These assumptions are important and should be verified before  proceeding further with the use of risk adjusted P-charts. When these  assumptions are not met, alternative approaches such as bootstrapping  distributions should be used.

P-chart

This section takes you through a step by step process of using a  P-chart; a type of control chart for analysis of mortality data. We introduce  the concepts behind P-charts in several steps. It is important that you take  each step and complete the assignment in the step before proceeding to the next.  To help you understand the concepts, the lecture focuses on data from one  hospital's mortality over 8 consecutive months.

Here is the data we need to analyze:

Calculate Mortality Rates

The first step is to create an x-y plot; where the x axis is  time and the y axis is mortality rates. Calculate mortality rates by dividing  number dead by the number of cases in that month.

Numbers are deceiving. They hide much. To understand numbers you  must see them by plotting them. Figure 1 shows the data plotted against time. 

Figure 2:  Observed Mortality in Eight Time Periods

What does the plot in Figure 1 tell you about unusual time  periods?  There are wide variations in the data.  It is difficult to  be sure if the apparent improvements are due to chance.  To understand if  these variations could be due to change, we can plot on the same chart two  limits in such a manner that 95% or 99% of points would by mere change fall  between the lower and upper limits. 

Setting limits

Figure 3 shows the steps in calculating control limits.

Figure 3:  Steps in Calculating Control Limits in P-charts

In step one the grand average p is calculated by dividing the total adverse  events by the total number of cases.  Note that averaging the rates at  different time periods will not yield the same results. Calculate the total  number of cases and the total number of deaths. The ratio of these two numbers  is the average P, the average mortality rate.

Next calculate the standard deviation of the data. In a binomial  distribution the standard deviation is the square root of grand average p  multiplied by one minus grand average p divided by the number of cases in that  time period. For example, if the grand average p is .25 and the number of cases  in the time period is 186, then the standard deviation is the square root of  (.25)*(.75)/(186). Table 3 shows the calculated standard deviations for each  time period:

Calculate the upper lower limit for each time period as grand  average p plus 3 times the standard deviation. This means that you are setting  the control limits so that 99% of the data should fall within the limits. If you  want limits for 90% or 95% of data you can use other constants besides 3.   The constant you use depends on number of observations in the time period and  can be read from table of t-values for different  sample sizes.  Table below shows the lower and upper control limits:

Please note that negative control limits in time periods 4, 6  and 8 are set to zero because it is not possible to have a negative mortality  rate.  Also note that the upper and lower control limit change in each time  period. This is to reflect the fact that we have different number of cases in  each time period. When we have many observations, we have more precision in our  estimates and the limits become tighter and closer to the average p. When we  have few observations, the limits go further away from each other.

Remember that we are trying to answer the question of whether  there has been improvements in the process. The control limits help answer this  question. If during a time period we have more mortality than can be expected  from chance then the process has deteriorated during that period. Any point  above the UCL indicates a potential change for the worst in the process. Any  point below the LCL indicates that mortality is lower than can be expected from  chance. It suggests that the process has improved. Figure 4 shows the resulting  plot:

Figure 4:  P-chart for Data in Table 1

Notice the peculiar construction of the plot, designed to help  attract the viewers attention to observed rates.  The observed rates are  shown as single marker connected with a line.  Any marker than falls  outside the limits is circled and highlighted.  The control limits are show  as a line without markers.   In the plot in Figure 3, all the data  points are within limit.

Risk Adjusted P-chart

See how to create risk adjusted P-charts in Excel  Video►   SWF►

P-charts were designed for monitoring the performance of  manufacturing firms. These charts assume that the input to the system is the  same at each time period. In manufacturing this makes sense. The metal needed  for making a car changes little over time. But in health care this makes no  sense. People are different. People are different in their severity of illness,  in their ability and will to recover from their illness and in their attitudes  toward heroic interventions to save their lives. These differences affect the  outcomes of care. If these differences are not accounted for, we may mistakenly  blame the process when poor outcomes were inevitable and praise the process when  good outcomes were due to the type of patients arriving at our unit.

Some institutions receive many severely ill patients. These  institutions would be unfairly judged if their outcomes are not adjusted for  their case mix before comparing them to other institutions. Similarly, in some  months of the year, there are many more severely ill patients. For example,  seasonal variations affect the severity of asthma. Holidays affect both the  frequency and the severity of trauma cases.

But even more significant source of change in the severity of  illness of our patients is our own actions. Many process changes lead to changes  in the kinds of patients attracted to our unit. Consider for example, if we  aggressively try to educate patients for the need for avoiding C-section, we may  get a reputation for normal birth delivery and we may attract patients who have  less pregnancy complications and wish for normal birth delivery. In the end, we  have not really reduced c-sections in our unit, all we have done is to attract a  new kind of patient who does not need cesarean births. Nothing fundamentally has  changed in our processes, except for the input to the process.

Risk adjustment of control charts is one method of making sure  that the observed improvement in the process are not due to changes in the kind  of patients that we are attracting to our unit.  To help you understand  this method of analysis, suppose we have collected the data in Table 5 over 8  time periods.   This table shows the patients severity of illness  (risk of mortality).  If you have forgotten what is severity and how we  measure it please click  here to return to a previous lecture on this issue.