Risk Assessment

25 people have rated this page an average of  3.72 out of 5 stars.  See their suggestions and their questions.  Rate this page. 93 people have rated this page an average of  4.12 out of 5 stars.  See their suggestions and their questions.  Rate this page. Risk assessment lecture (see related slides) Introduction Session Objectives Disease Staging Patient Management Categories APACHE Medisgroup Computerized Severity Index Best approach? What do you know? Presentations FAQ More?   Recently asked:  If we just have a comment on a lecture do we put it here in question box? Or in suggestions? I just had a comment, not a suggestion. I thought this was an interesting topic this week, and I find it helpful also going over the excel P-charts due next week in class together . THis makes it much easier to understand  See answer to this and other questions.

Introduction

The following lecture is based on Alemi F, Rice J, Hankins W. Predicting in-hospital survival of myocardial infarction: A comparative study of various severity measures. Medical Care 1990; 28 (9): 762-775.

Why measure severity and risk of adverse outcomes?

There are many ways to evaluate the quality of health care services. One increasingly popular way is to examine the rate of adverse health outcomes. Mortality is frequently used to categorize outcomes for patients who have acute, life threatening illness. Within both the group of patients who live and the group that dies, some patient might receive good care while others receive poor care. Some patients may be so ill that they cannot be saved by the best of care. Other patients may get well despite poor care.

One way to measure quality of health care is to examine across a group of patients the rates of mortality and decide that these rates are lower or higher than what would be expected given the patients' severity of illness. When we want to do so, it is important to be able to separate the influence of the patients' severity of illness from the quality of care. There are at least two ways for doing so. One is to randomly assign patients to different providers, making sure that providers have equal chance of treating patients of high or low severity. The other is to measure patient's severity of illness and to statistically explain the variance in mortality with these measures first and then attribute any remaining variance to quality of care. Because it is difficult to randomly assign patients to providers, the second approach is the most common approach for measuring quality of care and is known as severity adjusted assessment of outcomes.

What is severity?

It is difficult to define what is severity because it is difficult to separate the influence of quality from severity. One definition is that severity is the progression of the disease when left untreated. Since no one in his right mind leaves diseases untreated, it is difficult to observe the progression of the disease. Another definition is that severity is the progression of the disease given customary treatment. This definition is also flawed because it customary treatment of a disease includes both poor and good quality care.

These difficulties of observing severity of illness within one person has led many investigators to suggest that severity is a comparative concept, meaning that a reasonable way to observe severity is to examine people at different stages of the disease. These individuals have different risks for mortality or adverse health outcomes. While it is difficult to observe progression of a disease, it is easy to see that some patients are further along in their illness than others.

There are many measures of severity, some commercial and others available through academic journals. Because severity measurements is the reason for attribution of adverse outcomes to poor or good quality, it is important to use a severity measure that is well accepted by the people whom we are evaluating. Hence, many select commercial severity indices. In this lecture we review a sample of these indices and their major differences.

Session Objectives

1. Discuss the role of risk assessment in the process improvement.
2. Describe the five severity indices presented in the course.
3. Compare and contrast the indices based on the sources of data used, scores produced, and accuracy of predictions.

Disease Staging

This index is designed for a broad range of patients and is not limited to specific illness. Disease staging assumes that diseases are first localized and later spread to other parts of the body systems. As a disease advances to higher numbered stages, it is associated with increased risk for the patient. The Computerized Disease Staging has four stages which can be subdivided into additional categories:

1. Stage 1 includes conditions with no complications and minimal risk for the patient.
2. Stage 2 includes problems which are contained in one organ or system.
3. Stage 3 includes problems in multiple sites and general systemic problems.
4. Stage 4 is death.

As frequently implemented, Computerized Disease Staging assigns severity stages based on diagnoses codes available through billing information. These codes are often based on the International Classification of Diseases, version 9, clinical modification (ICD-9-CM). The same codes are used by the Medicare reimbursement system to pay hospitals and physicians. Therefore, these codes are in wide use. During hospitalizations five to ten codes can be assigned to the patient. The first code is known as the principal diagnosis. The remaining codes are complications or other co-morbidity that were treated during the hospital stay. The severity scale produced by Computerized Disease Staging is ordinal, meaning that it preserves the order but not the magnitude of the differences between severity indices. Thus a severity score of 3 is worse than a score of 2 but not 1.5 times worst than 2.

Patient Management Categories

For more information on this index please refer to Young WW. Measuring the cost of care using patient management categories. Final Report to the Health Care Finance Administration, Baltimore MD Grant No 18p97063/3, 1985.

This index is designed for a broad range of patients and is not limited to specific illness. Patient Management Categories specify the costs for different care regimens. A panel of experts were asked to describe the ideal treatment for specific patient groups and a cost was estimated to correspond to the ideal care. Patient Management Categories groups patients based on their diagnoses codes available through billing information. Like Disease Staging it relies on the International Classification of Diseases, version 9, clinical modification (ICD-9-CM).

Unlike Disease Staging it produces an interval scale, meaning that the scores assigned preserve the magnitude of the difference in the severity indices. Thus, a score of 10 is not just more ill than a score of 5 but twice as worse off as a score of 5.

APACHE

The Acute Physiological and Chronic Health Evaluation (APACHE) index was originally designed for critically ill adult patients but was later proposed for use by patients outside critical care units. The APACHE score is the sum of three components. These are:

• Deviations from norm on 12 physiological variables like heart rate, blood oxygen level, or respiratory rate.
• Age of the patient.
• Chronic illness include coma.

APACHE is usually measured during the first 24 hours of hospital admission. The most abnormal values during this period are recorded and scored. APACHE produces an interval scale, where the score of 10 is twice as bad as 5.

Medisgroup

This index is designed for a broad range of patients and is not limited to specific illnesses. It scores have five levels: 0 through 4:

• At level 0, there are no clinical findings.
• At level 1, there are minimal abnormal findings.
• At level 2, there are either acute findings or findings with an unclear potential for organ failure.
• At level 3, there are clinical findings with high potential for imminent organ failure.
• At level 4, organ failure is indicated.

The Medisgroup scoring does not follow specific mathematical rules like APACHE, where the scores of abnormal findings are added. Instead, Medisgroup relies on artificial intelligence if-then rules to score combination of clinical findings. These types of if then rules create a scoring system that has a lot more flexibility. Medisgroup relies on key clinical findings during the first 24 hours of admission of a person to the hospital. Key clinical findings may be specific laboratory findings or it could be clinical observations. It produces an ordinal severity scale.

Computerized Severity Index

This index is designed for a broad range of patients and is not limited to specific illnesses. It relies on both ICD-9-CM diagnosis codes and key clinical findings. It produces an ordinal severity index, where for example a patient with a score of 4 is worst than a patient with a score of 2 but not twice as ill. This index scores range from 0 to 4.

It begins with the patient's principal diagnosis and uses physiological markers to adjust the diagnosis. The internal working of this index are not publicly available.

Best Approach to severity measurement?

Reliance on different sources of information

The indices differ in the source of information used to measure severity of illness. Some rely on physiological markers, others on ICD-9-CM codes. Both could be affected by treatment. Physiological markers are affected by treatment on route to the hospital and during the hospital stay. For example, medications may show a normal blood pressure for a patient that minutes before admission was in shock in the ambulance.

Similarly, because ICD-9-CM codes are based on the treated diagnosis through out the hospital stay, they may be affected by complications that arise due to poor care. For example, when a patient falls and breaks her hip then this is added to her severity score while clearly the fall was not the condition the patient came in with. Because both sources of data could be affected by treatment, whenever possible both should be used.

Diagnoses also reflect a summary of what the clinician taking care of the patient considered most likely reason for admission. Clinical markers, in contrast, are at best a reconstruction of what the documented information implies. Since in busy clinical practice, much of the information is not documented, the use of key clinical findings may not be as good as diagnoses codes.

Differences in scores produced

Severity indices also differ in the type of scores they produce, some produce ordinal scales others produce interval scales. Interval severity scores are most helpful for bench marking, as these numbers can be averaged and used in various control charts. In contrast, ordinal scales must be transferred to interval scales before use in control charts.

The following table shows the differences among the various severity indices:

Accuracy of Predictions

The accuracy of the various severity indices in predicting mortality from Myocardial Infarction in Hospitals in New Orleans area in 1985 was as follows:

These data suggest that the performance of these indices for patients with myocardial infarction may not be substantially better than what can be expected from predicting that all patients will survive. Whenever possible multiple severity indices should be used to improve the accuracy of predictions.   For more on ability of severity indices to detect problems in quality of care click here. 

What Do You Know?

Advanced learners like you, often need different ways of understanding a topic. Reading is just one way of understanding. Another way is through writing. When you write you not only recall what you have written but also may need to make inferences about what you have read. The enclosed assessment is designed to get you to think more about the concepts taught in this session.

1. What problems do you think reliance on ICD-9-cm codes create for measuring severity of illness? Keep in mind that ICD-9 codes are based on the entire course of a patient's illness including possible adverse events due to poor care. 
2. What is an ordinal scale, what is an interval scale, and why would you prefer one to the other?
3. Which do you think is more likely to be affected by treatment, physiological markers collected during the first 24 hour of admission or ICD-9-CM codes classifying the purpose of the hospitalization?  Keep in mind that key clinical findings may be normal if the patient is managed well.

Please send an email to your instructor with your responses to the above questions.    Make sure that the email subject line includes the course number, topic name and your name, otherwise it will not get to the right place.  If you wish to receive a delivery receipt, you may request the receipt from your email program.  Please respond to all of the questions within the same email.  Keep a copy of all your emails to the instructor till the end of the semester.

Presentations

To assist you in reviewing the material in this lecture, please see slides or listen to narrated lectures.   Narrated lectures require use of Flash.  

More

• One way of measuring severity of problems is through classifying and monitoring medication errors.
• Pharmacies have rich databases about nature of illness of their patients.   Here are studies of use of pharmacy data to measure severity of patients' illness  
• Risk assessment is not as benign as it first looks.  Take a look at risks of risk assessment.
• Examine nursing severity indices developed from the perspective of nursing
• Uses severity indices to evaluate performance of clinicians through out a community (e.g. Cleveland Quality Choice project):  Community wide measurement of quality
• Review the abstracts of the latest papers on Disease Staging.
• For the abstracts of the latest studies on Patient Management Categories click here.  For use of ambulatory care databases, click here.
• For abstracts of latest papers on Computerized Severity Index click here.
• To review abstracts of the latest publications related to APACHE click here
• For abstract of the latest papers on Medisgroup
• For a full text article on how to assess risks in substance abuse treatment programs click here.  This paper is a review of factors affecting treatment outcomes. Access to George Mason University's library is required to view this paper from off campus computers.  You can also see the Expected Treatment Outcome Scale and its scoring procedures.
• How about the risk of risk assessment?  One could be misled by risk assessment.  Risk assessment may create the image that differences among the patients have been adequately accounted for.  This may not be the case.  In a recent study Iezzoni shows that severity indices used by her did not adequately explain the differences among the hospitals.  For an annotated bibliography on this issue, click here.
• This lecture focused on in-hospital severity.  Several measures are available for outpatient settings.  Some examples of these are included here.  Alemi and colleagues have developed an approach to episodes of illness that is based on the nature of the diagnosis, not the intensity of visits. 
• For more information on the Disease Staging index please refer to Gonnella JS, Hornbrook MC, Louis DZ. Staging of disease: A case mix measurement. Journal of the American Medical Association, 1984, 251.
• For more information on the APACHE index please refer to Knaus WA, Draper EA, Wagner DP, Zimmerman JE. APACHE-II: A severity of disease classification system. Critical Care Medicine 1985, 13: page 10.
• For more information on the Medisgroup index please refer to Brewster AC, Karlin BG, Hyde LA, Jacobs CM, Bradbury RC, Chae YM. A clinically based approach to classifying hospital patients at admission. Inquiry 1985, 12.
• For more information on the computerized severity index please refer to Horn SD, Horn RA, The Computerized Severity Index. A new tool for case-mix management. J Med Syst. 1986 Feb;10(1):73-8. 

Frequently Asked Questions

Ask a question and we will answer it within the next 48 hours.  If you have no questions, please review the answer to the questions asked by others: 

Question: In the lecture number 9, we were told that we could only calculate the budget overrun if were given the actual costs. Since we are given the actual costs for this rapid analysis how do we actually go about solving for the budget overrun? Is there a specific formula?  Answer: I am assuming that this question needs to be answered for the RA in the section titled Project Teamwork. The method of answering the RA in the section on teamwork was explained in the lecture titled Project Execution and Control. This lecture includes a RA which shows how to use Excel to accomplish this goal. In addition, if you are using Microsoft Project you can use built in Tables to accomplish this feature. You must first make the schedule and save it as baseline. Then you can enter actual start and ends in the table under View under Tracking. You can see the cost and schedule variance in the table under View under Variance.  This question was asked on 4/9/2008 11:59:28 PM and answered on 4/10/2008 7:22:33 AM.

Question: For number 2, all we have to do is calculate pert?  Answer: No you also need to display showing a GANT chart and portion completed.  This question was asked on 4/9/2008 12:12:21 PM and answered on 4/9/2008 1:33:37 PM.

Question: For RA Question 2, are all three completed tasks on the critical path?  Answer: They are at start (meaning all three have zero slack time) but once they are completed one is no longer in the critical path. In this sense you should assume that the first two tasks are on the critical path.   This question was asked on 4/8/2008 11:58:44 PM and answered on 4/9/2008 8:37:18 AM.

Question: which computers have Microsoft project?  Answer: Four computers in the School lab in the basement have Microsoft Project. We can give you the room but not the specific computer that has this component. You need to try and find which one has the software loaded on it.   This question was asked on 4/8/2008 10:18:41 PM and answered on 4/9/2008 8:35:53 AM.

Question: Could you please explain what we're supposed to do for WYK #1? I am confused by the question.  Answer: You are expected to think of a project and assess the risks associated with the project using the tool provided online. Then you are supposed to report if you found the process helpful.  This question was asked on 4/8/2008 9:51:22 PM and answered on 4/9/2008 8:34:17 AM.

Question: our microsoft project software still does not have a PERT analysis option. this makes it impossible to do the rapid analysis #2. how can we go about this?  Answer: You can use Excel to replace the PERT estimates with a calculated duration. Alternatively you can use Microsoft Project at school labs.   This question was asked on 4/7/2008 10:04:32 PM and answered on 4/7/2008 10:24:13 PM.

Question: since in the tutorial video it shows that there is no way to correctly determine the budget overrun, do we put in a number that we think the budget will go over by or should we leave it blank?  Answer: I think the correct answer here is to say that it cannot be done. But if you think you have a way of predicting the budget overrun from the data provided, I would be interested in your creative solution.   This question was asked on 4/7/2008 9:56:45 PM and answered on 4/7/2008 10:23:16 PM.

Question: I have been trying to figure out # 2 of rapid analysis for a while. However, since I have Merlin instead of microsoft project, I cannot find the PERT analysis. I have searched through the help feature and online and still cant find it. What should we do if Merlin doesnt have PERT? Has anyone else who has a mac run into this problem?  Answer: You can do the PERT analysis in Excel. It is relatively simple, perhaps even simpler than doing it in Microsoft Project. You simple calculate the PERT estimate of duration as (pessimistic + 4 * expected + optimistic)/6.   This question was asked on 4/7/2008 9:07:45 PM and answered on 4/7/2008 10:22:06 PM.

Question: how do we do the questions for the rapid analysis this week, there is no videos or information within this weeks readings to help us calculate these measures.   Answer: These are based on information provided in previous sessions; I will add a video to show you how you should answer this question by Monday night.  This question was asked on 4/6/2008 5:28:26 PM and answered on 4/6/2008 8:05:07 PM.

Question: How do you enter planned expenditure in PM?   Answer: I will put together a video for you so that you can see how this is done. Please check here by Monday night.  This question was asked on 4/6/2008 3:44:58 PM and answered on 4/6/2008 5:02:17 PM.

Question: I dont understand the revision of assignments. What class are we in so im not sure if im completing and evaluation/get a question for the right class topic  Answer: I am sorry for the confusion. Students asked that the assignment for the tutorial paper feedback should be done at a time when nothing else is due. This is why we inserted this session in the middle. Next week we go back to regular sessions and you can follow it best in the syllabus.   This question was asked on 4/2/2008 2:35:25 PM and answered on 4/2/2008 7:42:01 PM.

Question: If our tutorial paper is on a topic you discussed in class, can we use your examples?  Answer: Yes.   This question was asked on 4/2/2008 12:34:47 PM and answered on 4/2/2008 7:40:12 PM.

Question: I just want to make sure I understand the due dates for this assignment. For class on 3/26, the homework is not due until 4/9, so the only thing necessary for class is a draft of our tutorial paper.  Answer: Yes, that is correct.   This question was asked on 3/29/2008 10:10:09 PM and answered on 3/30/2008 7:16:57 PM.

Question: If we just have a comment on a lecture do we put it here in question box? Or in suggestions? I just had a comment, not a suggestion. I thought this was an interesting topic this week, and I find it helpful also going over the excel P-charts due next week in class together . THis makes it much easier to understand  Answer: Please put comments in the suggestion box and questions in the box titled "Get Answers." I am delighted that the P-chart face to face lecture helped you.   This question was asked on 2/26/2008 11:57:17 PM and answered on 2/27/2008 7:27:38 AM.

Question: In lecture, why did you say anything over a 85% survival rate was almost impossible?  Answer: What I had said was that predicting with 100% accuracy who will live and die is probably unlikely as there is some inherent chance events built into the process. Evey the healthiest people may die by change and sickest people may live by luck. So 100% accuracy is not possible, perhaps 85% to 90% accuracy is the ceiling of how well mortality can be predicted.   This question was asked on 2/26/2008 10:46:55 AM and answered on 2/26/2008 8:19:29 PM.

Question: Although i do think that using both ordinal and interval scale in predicting mortality will produce better results, is it possible to use just one scale if so what are the implications?  Answer: It is possible to use either an ordinal or an interval scale to predict patient's prognosis. For example, patients' mortality can be regressed on an ordinal scale by introducing several dummy variables, one for each value of the ordinal scale. There is no a priori reason to expect that either the ordinal or the interval scale will be more accurate but the regression on the interval scale will require fewer data items as there are fewer parameters to estimate. As the number of levels of the ordinal scale increases more parameters need to be estimated and therefore more data is needed.  This question was asked on 2/25/2008 1:12:07 AM and answered on 2/25/2008 7:02:49 AM.

Question: What are the problems with using the ICD-9 codes?  Answer: Do you mean in comparison to ICD-10 codes? The definition and number of the diseases change over time and therefore new versions of disease classifications are needed.  This question was asked on 2/24/2008 8:24:21 PM and answered on 2/24/2008 9:38:13 PM.

Question: please explain to me again the difference between Ordinal and interval scales...we do use them for the severity of the pain....Ordinal is 0 to 4 and Interval is 1 to 10??? I'm not sure I understand this.  Answer: Interval scale preserves how many times a patient is sicker than the other. Ordinal scales only preserve whether a patient is sicker than another and not how many times. For example, in an interval scale a patient scored 6 is 3 times sicker than a patient scored 2. But in an ordinal scale a patient scored 6 is only sicker than a patient scored 2 but not necessarily three times sicker.  This question was asked on 2/21/2008 8:44:45 PM and answered on 2/21/2008 8:47:22 PM.

Question: For the Medisgroup scoring, are the "if-then" groups updated every so often to keep up with new medical findings?  Answer: I am not sure how often these if-then rules are updated but you are certainly right in the importance of such updates.  This question was asked on 2/20/2008 1:53:53 PM and answered on 2/21/2008 6:46:14 AM.

Question: is it possible to have a severity scale that measures death as the severity of the case. for example, if patient A is going to die in 10 days vs. Patient B who might die in 3 days.  Answer: Many severity indices measure patients' prognosis (probability of mortality). Not only what you are suggesting is possible but it is the procedure most commonly used.   This question was asked on 2/20/2008 10:15:18 AM and answered on 2/21/2008 6:43:58 AM.

Question: Who or what group gets to set the ICD-9-CM codes and standards?  Answer: Here is a quote from World Health Organization regarding origins of ICD-9 codes: "ICD-10 was endorsed by the Forty-third World Health Assembly in May 1990 and came into use in WHO Member States as from 1994. The classification is the latest in a series which has its origins in the 1850s. The first edition, known as the International List of Causes of Death, was adopted by the International Statistical Institute in 1893. WHO took over the responsibility for the ICD at its creation in 1948 when the Sixth Revision, which included causes of morbidity for the first time, was published."  This question was asked on 2/19/2008 12:13:07 PM and answered on 2/20/2008 7:24:10 AM.

Question: Were DRGs and ICD codes developed so higher levels of care could be provided for sicker patients, or were they developed so finance departments could predict the cost of care?  Answer: ICD codes were developed to classify diseases for various purposes but mostly for billing purposes. DRGs were developed for reimbursement of hospitals.  This question was asked on 2/16/2008 3:35:53 PM and answered on 2/16/2008 4:28:56 PM.

Question: Do insurance companies look at ICD9 codes as well as an additional index such as APACHE for complex patients? Also if there are numerous codes is this a trigger for the insurance provider to determine if there was poor care?  Answer: Some insurance companies examine the risk adjusted performance of the providers on their panel and may drop a provider whose practice is not efficient.  This question was asked on 10/14/2007 9:51:29 PM and answered on 10/15/2007 3:35:59 PM.

Question: While all of these indices are useful for their own purposes, which ones are practical in everyday practice of medicine? In retrospective studies, researchers have to go through charts and get the impressions from the clinician. As mentioned, a busy medical practice will not allow time to document indices such as Computerized Disease Staging. Yet, clinical markers are sparse at best.   Answer: These are all practical indices that are in use in some clinic right now. If the index requires chart review, additional personnel are needed to complete the chart review. Otherwise, if the index requires ICD9 codes then the severity can be established without chart review and through analysis of billing data.  This question was asked on 10/2/2007 9:46:58 PM and answered on 10/3/2007 12:59:01 PM.

Question: If risk assessment is the measuring of two quantities of the risk,such as the magnitude of the potential loss, and the probability that the loss will occur. If is not true that what risk assessment does is to measure outcomes?  Answer: Yes risk indexes measure the probability of adverse outcomes.  This question was asked on 10/2/2007 2:01:10 PM and answered on 10/2/2007 8:50:52 PM.

Question: Which scale would you use, ordinal or interval scale and why?  Answer: interval, because to do severity adjustment you need an interval scale.  This question was asked on 10/2/2007 11:42:59 AM and answered on 10/2/2007 8:49:42 PM.

Question: Although a collaborative approach is the most useful with assessment tools. Are there instances in which one particular assessment is best?  Answer: I think if you have to choose one, select the approach which is most convenient to implement as the various approaches do not seem to differ in accuracy.  This question was asked on 9/30/2007 8:59:26 PM and answered on 10/1/2007 9:48:57 AM.

Question: SInce ICD9 codes are assigned by hand and dependent upon the documentation completeness of the clinicain doesn't the skill of the medical records coder add an additional layer of possible inprecision?   Answer: Yes it does. You should also note that the diagnosis itself is made by a clinician and there is a layer of unreliability added to it too. But physiological markers are also unreliable as many patients information at end of life are not recorded -- imagine a code blue and the rush to help the patient. In this rush many information items are not recorded.  This question was asked on 9/30/2007 8:14:19 PM and answered on 10/1/2007 6:54:04 AM.

Question: In this lecture, you state to use multiple methods to measure severity but which do you feel is the best method?  Answer: I particularly like ICD-9 based indices, as they require less investment  This question was asked on 9/28/2007 11:11:36 PM and answered on 9/29/2007 12:21:44 PM.

Question: In this lecture you state that the best way to measure severity is to use multiple methods but which method do you feel will serve the best?  Answer: I particularly like ICD-9 based indices, as they require less investment  This question was asked on 9/28/2007 11:09:08 PM and answered on 9/29/2007 12:21:23 PM.

Question: I think it important for severity to not be included while looking at the quality of care given. While it may be known to health professionals, I think patients and especially family member may over look this.   Answer: Why? Severity should be included as outcomes depends on it and without adjusting for severity adverse outcomes will be mistakenly attributed to poor care.  This question was asked on 9/28/2007 10:44:14 PM and answered on 9/29/2007 12:20:41 PM.

Question: No question, really just a compliment on how well the last class went. Thank you for reviewing the homework in class. The risk assessment lecture was interesting, however it seemed that there was not really enough time to spend on fully exploring the various modes of assessment. Lots of independent reading is needed to fully understand the concept  Answer: Thanks.  This question was asked on 9/28/2007 9:56:38 PM and answered on 9/28/2007 10:11:20 PM.

Question: Do physicians and hospitals commonly agree or disagree on personal choices for severity indexes?  Answer: I am not sure if they do. I do know that clinicians typically emphasize face validity while scientists focus on predictive validity.  This question was asked on 9/26/2007 7:32:26 PM and answered on 9/26/2007 8:20:45 PM.

Question: I just want to know whether these severity measures explain why there are differences in the length of hospital stays for some patients?  Answer: Yes they do. In one study 13 fold differences in hospital mortality was explained by severity of illness of their patients.  This question was asked on 9/25/2007 7:23:29 PM and answered on 9/25/2007 9:27:54 PM.

Question: Why are so many references to journal articles listed at the bottom of the lecture page when there is obviously no way we will have time to read through all of them. What is the purpose of those being there? What are we supposed to gain from reading any of these, and which one's are most important to look at?  Answer: The references are there in case you want to go to the original documents and pursue this in more depth. You are not required to do so.  This question was asked on 10/3/2006 1:59:16 AM and answered on 10/4/2006 10:58:52 PM.

Question: Each of the evaluation methods showed almost the same percentage of success in predicting patient health. In the lecture it said that combining methods would lead to a better conclusion, but I don't see mathematically why that would be. It seems to me that if you have two types of analysis that are both effective to 80% and are independent of each other, then in order to get their composite score if they were combined you would have to multiply their effectivenesses together. That would lead to an even lower rate of correct analysis. Now if the two types of analysis share dependencies then potentially they might be able to come up with a better overall outcome, but it seems that overall the amount of conflicting factors that would lower the potential effectiveness of the analysis would outweigh the amount of complimentary factors which would have to line up very well in order to increase the chance of estimating the patient's outcome correctly.  Answer: There is no reason to expect the instruments are independent of each other. Typically, one instrument is accurate on some patients and the other on another and the combination on a larger set of patients.   This question was asked on 10/3/2006 1:55:39 AM and answered on 10/4/2006 10:57:40 PM.

Question: Which one of the indices outlined in the lecture is more popular in today's clinical setting? Additionally, are there any other indices in development now?  Answer: The answer of popularity is really a commercial question and I do not want to take sides among these indices. There are however a number of studies of accuracies of the indices and it seems that the accuracy of these indices depend on the diseases being examined. I would encourage you to read under the more section more about each indice and their evaluations. There are always new indices under development as diseases change. We have recently patented an approach to measurement of episodes of illness. You can find this at http://gunston.doit.gmu.edu/healthscience/709/SeverityEpisodesIllness.asp  This question was asked on 10/16/2005 12:54:03 PM and answered on 10/17/2005 9:01:16 PM.

Question: In using ICD-9 coding principles at work, does the coding system used such as 3M or Precise have an effect on assessing the severity of illness and how can we go around it?  Answer: Obviously, ICD-9 based severity indices are more likely to report a higher score when the diagnosis coded is more serious. If you want to show that your patients are severely ill you should make sure that all serious diagnoses of the patient are listed and coded.  This question was asked on 9/27/2005 6:20:10 PM and answered on 9/27/2005 6:38:33 PM.

Question: In one of the answers to a FAQs, it was stated that severity and prognosis were considered interchangeable. That puzzles me since one can have a severe illness with agood prognosis. For example pancreatitis can be severe but have a good prognosis. Canyou clarify?   Answer: Severity and prognosis are intererelated concepts but of course one can have different values for each concept. The earlier point was that when we say the patient is severely ill we expect the patient to have poor prognosis.  This question was asked on 9/27/2005 2:05:16 PM and answered on 9/27/2005 6:36:49 PM.

Question: You said in the lecture that the best approach to measure severity is to combine more different severity indices. My question is which of these severity indices are more commonly used?  Answer: This is an interesting question. The severity indices used in hospitals depend on the whether the hospital is a teaching hospital or non-teaching and which indices the clinician is educated in as well as how much importance is put on technology. Using indices means you have to collect data and have the resources to do that. Ememrgency Rooms often use an index to help them decide proper placement of critical patients. The ones listed in the lecture are most widely known. Another one is the 3M APR-DRG classification which is based on ICD-9 codes and is rated at discharge.  This question was asked on 6/26/2005 2:13:18 PM and answered on 6/26/2005 6:19:34 PM.

Question: I was surprised that you did not discuss how the use of ICD-9-CM's to determine reimbursement for medicare/aid could potentially influence the measure of severity. Would that be considered part of the variance?  Answer: You are correct. In reality more detailed documentation by the care provider can lead to increases in severity when data is risk adjusted. Documentation and coding are the first elements that are analayzed when hospitals are monitoring indicators with expected outcomes.  This question was asked on 6/24/2005 10:13:39 PM and answered on 6/25/2005 5:44:21 PM.

Question: I thought you said that ordinal data can be converted to interval data for the purpose of creating a control chart. How is that done?  Answer: An ordinal scale can be transferred to an interval scale by additional analysis. For example, if we have an ordinal severity scale, one can calculate from this scale an interval scale by regressing patients' survival on the ordinal scale. Then average values for each level in the ordinal scale will have interval scale properties. Thank you for the question.  This question was asked on 6/24/2005 10:09:37 PM and answered on 6/28/2005 10:02:06 PM.

Question: When do we measure health outcomes? Do patients be assessed at the beginning of admission or before discharge? Is it enough to measure one time per hospitalization? In patient with chronic disease, the progress of disease is dynamic. How do we measure risk of adverse outcomes in this case?  Answer: Clearly, if the intention is to measure the quality of care, you should measure patient's severity before the care is given and patient outcomes after the care is given.  This question was asked on 6/20/2005 11:19:04 PM and answered on 6/22/2005 3:46:24 PM.

Question: The lecture says to use both key clinical finding and ICD-9-CM diagnosis codes whenever possible. The Computerized Severity Index does so. However, it is based on an ordinal scale. Does this ordinal scale have anything to do with the "fact" that it is only 77% accurate? Other indices use only key clinical findings or diagnosis codes but have higher accuracy. Any explanations?   Answer: The accuracy rates are alcualted as percent of variance explained. An ordinal and an interval scale can both explain high percentages of variances. The mere fact that something is ordinal should not affect its ability to predict survival.  This question was asked on 6/19/2005 5:37:35 PM and answered on 6/22/2005 3:45:33 PM.

Question: If these modes of severity measures indices only hope to accomplish an accuray rate of 76-82% accuracy, how confident can the physician be that the diagnosis isn't something else? A measure of 24%-18% chance error is huge, considering the industry gold standard for error rates less than 5%. Intuition can only make up for a small portion of this, what measures is the industry taking to improve this?  Answer: These are percent of variance explained and not percent of cases correctly classified, two different but related concepts. Imagine a situation where 100% of variance is explained. Then the outcomes are guranteed based on your severity of illness, no heroic saving of lives, no patient variations affecting outcomes, no DNR orders, and of course no differences in quality of service. You are attributing all of unexplained variance to poor care. This may not be accurate. Regarding your latter point of what is the industry doing, a great deal. More accurate measures have been devised and are reported in the literature. But for the time being CMS has decided to move away of publishing risk adjusted outcomes of hospitals because they do not beleive a sufficiently accurate index exist. Some analyst have devised new methods for benchmarking that allow feature matching of patients. See for example http://gunston.doit.gmu.edu/healthscience/730/severity.asp  This question was asked on 5/11/2005 10:58:35 AM and answered on 5/12/2005 3:29:14 PM.

Question: How can I get access to the Disease Staging, Medisgroup, and CSI indices to use for a study? Do they have to be purchased or are they available anywhere? Kyle Hanks  Answer: All three indices are available through commercial firms. You can search for the firm in goggle or alternatively you can write to the first author of the papers cited for these indices in our web page.   This question was asked on 4/22/2005 10:57:49 AM and answered on 4/23/2005 11:35:29 PM.

Question: Do we have to complete the survey "Questionnaire"?   Answer: No, there is no need to do so  This question was asked on 4/20/2005 5:40:42 PM and answered on 4/21/2005 7:57:06 PM.

Question: Where can I find the information regarding the risk analysis biwekly?  Answer: I am in the process of thinking through that and will post it before next Tuesday.  This question was asked on 4/6/2005 9:19:33 AM and answered on 4/8/2005 12:15:37 PM.

Question: Regarding question number one, I did not see this specifically answered in the lecture. Is this question open for our own opinion? I just want to make sure I'm understanding the focus of the question. Thank you!  Answer: I am looking for a discussion of reliability of ICD-9 codes and physilogical markers and how likely they are to be influenced by patient's care as opposed to patient's condition. You can of course rely on your own experiences to answer this question.  This question was asked on 4/5/2005 2:11:56 PM and answered on 4/8/2005 12:15:01 PM.

Question: The link under the Accuracy of Predictions section marked "click here" regarding more on the ability of severity indices to detect problems in quality of care is not working.  Answer: Thank you for pointing this out. I have fixed the link.  This question was asked on 3/29/2005 2:59:48 PM and answered on 3/31/2005 1:14:35 PM.

Suggestions for Improvements

Add your own suggestions or read below suggestions made by others regarding how to improve this session:

Suggestions: There was good information on the various comparitive tools on risk assessment.  This comment was left on 4/22/2008 11:16:28 PM. 

Suggestions: This guest lecture was the best because Lee did a good job of actually catering to the audience. His delivery was consise and effective. I guess he had an advantage by knowing who we were as far as our course content and classroom dynamic. I didn't get the sense that the other guest lecturers knew they were speaking to undergraduate students and that this is supposed to be an introductory course.  This comment was left on 4/9/2008 10:49:52 PM. 

Suggestions: I liked how we were able to apply the project management techniques that we have learned to real life projects in the WYK.   This comment was left on 4/9/2008 4:53:27 PM. 

Suggestions: The speaker was awesome.  This comment was left on 4/9/2008 4:16:01 PM. 

Suggestions: It is extremely hard to do the work for this class especially the rapid analysis section when we don't have the proper tools. I searched the computer labs for the correct program that would help us make a pert diagram, but was not successful. I think it would be a lot more useful and beneficial to all the students if we were given assignments that we can actually do on our computers in the way in which it is shown in the videos online  This comment was left on 4/9/2008 3:43:21 PM. 

Suggestions: I enjoyed the speaker. he was very enthusiastic   This comment was left on 4/9/2008 2:48:05 AM. 

Suggestions: I really enjoyed the guest lectures and Prof. Alemi's lecture on why his project 'failed' - real life examples demonstrate the lessons very well. Both Lecturers did extremely well at stimulating the class and engaging each person with questions and constant interaction.  This comment was left on 4/8/2008 10:42:30 PM. 

Suggestions: I really enjoyed this class. I thought the guest speaker was very informative and gave a different perspective on certain Healthcare issues.   This comment was left on 4/8/2008 11:53:17 AM. 

Suggestions: I enjoyed this weeks class. I think that the speaker was very informative and I found the peer review of the tutorial paper to be helpful.  This comment was left on 4/6/2008 6:10:10 PM. 

Suggestions: Last week's speech was rather long and complicated. The review of the tutorail I feel will help me with my final draft.  This comment was left on 4/5/2008 3:38:08 PM.