Buzzkill Part II: Health Care Quality Measures

Last month, I wrote a short article about healthcare pricing transparency and the opportunities and challenges presented by the newly available pricing data from hospitals and health plans. The article’s main message was that price transparency is a good thing, but it’s not without challenges.

This month, I’m focusing on the proliferation healthcare quality composite scores and report cards (usually in the form of a star ratings, grades, or provider rankings. For this article we’ll call them all “Composites”). Not surprisingly, my conclusion regarding the use of Composites parallels the use of pricing transparency data. Having information is a really good thing, but it must be used properly.

With that backdrop, let’s get to it.

Most of us have seen at least a few of the Composites such as CMS Hospital Compare Overall Star Ratings, U.S. News and World Reports Best Hospitals, or one of the many proprietary Composites from vendors peddling their own composite on a subscription basis. While these Composites have some differences, most are derived by complex, but well-tested analyses of utilization data, claims data, and/or hospital-reported outcomes data. These analyses get reduced to a single data point which, in theory, indicates whether or not high-quality health care has been provided, thereby predicting whether a patient can expect the same going forward.

This is where some of the problems with Composites begin. It’s not what they are but how they get used (or misused). To understand this, we must consider (1) the types of underlying quality measures used to develop the Composites, (2) the sources and limitations of the data used to develop Composites, and finally (3) how we use the Composites.

1. Types of Quality Measures Used to Develop the Composites

Quality measures used to build the Composites, like in other industries, fall into one of three general categories. These include (1) structural measures, which review a provider’s systems, staffing, and infrastructure to provide care; (2) process measures, which review what the provider does (or doesn’t do) in the delivery of health care; and (3) outcomes measures, which focus on the how well the treatment worked for the patient.

All three measure types are critical to truly understanding the provider and whether they will provide high-quality healthcare. Providers endeavor to rank high on all three because failure in one area can doom the provider even when they succeed in the others. For example, if a hospital fails to provide enough trained physicians and nurses to adequately care for patients (i.e., structural measure), there is a higher chance of poor care being delivered even if the policies and procedures used by the hospital are perfect. Likewise, if a hospital doesn’t follow well-established treatment protocols or practices (i.e., process measures), even highly trained medical professionals may fail to prevent poor care from being delivered. Finally, and most importantly, did the treatment work (i.e., outcome measure) regardless of the process or structure?

Based on the three categories, one might quickly conclude that the only measures that really matter are the outcome measures. Thinking this, however, could lead to incorrect conclusions about the care provided because it might fail to consider relevant factors such as risk adjustment, expected complications, and patient compliance among other factors. For a simple example, consider a hospital that treats only the really sick patients, but the doctors are highly skilled, and they follow the latest research regarding best practices to the letter. This provider’s outcomes lag another hospital that only treats slightly sick people. Without building some assumptions about risk factors, the simple outcome measure may not tell the whole story. To this point, most Composites adjust for underlying risk differences, but that is where the data is filtered through assumption-based algorithms that can alter the outcomes drastically and create vastly differing conclusions.

Likewise, a hospital with lots of highly skilled physicians who refuse to adopt best practices and treatment protocols can provide terrible care because they’re doing the wrong thing at the wrong time. This reminds me of a conversation I had back in my time as a hospital administrator with a well-respected member of the medical staff, where he warned me that “if we don’t follow the best practice, our physicians will get very good at doing the wrong thing.” The point is that each of the three types of measures are important in critically evaluating the quality of the health care provider, but none of them alone is perfect.

2. Sources of Data Used to Develop Composites

When it comes to the Composites, the most critical concepts to understand are the source of the data and the limitations. Typically, the data come from large sets of claims data (i.e., data providers submit to a payor to get paid). Less often, these report cards are based on data that are generated and submitted voluntarily by the providers themselves. Typically, these voluntary sources of data are limited scope (usually surgical) and type of procedure or process such as transplants and open-heart surgery to name a couple.

As it relates to claims data, regardless of whether they come from Medicare or commercial insurance databases, it’s important to remember that most claims-based data are generated by the provider to get paid rather than to measure quality. As a result, some important information may not be included that would be critical in really understanding the structure of the provider’s care model, the process used by the provider, and the outcomes of the care provided. For example, most payers claim forms, including electronic forms, limit the number of problems that must be reported by the provider to get paid. In reality, however, we all know some patients have many, many health factors at play, and to properly adjust for the risk of complications for those patients would require one to consider all of the problems that patient faces rather than the arbitrary limits placed on claims forms.

And then, of course, there are challenges with the source of the claims-based data itself and its applicability to all patients. Because Medicare is primarily focused on patients aged 65 years and older, its use limits applicability for care delivered to other populations, such as obstetrical care, Medicare data timing also creates a challenge because of the lag between treatment and reporting (i.e., data are typically almost 2 years old before available), so using them for current Composites can miss changes made by a provider (either good or bad) during the lag period.

The commercial claims data, while possibly more representative of the populations and timelier, have their own challenges including lack of standardization and not being readily available to compare. In addition, risk adjustment is also challenged due to the variations among the payors which can lead to statistical challenges in analyzing the data correctly.

Finally, another growing area of quality data is where most providers participate in voluntary submissions of data in the form of registries or data collaboratives. As noted above, these data sources resolve some of the problems of claims-based data (i.e., they were developed specifically to measure and improve quality), but these are typically limited to certain procedures (e.g., transplants, heart surgery, etc.) or certain processes (e.g., implementation of patient safety measures, protocols, etc.). This creates a separate set of challenges including provider bias in choosing which registries to with which to participate creating difficulties in developing a Composite.

3. So how do we use the Composites?

With the challenges with Composites in mind, the question is how we effectively use the data in a way that educates the members and health plan rather than mandating certain provider choices.

At ClaimDOC, we start by embracing the Composites as an important, but not the only, data point to consider when making provider choices. They should be considered along with lots of other information we’ve gathered regarding the local healthcare landscape. For example, patient preferences (identified in our Pave the Way™ program) as well as understanding local provider affiliations, may all play relevant factors in provider selection, and they should be considered alongside Composite data.

Second, ClaimDOC values the referral choices of trusted primary care physicians and a patient’s past experience with a provider. For example, ClaimDOC works with several direct primary care (“DPC”) companies who do an outstanding job of vetting high-quality, cost-effective specialists and facilities (incidentally, we find DPC to be an effective model for both ensuring quality as well as cost containment). If the DPC physician refers a patient even in the face of a lower Composite, we take the chance to get the information in front of the patient who can make a better-informed decision rather than push the patient based on the Composite alone.

Finally, we bring our own data and expertise to the table, including our own experts and claims data as well as other contracted partners to ensure we have the right approach. Overall, we believe this approach effectively achieves ClaimDOC’s triple aim of high-quality care, excellent member experience, and cost control.

So, as I did last month with pricing transparency, I’ll end where I began: Having quality data is critical in evaluating the value proposition, but we recommend caution in using Composites to mandate provider choices.