Quality: Beyond Measure?

Happy Labor Day! This holiday not only honors the American labor movement, it also unofficially represents a goodbye to summer, the start of football season, the end of wearing white (no judging here!), and the start of a new school year. Last week I brought my daughter to an orientation at her preschool—since she is my oldest, it was a new experience for the both of us. As I watched her mingle with the kids in her class, I started up a conversation with another mom. She looked absolutely exhausted. I asked her how her summer went and she unloaded about all of the time spent studying with her child in preparation for school to ensure her daughter is at the top of her class. Her four year old is reading books, learning her third language, and is wise beyond her years. Although all of her child’s achievements are very impressive, it made me wonder if our measurements of success are misguided.

“Not everything that can be counted counts, and not everything that counts can be counted”

Two of our most vital industries—education and health care—have become increasingly subjected to metrics and measurements. For most schools, success is attributed to standardized test scores. Programs like “No Child Left Behind” have pressured schools to use tests to measure student learning and teacher and school quality, and to impose sanctions based on test scores. In return, our teachers are compelled to “teach the test,” which results in narrowing the curriculum and pushing teachers out of the profession, all the while undermining the harder-to-measure gains in quality, such as student engagement, school climate, and critical-thinking skills.

Likewise with health care, the emphasis on measurements have spun out of control. Up until now, providers have been participating in the pay-for-reporting (P4R) model where they are incentivized just for reporting a set of measures (á la MU and PQRS). However, the tides are set to change next year, as most providers will live in the pay-for-performance (P4P) world where quality is assessed through a list of prescribed quality measures and mastery of EHR data entry. And much like our teachers, providers may find themselves in a medical equivalent to “teaching the test,” by diverting their attention to the aspects of care being measured at the expense of those not being measured.

Robert Berenson, MD, a fellow at the Urban Institute in Washington believes more focus should be placed on measuring a provider’s quality based on clinical reasoning, making timely diagnosis, and patient engagement, along with soft skills such as being personable, empathetic, forthright, and respectful, and providing a great patient experience.

You’re only as good as the data you enter

Even if the measures under P4P perfectly define the quality of a provider’s care, there’s still the challenge of mastering the art of data entry. For example, I recently went on a bike ride and forgot to log my activity on my Fitbit. Does this mean I didn’t take the bike ride? Of course not. Luckily for me, I didn’t need to prove my ride to anyone. However, for many providers, you will only be as good as the data you enter in your EHR accurately—regardless if the outcome is the same.

A recent analysis of the controversial dialysis 5-star ratings found that dialysis facilities that dramatically improved their star ratings likely benefited from reporting improvements, rather than improvements in clinical care.

One facility improved their Kt/V scores from the lowest percentile nationally to the 60th percentile in a single year. Another facility went from 0% to 78% of patients meeting the dialysis adequacy standard in a single year. How can these numbers change so drastically in one year? According to the analysis, the dramatic improvement could be attributed to better claims coding in year 2. In year 1, facilities may not have reported Kt/V on their dialysis claims form (probably because it doesn’t affect reimbursement). Once they learned the “rules”, and they knew where to enter the data, their performance improved by 100% or more. Amazing how not entering data in the appropriate fields can inaccurately deem you the worst facility in the nation in Kt/V scores.

It will be incredibly difficult to understand if true improvements in quality are occurring OR if providers are finally become more checkbox-savvy.

Intrinsic motivation > extrinsic reward

Perhaps one of the most dangerous unintended consequences of P4P and quality measurement could be the loss of intrinsic motivation to perform well across the board. For those who prescribe to the Daniel Pink school of thought, it wasn’t about the carrot and stick this entire time.

Pink, author of the book Drive, believes extrinsic rewards (money) can lead to demotivation for tasks that are intrinsically interesting or rewarding and actually harm performance in areas of work that require the least bit of cognitive skill. For work that requires mainly mechanical skill, bonuses made an impact on performance. But for work that requires managing complex situations, a larger bonus led to poorer performance.

In Drive, Pink states, “Organizations continue to pursue practices such as short-term incentive plans and pay-for-performance schemes in the face of mounting evidence that such measures usually don’t work and often do harm.” When a reward is offered, the focus shifts intently on achieving the reward which can motivate a single-minded focus, versus the need to think creatively and multi-dimensionally.

A viewpoint published in the Journal of the American Medical Association found that contrary to the motives of a financially oriented system, money does not always translate to the central need of most physicians. The relationship between target income and actual income has shown that physicians often reach a plateau in their desire for financial incentives after several years of practice. Instead, more attention needs to be given to intrinsic motivation. Physicians who are satisfied with their work lives provide better care. Rewards should reinforce, not undermine, intrinsic motivation to pursue needed improvement in health system quality.

“It is wrong to suppose that if you can’t measure it, you can’t manage it—a costly myth” –W. Edwards Deming

There is no doubt that quality measurement and value-based payment is appropriate as a concept, however the challenge for CMS will come in accurately measuring a provider’s overall value. This means deciding which measures are valuable, determining how to assess statistical significance, and learning to avoid unintended consequences, such as “gaming the system,” physician burn-out, and killing intrinsic motivation.

What do you believe is a good measurement of provider quality? Or can it not be measured? We would love to hear your thoughts below!

Diana Strubler, Policy and Standards Senior Manager, joined Acumen in 2010 as an EHR trainer then quickly moved into the role of certification and health IT standards subject matter expert. She has successfully led Acumen through three certifications while also guiding our company and customers through the world of Meaningful Use, ICD-10 and PQRS.