New Journal of General Internal Medicine study indicates many established quality benchmarks are poor predictors of hospitals’ clinical proficiency
BOSTON (Mar. 26) – New research from Tufts Medical Center, published today on the website of the Journal of General Internal Medicine, found that some of the quality measures (QMs) developed by the Centers for Medicare & Medicaid Services and used by a myriad of organizations to rate and rank hospitals, may be inadequate, inaccurate or even potentially harmful to patients. Based on a review of nearly 40 previous studies, the article, “Quantity over Quality: How the Rise in Quality Measures is Not Producing Quality Results,” argues that QMs should be subjected to a thorough, expert, evidence-based peer review process, before receiving approval as quality benchmarks.
“Quality measures were established to set a standard of health care quality by which all health care institutions could be measured objectively. The implementation of the Affordable Care Act and pay-for-performance has only increased the importance of QMs to clinical practice,” said Deeb N. Salem, MD, Physician-in-Chief and Chairman of the Department of Medicine at Tufts Medical Center, and first author of the study. “QMs have a significant bearing on national standards for the delivery of safe, effective patient care, and should be subjected to stringent scientific evaluation and re-evaluation to ensure they are serving their intended purpose.”
Among the 11 underperforming QMs identified in the study, the researchers placed a particular focus on the use of beta blockers in non-cardiac surgery and glucose control in critically ill patients. Preliminary research indicated that both these interventions reduced mortality risk, and they consequently were established as QMs. However, subsequent large-scale randomized clinical studies found that perioperative use of beta blockers was associated with an elevated risk of stroke and hypotension, while glucose control in critically ill medical patients heightened the chance of hypoglycemic episodes. Contrary to initial expectations, these two interventions appeared to actually increase patient mortality rather than limiting it.
The authors also found problems with patient safety indicators (PSIs), markers of inpatient care quality, which were developed by the Agency for Healthcare Research and Quality to determine the risk of post-surgical or post-procedural complications. In recent years, hospital ranking systems have increasingly integrated PSIs into QM calculations, despite evidence from new research that they are unreliable in assessing preventable events and have a low positive predictive value of quality care delivery.
“Many of these performance measures were prematurely selected as quality standards before being scrutinized in definitive studies to confirm their validity,” said Salem. “Large randomized clinical trials have since disproven the effectiveness of a number of quality measures and patient safety indicators, yet their use continues.”
“Just like any other new treatment strategy, quality measures should be tested rigorously for their usefulness, effectiveness and safety, but they are seldom held to this standard,” said Harry P. Selker, MD, Executive Director of the Institute for Clinical Research and Health Policy Studies at Tufts Medical Center, and a co-author of the study. “These metrics are an important part of a national effort to improve care and contain costs, so we urge against their institution as quality benchmarks until they are tested for their impact on care, as is required for other changes in health care strategy.“
The Journal of General Internal Medicine article in its entirety can be found here. The paper will appear in the print edition of the journal at a future date to be determined.
The authors reported no conflicts of interest related to the study.
About Tufts Medical Center and Tufts Children's Hospital
Tufts Medical Center is an exceptional, not-for-profit, 415-bed academic medical center that is home to both a full-service hospital for adults and Tufts Children's Hospital. Conveniently located in downtown Boston, the Medical Center is the principal teaching hospital for Tufts University School of Medicine. Tufts Children's Hospital is the full-service children's hospital of Tufts Medical Center and the principal pediatric teaching hospital of Tufts University School of Medicine. Tufts Medical Center is affiliated with the New England Quality Care Alliance, a network of more than 1,800 physicians throughout Eastern Massachusetts. For more information, please visit www.tuftsmedicalcenter.org.