Will AI Improve Healthcare Quality?
Understanding payer and provider incentives in quality measurement & improvement
Tl;dr
Large healthcare delivery organizations and health plans have significant financial incentives to perform well on measures of quality
These entities can spend millions of dollars annually on measuring and reporting quality metrics, often through manual and labor-intensive workflows
Providers and payers typically use a patchwork of homegrown tools, internal teams, and sometimes external consultants to manage quality measurement, reporting, and improvement work
New AI and ML tools enabling automated data abstraction, analytics, and efficient and targeted outreach to patients can reduce the manual burden, improve accuracy, and surface new insights related to quality
But with their plodding track record on tech adoption, it remains to be seen whether providers and payers will take on the hurdles of intensive vendor procurement processes and change management to bring on these new tools in a meaningful way
When it comes to ensuring their customers receive high-quality healthcare, healthcare providers (hospitals, doctor’s offices) and health insurance companies (“payers”) have a lot at stake. Reputation matters: academic medical centers jockey for top spots in rankings like the U.S. News Honor Roll, many health systems tout their accreditations with organizations such as The Joint Commission, and health plans and hospitals are publicly rated on measures of clinical quality and customer experience on the Medicare Plan Finder Tool and Medicare’s hospital quality tool. These signals can meaningfully impact brand perception and customer acquisition.
The financial incentives for quality performance are also significant. Clinicians caring for Medicare patients must meet certain quality performance thresholds to avoid a -9% adjustment to their Medicare reimbursements, and hospital reimbursement can be reduced if facilities perform poorly on metrics such as preventable readmissions. On the health plan side, poor performance on Medicare quality metrics (measured by the Star ratings program) can lead to multi-billion dollar swings in public company earnings, and persistently poor performance can lead to plans being terminated from the Medicare or Medicaid programs.
With so much at stake, it’s no surprise that healthcare providers and payers dedicate resources to measuring and managing quality. These organizations need to report on many different quality measures to disparate entities – CMS alone tracks hundreds of quality measures across its various programs, and a 2024 JAMA study showed that primary care physicians reported on an average of 57 quality measures across their value-based contracts. To do this, they must prioritize which measures to focus on at any given time, invest in quality improvement initiatives, and develop infrastructure to capture, clean, and consolidate the underlying data for reporting. Research suggests that a health system could spend $5M-$10M annually on personnel costs for quality measurement and reporting, and U.S. physician practices in aggregate spend upwards of $15B annually to report quality metrics. Large payers may be spending even more than providers, considering the sizable swings in their financial performance that can be associated with their quality outcomes. The emphasis on quality measurement is likely to grow as CMS continues to shift reimbursement to value-based care models, which inherently prioritize quality over quality of care.
In the age of AI, providers and payers have access to new tools to optimize how they measure and report on healthcare quality and drive improvements in outcomes. Startups like Carta Healthcare, Pharos Health and Quality Health are building infrastructure to automate data abstraction and reporting, and companies like MedOrion and Cadence have developed solutions to help payers and care delivery organizations to improve performance on quality metrics. The question remains, how ready are provider and payer organizations to adopt and pay for these solutions?
What’s in a Quality Metric?
Providers measure and report out on quality measures set by many different entities – the government (CMS, state departments of health), non-governmental payers (such as commercial health plans), accreditation organizations (like The Joint Commission), and quality data collection organizations (like The Leapfrog Group). In the hospital setting, a common area of focus is rates of dangerous infections like sepsis, catheter-associated urinary tract infections, and others. Separately, health plans report their performance on quality measures to CMS and the Agency for Healthcare Research and Quality (AHRQ). These can include screening measures (e.g., did we screen you for cancer?), process measures (e.g., were you prescribed the right medication for your diagnosis?), outcome measures (e.g., did your blood pressure decrease if you have hypertension?), and customer experience measures (“what was the call center’s average hold time?”).
Healthcare quality was in the news quite a bit in 2024 as some of the largest payers (Humana, Elevance, UnitedHealth, and others) saw deteriorating performance on Medicare Advantage quality measures, aka the Star Ratings program. At a high level, Star ratings comprise a set of measures that Medicare Advantage plans have been evaluated on every year since 2006. Plans get a publicly posted report card with a grade of 1-5 summarizing how they did across all of the quality measures in question. Plans that earn a 4-star rating or higher get a 5% payment bonus from CMS as well as increased rebates that enable them to offer more benefits to enrollees. Plans are graded on a curve, and average Star ratings have been decreasing over the last few years. Many health plans have been up in arms about their decreased Star ratings and 2024 saw many lawsuits against CMS about the Stars calculation methodology.
This is because the dollars attached to Stars can be huge – according to a highly informative Health Tech Nerds guest post explaining the Stars program, decreases in Humana’s latest Star ratings could reduce the company’s 2026 earnings by close to $3B. On the flip side, strong performance on quality comes not just with bonus payments, but also other perks that can boost a health plan’s financials. Medicare Advantage plans with the highest possible rating of 5 stars are uniquely allowed to enroll new members year-round, not just during CMS’s strictly defined annual enrollment period for Medicare.
A Quality Metric by any Other Name Would be Just as Tedious to Measure
Providers and payers typically have dedicated Quality teams focused on measuring and maximizing performance on quality metrics. The largest payers typically offer many health plans across different lines of business (e.g., Medicare, Medicaid, Commercial). They often have to track different quality measures for different plans – for example, Medicare and Medicaid do not have the same reporting requirements, and Medicaid quality measures vary state-by-state. Within a given line of business, like Medicare Advantage, each plan the payer offers (say, Humana’s Kentucky HMO vs. Kentucky PPO plan) is scored separately, so payers need to think about which quality measures to focus on for each plan.
In thinking about optimizing Star ratings, large payers typically look at their past performance on each plan’s quality measures and upcoming changes to the CMS grading methodology (for example, modifications to the weightings of certain quality measures towards the overall Star rating calculation), and then try to forecast what CMS’s “grading curve” for the measures will look like that year. Large payers may also hire consultants like Accenture and others to support this work, as well as auditors who are involved in data validation before it is submitted to CMS. Throughout the year, Quality team members review internal reports monitoring ongoing performance against quality measures for each plan offered by the payer. This can help drive quality improvement initiatives, including outreach to providers in the payer’s network to encourage them to take actions to improve certain metrics for their patient population. These activities are important for smaller payers as well, but they typically have smaller budgets to work with.
On the provider side, hospitals and health systems also have Quality teams that are responsible for collecting data, submitting reports to external entities, and partnering with internal stakeholders to improve quality performance. A study of Johns Hopkins Hospital’s Quality team showed that the hospital spent over 100,000 person-hours, exceeding $5M in labor cost, to report on 162 quality metrics in 2018. Data collection and validation, which tends to be very labor intensive, comprised 65% of the time. This typically entails teams of nurses and administrative staff manually combing through medical claims, patient charts in the electronic health record (EHR), patient survey data, and other sources to find the needed data. Think windowless room, harsh fluorescent lighting, walls of filing cabinets, and nurses inputting data from Epic on one monitor into a database on their other monitor (at least, this is what I observed back when I worked at a hospital in San Francisco). While some metrics, known as electronic Clinical Quality Measures (eCQMs), can be automatically pulled from the EHR, this comes with the tradeoff of increasing the documentation burden on clinicians. On top of labor costs, health systems also pay vendors to support their quality work; for example, Johns Hopkins Hospital in 2018 spent ~$600k on vendors including US News & World Report, Vizient, and a patient experience survey vendor.
While Johns Hopkins’ $5.6M spend on quality sounds like a drop in the bucket compared to the hospital’s $2.4B annual expenses that year, it is not insignificant in the context of many hospitals’ financial constraints and slim operating margins (well, maybe not for NYU, which spent an estimated $8M on a Superbowl ad). The quantified cost also doesn’t account for other burdens of quality reporting, such as requiring clinicians to document work in specific and often cumbersome ways, hiring nurses to staff Quality teams, and expenses borne by other departments in the health system to contribute to quality measurement and management.
Draw Thy [AI] Tool
As with many heavily manual tasks in healthcare, there is undoubtedly potential for automation and AI to improve efficiency in quality measurement, reporting, and improvement efforts. On the measurement and reporting side, data abstraction is already being automated, reducing some of the need for people to manually review documentation and pull out relevant information. Considering Johns Hopkins Hospital spent >60,000 people-hours on data abstraction for quality reporting in 2018, these types of solutions could make a meaningful difference. In addition to companies that are automating data abstraction specifically for quality reporting, one can imagine that the ambient scribing companies (e.g., Ambience, Abridge, etc.) could expand into this space given their deep integration into health systems’ data streams and their existing use of AI to turn that data into useful insights.
Beyond measurement and reporting, a number of companies have emerged to help health systems and payers improve performance on quality metrics. Many provide solutions to turbocharge health systems’ or payers’ care management teams, which typically comprise social workers and nurses outreaching patients to support specific health goals (e.g., increasing cancer screening rates, managing chronic illnesses). For example, Cadence provides health systems with end-to-end IT and clinical staffing infrastructure to run chronic disease management programs. MedOrion uses behavioral science to equip health plans to outreach their members with specific, personalized messaging intended to prompt actions – for instance, completing a cancer screening – that can improve clinical outcomes and boost quality measures. Dozens of conversational AI solutions, like Ellipsis and Laguna, automate calls to patients on behalf of care management teams.
These types of solutions could reduce manual work and the associated human error (e.g., in data collection and manipulation) inherent in manual tasks, and they could improve health systems’ and payers’ ability to get their patients/members to take actions that better their health (and thereby improve the organizations’ performance on quality measures). But this is just the tip of the iceberg; better data and insights on quality could also meaningfully improve care. Quality Health CEO Dr. Chris Williams shared an example of applying AI to healthcare data to trace the source of a surgical site infection by identifying that two infected patients had both used the communal restroom. Quickly surfacing such insights could save lives.
O AI, AI, Wherefore Art Thou AI?
Despite these benefits, it remains to be seen whether health plans and systems will adopt such solutions anytime soon, or at meaningful scale. Culturally, the type of behemoth healthcare organizations that could benefit the most from AI are often slow to change the way things are done – even when doing so could lead to cost savings or revenue improvements – or prefer to try to build solutions in-house rather than outsourcing. I’ve heard from industry experts that many payers and providers have grown accustomed to using a patchwork of homegrown point solutions to manage Quality efforts, and it can be difficult to convince so many different stakeholders to give up their fiefdoms for replacement by a vendor. Even when there is an openness to change, purchase processes can be long (18+ months) as solutions are considered by department leadership (in this case Quality) and then go through legal reviews (NDA, BAA, contracts), compliance reviews, data/security/AI reviews, implementation preparation such as data sharing or IT integrations, and potentially additional steps depending on the nature of the solution. On top of this, I’ve heard concerns from non-healthcare organizations about the suboptimal accuracy of today’s AI data abstraction tools – even if the AI is, say, 98% accurate, this may not be good enough to justify using it over humans. Given these hurdles, even the larger health systems that are spending $5M+ annually on Quality efforts may not move quickly to become more efficient in this area.
Perhaps the payer side is somewhat more promising. The significant financial impacts of Star ratings, especially in today’s uniquely challenging environment for Medicare Advantage plans, may make quality improvement a more pressing priority for payers. The largest payers by revenue are much larger than the largest health systems by revenue, and payers’ operating margins tend to be healthier than health systems’, so large payers likely have more funds to deploy in this space compared to providers. However, payers still face similar cultural and administrative barriers to bringing on new vendors and changing processes. Payers may also act cautiously given regulatory uncertainty around the use of AI, as the federal and state regulations have evolved rapidly in recent years, notably including on Medicare Advantage organizations’ deployment of AI in utilization management and prior authorization processes. For those that do adopt AI for this use case, since plans often prioritize a few quality measures at a time, it may be difficult for vendors to sell large contracts supporting more than a few quality measures.
Healthcare tech adoption tends to be slow, and these tools and the regulations surrounding them are still relatively new and evolving. With ambient scribing at the forefront of providers’ current tech priorities, and the incumbency of consultants and sprawling internal Quality organizations in large payers, even the best of AI tools may take time to gain traction.
Thank you to the experts who generously helped me dig into this topic, especially Dave Burianek (MedOrion) and Dr. Chris Williams (Quality Health).