By KIP SULLIVAN, JD 
Egged on by the Medicare Payment Advisory Commission (MedPAC), Congress has imposed multiple pay-for-performance (P4P) schemes on the fee-for-service Medicare program. MedPAC recommended most of these schemes between 2003 and 2008, and Congress subsequently imposed them on Medicare, primarily via the Affordable Care Act (ACA) of 2010 and the Medicare Access and CHIP Reauthorization Act (MACRA) of 2015.
MedPAC’s five-year P4P binge began with the endorsement of the general concept of P4P at all levels – hospital, clinic, and individual physician – in a series of reports to Congress in 2003, 2004, and 2005. This was followed by endorsements of vaguely described iterations of P4P, notably the “accountable care organization” in 2006 [1], punishment of hospitals for “excess” readmissions in 2007 [2], the “medical home” in 2008 and the “bundled payment” in 2008. None of these proposals were backed up by anything resembling evidence.
Congress endorsed all these schemes without asking for evidence or further details. Congress dealt with the vagueness of, and lack of evidence supporting, MedPAC’s proposals simply by ordering CMS to figure out how to make them work. CMS staff added a few more details to these proposals in the regulations they drafted, but the details were petty and arbitrarily adopted (how many primary doctors had to be in an ACO, how many patients had to sit on the advisory committee of a “patient-centered medical home,” how many days had to expire between a discharge and an admission to constitute a “readmission,” etc.).
New rule, new culture
This process – invention of nebulous P4P schemes by MedPAC, unquestioning endorsement by Congress, and clumsy implementation by CMS – is not working. Every one of the proposals listed above has failed to cut costs (with the possible exception of bundled payments for hip and knee replacements) and may be doing more harm than good to patients. These proposals are failing for an obvious reason – MedPAC and Congress subscribe to the belief that health policies do not need to be tested for effectiveness and safety before they are implemented. In their view, mere opinion suffices.
This has to stop. In this two-part essay I argue for a new rule: MedPAC shall not propose, and Congress shall not authorize, any program that has not been shown by rigorously conducted experiments to be effective at lowering cost without harming patients, improving quality, or both. This will require a culture change at MedPAC. Since its formation in 1997, MedPAC has taken the attitude that it does not have to provide any evidence for its proposals, and it does not have think through its proposals in enough detail to be tested. Over the last two decades MedPAC has demonstrated repeatedly that it believes merely opining about a poorly described solution is sufficient to discharge its obligation to Congress, taxpayers, and Medicare enrollees.
I will illustrate these attitudes and the problems they create by reviewing the genesis and performance of the Hospital Readmissions Reduction Program (HRRP), in particular the punishment of hospitals for “excess readmissions” of patients diagnosed with congestive heart failure (CHF). In this first installment, I will document the following conclusions:
- In 2007, MedPAC proposed an inexcusably vague version of the HRRP (the worst defects were MedPAC’s failure to determine whether a readmission could be determined to be “related” to a prior admission with claims data, whether a hospital’s “excess readmissions” rate could be accurately risk adjusted, and what it costs hospitals to reduce readmissions), and they did so on the basis of no evidence;
- Congress simply accepted MedPAC’s vague, evidence-free proposal, and in 2010 authorized CMS to figure out how to make it work; and
- CMS implemented the HRRP in October 2012 in an even worse form than the original MedPAC version, a form that raised the odds that sicker and poorer patients would be harmed.
In Part II, I will review the research that sought to determine the impact of the HRRP on readmissions and mortality of CHF patients. I will draw two conclusions: (1) The evidence suggests that the HRRP has harmed some CHF patients and possibly patients without a CHF diagnosis as well; (2) CMS should have conducted a controlled trial of the HRRP before inflicting it on thousands of hospitals and tens of thousands of patients.
Evidence-free diagnosis, evidence-free solution
The notion that hospital quality can be measured accurately with claims data (as opposed to claims data plus medical records data) emerged in the late 1980s. By the early 1990s, death during or shortly after heart surgery was widely treated as a quality measure despite the crudeness with which mortality rates were calculated with claims-only data. (See my discussion of the inaccurate New York coronary artery bypass graft [CABG] mortality report card here. This report card is grossly inaccurate even though it relies on medical records data as well as claims data.) By the late 1990s, conventional wisdom among health policy “thought leaders” had elevated readmission rates to the same status as mortality rates – it was a legitimate quality measure that could be safely administered to thousands of hospitals and millions of patients.
However, the conventional wisdom remained unproven by the early 2000s, and remains unproven to this day. “[T]he link between early readmission and quality of care is still uncertain,” is how five experts put it in a 2004 article. “[T]he association between the HRRP implementation and mortality is not known,” reported Gupta et al. in January 2018. “[T]he suitability of early re-hospitalization as a correct target for good medical practice is highly debated,” said Palazzouli et al. in a September 2018 article.
The absence of evidence did not stop MedPAC from endorsing the emerging folklore that hospitals can reduce readmissions by spending more money on “care management services” (reconciling drug prescriptions prior to discharge, for example) and, therefore, readmission rates should be treated as quality indicators. In Chapter 5 of its June 2007 report to Congress, MedPAC recommended that CMS punish hospitals for “avoidable” readmissions. In that chapter MedPAC failed to offer evidence supporting their diagnosis (that unnecessary readmissions are occurring at very high rates), and they made no effort to demonstrate that their solution (measuring readmissions with claims data and punishing hospitals with rates alleged to be above average) would work.
MedPAC began its diagnosis of the alleged excess-readmission problem with a statement that was based on solid evidence: 18 percent of all Medicare patients discharged from a hospital are readmitted within 30 days [3]. Then they made the astonishing claim that 76 percent of those readmissions “were … potentially preventable” (pp. 107-108). They based this claim on a study they conducted using software recently developed and trademarked by 3M [4].
MedPAC claimed that 3M’s software, the Potentially Preventable Readmissions Classification System, can identify “potentially preventable readmissions” (PPRs) simply by feeding it claims data. MedPAC offered this brief description of 3M’s method: “3M combed through all permutations of diagnoses for an initial stay and for a readmission and evaluated the likelihood that a given readmission diagnosis was related to the first admission and, therefore, was potentially preventable.” That was it. What does “related” mean? Who at 3M determined that diagnosis x is “related” to diagnosis y? What criteria did those unknown people use to determine “relatedness”? MedPAC offered no clue. They offered only this cryptic caveat: “Policymakers will need to consider the importance of distinguishing the clinical underpinning of readmissions” (p. 107). In other words, “We don’t know if 3M’s program or any other program can determine the ‘clinical relatedness’ of one admission to another, but even though that assumption has to be true for our recommendation to work, we don’t intend to document that assumption. Someone else will have to do it” [5].
Despite the caveat, MedPAC treated the PPRs identified by 3M’s algorithm as always preventable [6].
MedPAC was equally cavalier about another essential assumption: That CMS can accurately adjust readmission rates (a process known as “risk adjustment”) to reflect factors beyond hospital control, again using only claims data (which is limited to the person’s age, sex and a few diagnoses). MedPAC dealt with this important and complex issue with a single sentence: “In comparing hospitals, we need to adjust for differences in the types of cases and the severity level of patients” (p. 110) [7].
Crude risk adjustment always punishes providers who treat sicker patients and rewards those who treat healthier patients, and even the best risk adjusters are crude. We have that on authority from none other than MedPAC. In its June 2014 report to Congress, MedPAC demonstrated how inaccurate CMS’s risk adjuster is (that risk adjuster is the most studied, and probably the most accurate, of all risk adjusters ever invented) and how severely it punishes providers who treat sicker patients. (For a discussion of that MedPAC report, scroll about halfway through a comment I posted here a year ago.)
Having established to their satisfaction that they had dealt adequately with two very complex problems – accurate identification of unnecessary readmissions, and accurate risk adjustment of readmission rates – MedPAC proceeded to construct a bar graph allegedly showing “wide variation” in (crudely risk-adjusted) CHF readmission rates among hospitals. With no evidence, MedPAC then issued its pre-ordained, evidence-free conclusion: Not only could hospitals with above-average CHF readmission rates lower them, even hospitals with below-average rates could lower their rates even further.
Free-lunch syndrome
The third inexcusable error MedPAC committed in its 2007 report was to make no effort to estimate what it would cost hospitals to reduce readmissions for CHF, chronic obstructive pulmonary disease, or CABG surgery, the three types of readmissions they recommended CMS start with, or any other type of readmission. Here was the sum total of MedPAC’s wisdom on this subject: “A related issue that is beyond the scope of this chapter is the lack of funding for care management services…. Perhaps once experience is gained in how much hospitals can improve and what resources are needed to achieve improvement, policymakers can consider the need for any explicit financing for care management services as a complement to a change in readmission payment policy” (p. 115). What possible excuse is there for admitting that MedPAC’s proposal will impose new costs on hospitals but not asking whether hospitals can afford the new costs and whether the new costs will outweigh the savings allegedly to be achieved from reduced readmissions? [8]
The three defects in MedPAC’s proposal I have discussed here – no documentation of their claim that it’s possible to determine the “relatedness” of one admission to another, no documentation of their claim that it’s possible to risk-adjust readmission rates accurately, and no discussion at all of the costs their proposal would impose on hospitals – guaranteed that their proposal would harm hospitals that treat an above-average share of sicker and poorer patients. Experts both on and off the commission tried to warn pro-HRRP commissioners and staff this would happen. In fact, it did happen. I will discuss the research documenting that statement in Part II of this series.
Congress rubber stamps MedPAC
You might think the smart people who staff the congressional health committees (or perhaps a few members of those committees) would catch all these errors and either reject MedPAC’s proposal or send it back for more details (and maybe a smidgeon of supporting evidence). In your dreams. Congress not only accepted MedPAC’s vague 2007 version of the HRRP, the version they endorsed was even vaguer than MedPAC’s.
Congress endorsed MedPAC’s vague recommendation in Section 3590 of the Affordable Care Act. That section gave CMS carte blanche to set up the HRRP program any way they wanted. Section 3590 left it up to CMS to determine virtually every essential feature of the program, including the types of readmissions that would be measured (CHF, etc.), the time period within which an admission would be judged to be a readmission (15 days, 30 days?), how “relatedness” would be determined, how readmission rates would be risk adjusted, and the minimum number of readmissions needed to subject a hospital to punishment (CMS ultimately chose 25, believe it or not). Here are the only instructions Congress gave to CMS in Section 3590:
- penalized readmissions could not include those “unrelated” to the index admission;
- penalties would apply to payments for all Medicare patients (not just those who were readmitted “unnecessarily”);
- penalties couldn’t exceed a certain amount (the ceiling was to rise to 3 percent by 2015 and stay there); and
- the program had to begin on or after October 1, 2012.
CMS then proceeded to set up a version of the HRRP that was even worse than the one MedPAC had proposed. CMS didn’t even bother to pretend they needed software like 3M’s PPR package to determine the “relatedness” of one admission to another. CMS simply treated all unplanned admissions within 30 days of a discharge as “readmissions.” If, for example, a patient diagnosed with CHF was discharged on January 1 and was readmitted on January 25 with the flu, that was a “readmission” the hospital should have prevented.
The HRRP went into effect on October 1, 2012. It was not a pilot test. It exposed approximately 5,000 acute care hospitals to punishment if their readmission rate of patients diagnosed with heart attack, CHF or pneumonia exceeded the national average (hospitals below the national average are neither punished nor rewarded). In Part II of this series I will review the research on the effect the HRRP has had on CHF readmissions and mortality. I will compare MedPAC’s June 2018 report to Congress, which concluded the HRRP has reduced readmissions without harming patients, with several peer-reviewed articles suggesting the HRRP has harmed patients.
Footnotes
[1] Several years ago I would have cited the transcript of MedPAC’s November 2006 meeting to document my statement that MedPAC endorsed the ACO concept in 2006. I can’t do that anymore because some time in late 2016 or early 2017, MedPAC removed all transcripts of meetings held prior to September 2010 from its website.
[2] MedPAC laid out their inadequate rationale for the HRRP in Chapter 5 of their June 2007 report to Congress, but did not actually vote on recommending it. That vote occurred later. It was reported in Chapter 4 of their June 2008 report. The 2008 chapter merely summarized the argument MedPAC had made in their 2007 report. For that reason, I have discussed only the 2007 report in this article.
[3] I refer to the excess-readmission problem as the “alleged” problem not because I deny that some readmissions are unnecessary, but because the hullaballoo about readmissions creates the impression that underuse of hospital services is a trivial or non-existent problem. Data from the Organization for Economic Cooperation and Development and other sources suggest that the net rate of overuse and underuse of hospital services in the US may be negative, that is, that hospital services are underused.
[4] MedPAC’s report on its own experiment running claims data through 3M’s PPR algorithm was the only evidence MedPAC introduced to support its argument for its HRRP proposal. MedPAC also cited a study that concluded that 85 percent of all 30-day readmissions following coronary artery bypass graft (CABG) surgery in New York were due to “complications directly related to the CABG.” However, MedPAC neglected to inform its readers that the authors of that study, Edward Hannan et al., warned against using CABG readmissions as a quality measure because so many of the complications that led to CABG readmissions were beyond the hospitals’ control.
[5] MedPAC did offer three examples of “permutations of diagnoses” that 3M determined were “related” and two that were “unrelated.” 3M declared, for example, that an “admission for diabetes following a discharge for AMI” (heart attack) to be “potentially preventable,” whereas an “admission for trauma following a discharge for AMI” would not be a PPR (box p. 109). The rationale for expecting hospitals to respond to any and all signs of diabetes during an admission for a heart attack was not explained.
An article published a year later by four 3M employees and three colleagues offered a much more detailed description of 3M’s algorithm, including a statement that they divided each diagnosis into four “severity” levels depending on the patient’s health. But the authors offered no assurance that this division achieved accurate risk-adjustment of readmission rates. All the authors said about risk adjustment was, “Risk adjustment … is … necessary in order to create fair evaluations of readmission rates.” (p. 90) The authors offered examples of diagnosis pairings that they thought did and did not constitute PPVs. The authors stated, for example, that “a readmission for a urinary tract infection within 7 days following an admission for major bowel surgery” would constitute a PPV. So too would an admission for pneumonia following CABG surgery (no time limit was mentioned). The first example seems obvious, the second one does not, especially if the second admission followed the first by more than a month or two.
[6] Isn’t “potentially preventable” redundant, like “possibly feasible” and “conceivably possible”? What, besides confusion and the opportunity to escape accountability, is added by “potentially,” “possibly,” and “conceivably”? Readmissions are either preventable or they’re not.
[7] The only other comment MedPAC made about risk adjustment in Chapter 5 of their 2007 report was a useless suggestion at the end that “policymakers” might want to consider allowing hospitals to declare certain patients “nonadherent” and thereby remove them from the readmission pool.
[8] MedPAC was as oblivious to the costs their proposal would impose on CMS as they were to the costs it would impose on hospitals. MedPAC made no mention of CMS’s costs in their June 2007 report. It did occur to the commission to mention this issue one year later in their June 2008 report to Congress. In Chapter 4 of that report, in which MedPAC recommended bundled payments and punishing hospitals for “excess” readmissions, the commission noted these proposals “will undoubtedly require more administrative resources for CMS” (p. 85) and, “Given the complexity and breadth of these demands on CMS, the Congress may wish to consider making a special appropriation to CMS, much as it did when it passed the Medicare Prescription Drug, Improvement, and Modernization Act of 2003” (pp. 92-93). Far be it from MedPAC to make even a wild guess as to what it would cost CMS to administer their complex proposals.
Kip Sullivan is a member of the Health Care for All MN advisory board, and of MN Physicians for a National Health Program.