Study on Five Star Ratings

McKnight’s had an article on Medicare’s 5 Star Rating.  The level of quality care in nursing homes is in erratic and inconsistent, as more than a quarter of
5-star-rated facilities have slid and fallen to the lowest rating, while more than two-fifths have climbed from the bottom to the top, achieving the highest rating in 11 years. The study published in the Journal of the American Medical Directors Association found that “performance mobility” is a key feature of the CMS Care Compare star ratings system.

“We were really surprised by how frequently [providers] move,” lead author Tory Hogan, PhD, associate professor of health services management and policy at Ohio State University, told McKnight’s Long-Term Care News. “The organizational performance ebbs and flows. Much of that is based on real decisions that teammates are making to work toward a common goal. But you also really can’t rest on your laurels.”

Hogan and her fellow researchers sought to determine and understand how easily nursing homes can change their quality star ratings. These quality star measures are used by health systems, policymakers, and consumers to determine the level of quality care their loved one will receive. This study examined the overall star ratings without considering the degree of movement in each of the three categories —staffing, health inspections, and quality measures —that affected the overall ratings.

Hogan set out to establish a baseline for analyzing the level of control nursing homes have over their Care Compare systems. Despite the limits that CMS places on the number of nursing homes that can be in each quality tier, most facilities experience a change in their ratings over time. In all, 40.32% of 1-star nursing
homes later received a 5-star rating, while 27.97% of 5-star nursing homes later received a 1-star rating for at least one month. Nursing homes can rapidly shift from one quality tier to another.

The team took data from more than two million monthly ratings for 16,568 unique Medicare-certified nursing homes and found that moving between the upper and lower extremes of five stars took an average of 54 months while moving from just one to three stars took an average of 22 months. Hogan explains how this study could be helpful to nursing home managers looking to improve their rating through “true quality improvements.”

Although many data analysts focus on improving star ratings, the JAMDA study did not investigate what causes changes in those ratings. Other researchers have demonstrated the impact of ownership changes on quality and staffing metrics; however, experts argue that these factors do not fully account for the shifts observed in overall star ratings. Rachel Prusynski, DPT, PhD, an assistant professor in the Department of Rehabilitation Medicine at the University of
Washington, has recently co-authored two papers regarding quality effects on nursing home sales.

Prusynski told McKinights that changes in ownership are associated with lower and worse quality measures, but the “huge caveat” to all of this research is that the effects are minimal. Prusynski does not believe that you can attribute a large portion of the declines in overall star ratings to ownership; instead, there is more behind a change in overall star rating that can be pinpointed to any operating change or sale. Hogan is seeking to continue this research and is
requesting grants to further investigate what sustains nursing homes in the high-performance category and which variables are most significant in influencing star mobility.

She adds, “It is so complicated, and from a policy perspective, I think that we can do better to make it clearer to patients about what the stars mean.”

Hogan and her co-authors suggest that the star ratings system may be “hypersensitive” to small changes in the underlying metrics used to calculate them. Additionally, they also note that changes in CMS policy, including incentives and reporting requirements, may contribute to this shift. They identified two main theories to explain the movement. One theory is that nursing homes have the potential to “teach to the test” without making actual changes in care delivery and can improve ratings by focusing on the metrics rather than actual Care, which in turn neglects the residents.

Another theory is that the data may reflect genuine efforts by these nursing homes to improve the level of Care, as nursing homes often fluctuate between close
ratings. Spencer Blackman of StarPRO, a ratings dashboard software company, has provided additional insight into how quickly they can make improvements. Similar to consumers, providers must pay close attention to improving their understanding of the weight different metrics carry in determining the overall star ratings.

Staffing changes can quickly affect star ratings, but primarily for facilities at the very low or very high end of the ratings. Staffing is challenging to improve due to the high costs and hiring challenges that facilities face when attempting to enhance their staffing levels.