Association Between Medicare Advantage Plan Star Ratings and Enrollment (2024)

ImportanceThe US Centers for Medicare & Medicaid Services publishes star ratings reflecting Medicare Advantage plan quality to inform enrollment decisions.

ObjectiveTo assess the association between publicly reported Medicare Advantage plan quality ratings and enrollment.

Design, Setting, and ParticipantsCross-sectional study of 2011 Medicare Advantage enrollments among 952352 first-time enrollees and 322699 enrollees switching plans.

Main Outcome MeasureAssociation between star ratings and enrollment was modeled using conditional logit regression, controlling for beneficiary and plan characteristics.

ResultsAmong the 952352 included first-time enrollees, a 1-star higher rating was associated with a 9.5 (95% CI, 9.3-9.6) percentage-point increase in likelihood to enroll. The highest rating available to a beneficiary was associated with a 1.9 (95% CI, 1.8-2.1) percentage-point increase in likelihood to enroll. Among the 322699 enrollees switching plans, a 1-star higher rating was associated with a 4.4 (95% CI, 4.2-4.7) percentage-point increase in likelihood to enroll. A rating at least as high as a beneficiary's prior plan was associated with a 6.3 (95% CI, 6.0-6.6) percentage-point increase in likelihood to enroll. Star ratings were less strongly associated with enrollment for black, rural, low-income, and the youngest beneficiaries.

Conclusion and RelevanceMedicare's 5-star rating program for Medicare Advantage is associated with beneficiaries' enrollment decisions.

To inform enrollment decisions and spur improvement in the Medicare Advantage marketplace, the US Centers for Medicare & Medicaid Services (CMS) provides star ratings reflecting Medicare Advantage plan quality.¹ A combined Part C and D overall rating was created in 2011 for Medicare Advantage and prescription drug (MAPD) plans. The star ratings incorporate data from Healthcare Effectiveness Data and Information Set quality measures, Consumer Assessment of Healthcare Providers and Systems surveys, the Health of Seniors survey, and other administrative data.² In 2011, MAPD star ratings ranged from 2.5 to 5 stars. Only 3 MAPD contracts received 5 stars; some were unrated because they were too new or small.

Starting in 2012, Affordable Care Act provisions and a CMS demonstration program, which has garnered some criticism, began to award plans rated 3 stars and above with bonuses and award 5-star plans with year-long open enrollment.³ Owing to recent Medicare Advantage payment rate changes, many insurers are devising strategies to improve their star rating to obtain bonuses.^4-6

While star ratings clearly matter to insurers, it is unclear whether they matter to beneficiaries. One poll showed low beneficiary awareness of the star ratings.⁷ Moreover, another CMS 5-star reporting program, the Nursing Home Compare program, has yielded modest effects on beneficiary choices.^8,9 Research suggests, however, that patients consider quality information when choosing health plans,^10,11 including Medicare Advantage plans.¹²

Assessing plan quality requires significant resources, so it is important to understand whether star ratings are associated with beneficiaries' choices, and if so, to what degree.¹³ No prior study has specifically assessed the association between Medicare Advantage plan star ratings and enrollment.

We conducted a beneficiary-level analysis of 2011 MAPD enrollments in 2 populations of Medicare beneficiaries: those enrolling in Medicare Advantage for the first time and those switching between Medicare Advantage plans. While star ratings existed previously, we focused on 2011 because it was the first year of the overall MAPD rating, the proportion of plans rated was high, and it preceded bonuses or open enrollment extension based on star ratings. We matched beneficiaries to their choice set of plans, which was defined by the county of their address of record on their Medicare Advantage enrollment start date. We used conditional logit regression to assess the association between star ratings and enrollment by beneficiary characteristics, controlling for plan benefits and generosity.

In Medicare Advantage, sponsoring organizations (eg, Aetna, Humana) offer contracts to CMS representing various coverage types (eg, health maintenance organization [HMO], local preferred provider organization [PPO]) for specific geographic service areas. Each contract comprises 1 or more plans of varying generosity. Contracts and plans may be limited to either a single county (or, rarely, a portion thereof) or extend across multiple counties or states.

Beneficiaries choose from a set of plans available to them based on their county; these choice sets contain plans from multiple contracts and sponsoring organizations of varying geographic breadth and overlap. Because CMS calculates star ratings at the contract level, a single star rating applies to all plans within a given contract and across all geographic service areas for that contract.

We derived enrollment and demographic data from CMS's Integrated Data Repository¹⁴; plan premiums and benefit packages, average estimated out-of-pocket costs, and service areas from CMS's Health Plan Management System¹⁵; and star ratings from publicly available files.²

We focused on 2011 non–employer-sponsored MAPD enrollments. We concentrated on MAPD plans rather than Medicare Advantage stand-alone plans without drug benefits because they represent the majority of Medicare Advantage enrollments. To ensure assignment to the correct choice set, we excluded beneficiaries whose address could not be definitively mapped to 1 county, beneficiaries approved for out-of-service area enrollment, and beneficiaries enrolled in plans whose service area did not include their address. To ensure a common choice set in each service area, we excluded Special Needs Plans, Program of All-Inclusive Care for the Elderly (PACE) plans, and employer-sponsored plans. We excluded cost plans because none received overall ratings in 2011 and many had limitations on new enrollment. We excluded beneficiaries whose address of record was outside the United States and whose choice set included fewer than 2 distinct contracts (<1% of otherwise eligible beneficiaries).

First-time enrollees were identified as having their first-ever Medicare Advantage enrollment in 2011.

Switches between plans were defined as a 2011 enrollment in an eligible MAPD plan preceded by a disenrollment from an eligible MAPD plan on or after December 31, 2010. When more than 1 switch occurred, we used the first switch. We then included only switches in which contracts for the prior and subsequent enrollment differed. To ensure switches were volitional, we excluded those in which the applicable disenrollment record lacked a voluntary reason code or when the prior contract was not available at the beneficiary's address at the time of new enrollment (indicating a plan discontinuation or beneficiary move). Because we sought to compare each plan's rating with the prior plan, we excluded beneficiaries whose prior contract was not rated in 2011. Because they could not be chosen by definition, we excluded plans within the prior contract from the beneficiaries' choice sets. We accounted for contract number changes using a crosswalk file.¹⁶

We used conditional logit regression to statistically relate each beneficiary's choice to the characteristics of plans in his/her choice set. Mathematically related to but distinct from the familiar logistic regression, conditional logit models allowed us to assess associations between plan characteristics and choice and whether these associations varied by beneficiary characteristics. In Medicare Advantage, the number, identities, and characteristics of plans within choice sets vary across beneficiaries and each beneficiary chooses only 1 plan from his/her finite choice set. Correspondingly, the data included an observation for every pairing of a beneficiary and each plan in their county-defined choice set; the outcome was a binary indicator of enrollment (1, the plan chosen; 0, all others in the choice set).

Plan Ratings. Our models incorporated both the overall star rating itself (via a continuous variable for the star rating and a binary unrated plan indicator) and a binary variable representing the plan's rating compared with a beneficiary's other plan options. For first-time enrollees, this variable indicated whether a given plan's star rating was the highest available in the choice set, while for beneficiaries switching plans, this variable indicated whether a given plan had a star rating greater than or equal to the beneficiary's prior plan. Thus, we estimated the association between enrollment and a 1-star increase in rating, controlling for how a plan's star rating compared with a beneficiary's other options. In sensitivity analyses, we modeled the star rating itself and the star rating compared with other plan options separately, and confirmed that our analyses were robust to exclusion of plans with no first-time enrollees (n=21 plans) or enrollees switching to them (n=99 plans) (eTables 1-4).

Other Plan Characteristics. We controlled for plan characteristics potentially relevant to enrollment decisions,^17-19 focusing on those available on the Medicare Plan Finder website, including categorical variables for coverage type (HMO, HMO point of service, private fee for service, local PPO, or regional PPO); Part C and D premiums minus rebates; estimated average combined Parts C and D out-of-pocket costs; maximum in-network Part C out-of-pocket costs; and maximum co-payments or co-insurance for in-network primary care, specialist, and outpatient visits; and binary variables for having Part C and D deductibles; and dental, vision, hearing, and prescription gap coverage.

Because preferential enrollment could derive from word of mouth or familiarity,^12,19-21 we also controlled for the local market share of plans' sponsoring organizations. This lagged continuous variable represented a sponsoring organization's percentage of all non-PACE, non–employer-sponsored Medicare Advantage enrollments in a beneficiary's county as of December 1, 2010. The sponsoring organization represents the recognizable brand of a Medicare Advantage plan, and can encompass more than 1 contract in a given area; thus star ratings may vary among a sponsoring organization's contracts in a given market.

Beneficiary Characteristics. Our models accounted for the beneficiary characteristics of age, sex, race/ethnicity, deemed eligibility for low-income subsidy, urban vs rural residence, and US Census region.

In those switching plans, we accounted for health status using 2010 Part C Hierarchical Condition Category scores (community or new enrollee score as appropriate). The scores are calculated using age, sex, low-income status, and 1 year of claims data; the average score is 1 and higher scores correspond to higher predicted health care costs.²² We did not account for health status among first-time enrollees because many were new to Medicare.

In sensitivity analyses (because the number of options can influence the ability to choose well^23-25), we accounted for choice set size (eTables 5-6). Because low-income beneficiaries may have a distinct incentive to choose based on cost or other considerations, we repeated our main analyses excluding these beneficiaries (eTables 7-8). In addition, we repeated analyses including interactions of rating variables with key plan characteristic variables (coverage type, premiums, estimated out-of-pocket costs) (eTables 9-10).

Regression Analyses. Using conditional logit regression, we evaluated the association between enrollment and star ratings, accounting for beneficiary-level characteristics and controlling for plan benefits and generosity and a sponsoring organization's local market share. To estimate differential associations between star ratings and enrollment, we assessed interactions of beneficiary characteristic variables with each of the rating covariates. Because the conditional logit model is nonlinear, we express the results as average marginal effects for interpretability. These represent the absolute percentage-point change in an average beneficiary's likelihood of enrolling in a given plan attributable to a change in star rating.

For accurate calculation of the marginal effect of the continuous star rating variable, it was important that the unrated plan indicator variable also did not vary; therefore, we calculated marginal effects with this variable fixed to indicate a rated plan. Marginal effects for beneficiary subgroups were calculated with the covariate of interest fixed to indicate the subgroup, with other parameters varying as observed in the data. Marginal effects were estimated with the full population; for computational efficiency, we estimated standard errors with a 5% random sample. All tests of statistical significance were 2-sided with a critical threshold of .05. Descriptive analyses were conducted using SAS statistical software version 9.3 (SAS Institute Inc) and regression analyses with Stata version 12 (StataCorp).

Of the 7.6 million beneficiaries with an eligible 2011 MAPD enrollment, the study population was composed of 952352 first-time enrollees and 322699 enrollees switching plans. First-time enrollees were younger than those switching plans; the 2 groups were otherwise similar. For new enrollees, the average choice set contained 16.7 plans, of which 2.0 received 4 or more stars. For those switching plans, the average choice set contained 17.3 plans, of which 1.7 received 4 or more stars (Table 1).

In a descriptive analysis of key characteristics of included plans by star rating, the highest-rated plans more often had higher premiums, while unrated plans more often had higher out-of-pocket costs or were private fee-for-service or local PPO plans (Table 2).

Each panel in the Figure compares the unadjusted distribution of star ratings for all plans available across beneficiaries' choice sets (left column of figure: each beneficiary counted once per plan in choice set) with the distribution of star ratings among actual enrollments (right column: each beneficiary counted only once). In other words, the left column could describe the distribution of star ratings expected if beneficiaries chose randomly among plans available, while the right column presents the distribution of actual enrollments. Most beneficiaries selected plans with 3 to 3.5 stars (first-time enrollees: 62.5%; those switching plans: 67.2%). Highly rated plans comprised a greater proportion of actual enrollments than plans available in choice sets.

In regression analyses among first-time enrollees, higher star ratings were associated with increased likelihood to enroll in a given plan (9.5 [95% CI, 9.3-9.6] percentage points per 1-star increase). The highest rating available to a beneficiary was associated with a 1.9 (95% CI, 1.8-2.1) percentage-point increase in likelihood to enroll (Table 3). Consider a plan in a hypothetical first-time enrollee's choice set of 14 plans (arbitrarily set at sample median), and assume that this plan had a 7.1% likelihood of enrollment (arbitrarily set at sample mean for a 14-plan choice set); this likelihood would have increased to 16.6% if that plan was rated higher by 1 star. Star ratings were less strongly associated with enrollment for the youngest, black, low-income, rural, and Midwestern enrollees.

In regression analyses among those switching plans, higher star ratings were associated with increased likelihood to enroll in a given plan (4.4 [95% CI, 4.2-4.7] percentage points per 1-star increase). A star rating at least as high as a beneficiary's prior plan's rating was associated with a 6.3 (95% CI, 6.0-6.6) percentage-point increase in likelihood to enroll (Table 4). For a hypothetical beneficiary switching plans, consider a plan in his/her choice set of 15 plans (arbitrarily set at sample median) and assume that the plan had a 6.7% likelihood of enrollment (arbitrarily set at sample mean for a 15-plan choice set); this likelihood would have increased to 11.1% if the plan was rated 1-star higher, or to 13.0% if it was rated at least as highly as that beneficiary's prior plan. Star ratings were less strongly associated with enrollment among the youngest, low-income, and rural beneficiaries, and negatively associated among Midwestern beneficiaries. Compared with other races/ethnicities, star ratings were more strongly associated with enrollment for white beneficiaries.

With few exceptions, the interactions of the star ratings with variables for beneficiary characteristics were statistically significant; the direction and significance of the results for marginal effects reported in the main results corresponds with that of the coefficients in eTables 11-12. Results of our sensitivity analyses (eTables 1-10) are qualitatively similar to those presented in the main results.

We observed a positive association between enrollment and publicly reported Medicare Advantage star ratings reflecting plan quality. For first-time enrollees, a plan's star rating itself was more strongly associated than its being the highest-rated plan available, whereas for those switching plans, a plan's star rating itself was less strongly associated than its being rated at least as highly as a prior plan. Although the association was consistently positive, enrollments for key subgroups (eg, black, low-income, rural, and the youngest beneficiaries) were less strongly associated with star ratings.

Given the study's cross-sectional nature, it is important to underscore that one cannot conclusively ascribe the observed preferential enrollment to the star rating program itself. We cannot know whether highly rated plans would have had preferential enrollment for other reasons, absent the star ratings. More highly rated plans may coincidentally be the same ones with more aggressive marketing strategies. Ratings and marketing may not be independent; plans use star ratings in their marketing approach,²⁶ and resources available for both marketing and quality improvement may be related to cumulative enrollment. Adjustment for local market share should ameliorate this source of confounding. Simultaneous temporal changes in the star rating program, attention garnered by the star ratings, and MAPD plan offerings limit the potential to capitalize on a clean natural experiment that might allow for causal inference. Despite the limitations, these findings represent an important first assessment of the relationship between publicly reported Medicare Advantage plan quality and enrollment.

Recognizing these caveats and that this work represents a preliminary investigation, one interpretation of these findings suggests that publicly reported star ratings could be achieving one of their intended purposes of guiding beneficiaries toward higher-quality plans. Consequently, CMS may consider continued evolution of the rating methods to ensure that the quality information conveyed continues to reflect attributes important to both the agency and beneficiaries.²⁷ Additionally, CMS could contemplate communicating information specific to plans or service areas within the larger, contract-level star rating to enhance their relevance. Currently, CMS rewards quality with performance-based bonuses and extended open enrollment for the highest-rated plans. Our finding of a positive association between star ratings and enrollment provides insurance companies with additional justification to pursue higher quality. In addition, as state insurance exchanges are created, our findings argue for careful consideration of communication of quality information when presenting health plan options.

Given highly rated plans' greater cumulative enrollment, additional study is needed to understand how preferential enrollment or disenrollment may influence market share. Incorporation of complaints, appeals, audits, and satisfaction data into star ratings could produce differential disenrollment. It will also be important to assess whether selection of highly rated plans has implications for beneficiaries' subsequent quality of care, health outcomes, or satisfaction, and whether the star rating program results in improvement in these attributes in the Medicare Advantage program as a whole.

The association we observed may conflict with reports of limited awareness of the star ratings,⁷ and prior evidence suggesting that plan information is difficult for beneficiaries to use in decision making.^25,28,29 However, beneficiaries consult other sources (eg, physicians, friends, and family, state health insurance assistance programs, and the plans themselves) for information about Medicare options³⁰; these parties may consult star ratings or other plan quality information, even if beneficiaries themselves do not do so directly.

We recognize that a highly rated plan may not always be the best plan for a beneficiary. While not the primary focus, our analyses also revealed some preference toward plans with lower premiums or more liberal benefits (eg, PPOs, gap coverage, no deductibles, lower out-of-pocket costs). Further underscoring this point are the lesser associations between rating and enrollment observed among rural and low-income beneficiaries for whom clinician proximity or premiums and other costs may be especially salient. Nonetheless, as decisions to enroll in Medicare Advantage are also associated with sociodemographic characteristics,³¹ these groups may represent opportunities for outreach regarding selection of high-quality plans. Research has demonstrated that quality is one of many attributes affecting health plan decisions^10,19; further study is needed to understand the relative contributions of cost, generosity, and quality considerations for Medicare Advantage choices.

Our study has other limitations. The range of the star ratings is narrow; therefore, we cannot conclusively ascertain whether beneficiaries seek out highly rated plans or simply eschew the lowest-rated or unrated plans. However, either could be interpreted as informed by quality. We also presumed that decisions to enroll in an eligible MAPD plan at all were made independent of consideration of the star ratings and characteristics of plans available, which may not be true if a dearth of high-quality plans in a market prompted beneficiaries to remain with their prior plan or traditional Medicare. Exclusion of employer-sponsored, PACE, and Special Needs Plans limits generalizability to beneficiaries eligible for these options. Our focus on MAPD limits applicability to beneficiaries for whom traditional Medicare or Medicare Advantage stand-alone plans are attractive (eg, those with credible drug coverage). We cannot directly assess the associations between enrollment and beneficiaries' relationships with clinicians or plans' local reputations.³² To the extent that coverage type (eg, HMO vs PPO) is a proxy for network inclusivity, and an organization's local market share could reflect availability of clinicians preferred within the market or due to word-of-mouth recommendations, these may be represented in our analysis. In addition, the beneficiary-level data used are not generally available to researchers outside CMS, limiting replicability; these data enable an analytic approach not otherwise possible.

We found a positive association between CMS's 5-star Medicare Advantage quality ratings and enrollment. Bolstering the business case for quality in the Medicare Advantage market, these findings may provide firms with additional incentive to cultivate higher quality, CMS with justification to continue to advance public reporting, and policy makers with a rationale to pursue quality reporting in other health insurance markets.