Covered California has awarded the 2016 quality star ratings to health plans based on just two questions from a questionnaire surveying consumer’s experience in 2014. In addition, the quality ratings were based on a completed sample size of approximately 100 respondents out of 1,000 surveys sent out.
Covered California star rating based on thin data
I am firm believer that a professionally created survey with an adequate sample size will yield meaningful results. While the results of the Covered California consumer survey generated results, I question the wisdom of using such thin data to apply quality ratings to health plans when the consumer experience is over a year old. (Health Plan Quality Reporting Open Enrollment 2016 can be viewed at end of post.)
Consumer experience from 2014
Dr. Lance Lang, Chief Medical Officer for Covered California, reviewed how the survey was conducted and the lack of responses at the Covered California Board meeting on October, 8, 2015. (His comments can be viewed on a recording of the meeting. He speaks between minutes 20 and 27 of the video.) Dr. Lange noted that the ten question survey was sent out during the winter of 2015 and surveyed consumer’s experience for the 2014 enrollment year. Of 1,000 surveys sent out, only 21% were returned.
Responses to two questions determine Covered California star ratings
Because of the lack of response, and I am assuming because not all the questions were answered on each returned survey, they could only use two of ten questions to determine a star quality rating. Of the two questions used, one dealt with the health plan and the other for the quality of the health care services received. The final sample size to determine the star ratings that are attached to each carrier on the 2016 Shop and Compare Tool is approximately 100. From over one million people enrolled in health plans with Covered California, 100 consumers, based on two questions, are determining the star ratings for all of the health plans.
Dr. Lang’s comments on Covered California star ratings begin on minute 20:00 of the Board meeting recording.
Narrow slice of the picture
Dr. Lang noted that the two questions for which they had usable data were very good indicators for the overall performance of the plan and the consumer’s satisfaction. Executive Director Peter Lee also mentioned that the star ratings are only a “…narrow slice of the picture.” I agree with Mr. Lee that the small sample size and the use of only two questions is only a sliver of the big picture of a health plan’s performance. Mr. Lee also noted that California was the first to use star ratings based on consumer experience. But there may be good reasons why not to use consumer experience from the first year of the Affordable Care Act’s Marketplace exchange (Covered California) to label health plans with a quality rating score.
Health insurance was a mess in 2014
2014, the first year of the new ACA health plans was a mess, to put it mildly. Preferred Provider Organization (PPO) plans got hit hard. There were problems with accurately determining which doctors were in the new narrow networks. The billing systems of many carriers were not fully functional. Covered California had their own problems with getting enrollment data to the health plans. There were a number of issues with the health plans that had nothing to do with the performance of the insurance or the health care delivered. In short, many consumers, and possibly those who participated in the survey, had a very nasty experience and a dim view of the health plans and the ACA overall.
Poor communication soured consumer experience
While I would like to think the consumer survey was able to adjust for the bias of the negative 2014 experience, such a consideration was not noted in Dr. Lang’s comments. In other words, did the respondents to the survey conflate any negative experience because of IT problems with billing, enrollment, or provider confusion, with the actual performance of the health plan and delivery of health care services? Plus, what role did the doctor’s office play in providing wrong information to consumers about the new individual and family plans they accepted. Many doctors were turning away consumers because they decided they didn’t want to accept the new plans even though they were listed as in-network providers.
Kaiser members for life
Of the respondents to the consumer survey, how many of them had unrealistic expectations about their health plans? Did they even know how the health plans worked? The health plans that scored the highest were Kaiser, Sharp, and Western Health Advantage. All of these plans are HMO’s. My anecdotal experience is that all of these plans attracted consumers who were already members or had been members in the past. I think this is particularly the case with Kaiser. People either love Kaiser or they are less than enthusiastic about Kaiser.
Do you know what to expect from health insurance?
Kaiser members know and understand how the system works. And it works well for those consumers who continue to enroll in Kaiser health plans. Because they know what to expect, Kaiser members are less likely to have a negative experience. Contrast this to a family who enrolled in a PPO plan who thought their physician group was in-network only to find the doctors decided to opt out of the health plan’s PPO network. Or being inadvertently sent to an out-of-network specialist and getting a large invoice for the services.
Did Molina get a bum one star rap?
It’s interesting that while Molina gleaned only a one star rating from the Covered California consumer survey, the Molina Medi-Cal plan received two out of three stars in Sacramento County on many measurements for health care. How can the consumer experience be so drastically different within the same health plan company? Similar to Kaiser, the members involved in the Molina Medi-Cal plan already knew how to navigate the system. It’s possible that Molina, like many of the other health plans, just couldn’t educate their new members in an efficient manner for the members to properly access services without issues or problems.
Which metal tier level did the consumers have? Bronze?
But can we really extrapolate quality ratings from the experience of consumers across four different metal tier levels, ten different health plans, and nineteen different regions? How many of the respondents actually were enrolled in a Molina health plan? What was the bias of those who returned the survey? Were they angry about the first year mess of the ACA? How many of the consumers rating health plans low because they enrolled in a Bronze plan where virtually nothing is covered? Were consumers led to have unrealistic expectations about their new health insurance? Of the respondents, how many were enrolled in health insurance for the first time in their lives?
When will doctor’s be rated?
The star ratings don’t represent California’s unique separation between hospitals and doctors. Because hospitals can’t hire doctors in California, patients receive one bill from the hospital or outpatient facility and another from the doctor. PPO health plans have no control over this. The separate bills come as a shock to many consumers who in turn blame the health plan. There is a learning curve on how to use health insurance to maximize benefits and reduce costs. Many consumers did have a negative experience because the health plan worked the way it should and they got hit with a big bill or denied an excluded health care service. This is an education issue, not a health plan performance issue.
Health plans improved tremendously in 2015
I am no big fan of the health insurance industry. But I have to admit that most plans have made tremendous improvements from 2014 across the board from member communication, billing, and education. Frankly, I think it is unfair to label a health plan with a quality star rating based on the first year of a nationwide experiment in realigning the health insurance industry.
Know the health plan for best experience
The best strategy to make sure a prospective health plan will meet your expectations is to do your homework. You should confirm your favorite physicians and hospitals are in-network. Review the drug formulary to see if your medications are covered and how much they will cost. Review the Evidence of Coverage if you have any special or unique situations to learn how the health plan will handle the health care. In short, you have to perform you due diligence in vetting a health plan. (Most Certified Insurance Agents will help you with these tasks.)
Never rely solely on star ratings
Never rely on a star rating based on 100 consumers who were probably still mad that they had to wait on the phone for over three hours just to talk to a Covered California service representative to get enrolled. I would conjecture that there are some Yelp reviews that are a better indicator of a service than these star ratings. Oh yeah, and when is Covered California go to assign a star rating to themselves. If they do, I can guarantee you it won’t be on their 2014 performance, but on their much improved customer service of 2015…just like the health plans should be rated as well.
Download for Executive Director Report, October 2015, that includes Health Plan Quality Reporting Open Enrollment 2016
- [wpfilebase tag=fileurl id=922 linktext=’ PPT – Executive Directors Report October 8, 2015′ /]
