The federal government recently redesigned a digital tool that helps seniors navigate complicated Medicare choices, but consumer advocates say it’s malfunctioning with alarming frequency, offering inaccurate cost estimates and creating chaos in some states during the open enrollment period.

Diane Omdahl, a Medicare consultant in Wisconsin, said she used the tool Friday to research three prescription drug plans for a client. The comparison page, which summarizes total costs, showed all but one of her client’s medications would be covered. When Omdahl clicked on “plan details” to find out which medicine was left out, the plan finder then said all of them were covered.

So she started checking the plans’ websites, and it turns out there are two versions of the same high blood pressure medication. One is covered. The other is not. The difference in price: $2,700 a month.

In Nebraska, miscalculations offered through the new Medicare Plan Finder were so worrisome that the state in late October temporarily shut down a network of about 350 volunteer Medicare advisers for a day because without the tool, narrowing the numerous choices — more than 4,000 Medicare plans are available nationwide — down to three top selections would be nearly impossible.

Days later, EnvisionRxPlus, a prescription drug plan, sent an email to independent insurance brokers nationwide recommending they not use the Medicare Plan Finder because of incorrect estimates on drug prices and patient deductibles. (It’s a warning they had yet to retract some two weeks later.)

Minnesota’s Association of Area Agencies on Aging said in a news release on Nov. 14 that the Medicare Plan Finder “continues to produce flawed results,” including inaccurate premium estimates, incorrect prescription drug costs and inaccurate costs with extra help subsidies.

More than 60 million people use Medicare, which covers those over 65 and the disabled. Users have to pick their plans annually. The current open enrollment period ends Dec. 7. Medicare advisers — as well as advocates for seniors — worry that the full weight of the tool’s inaccuracies will not be felt until the 2020 coverage year begins and seniors head to pharmacies to fill prescriptions or show up for medical appointments. For many Medicare participants, selections made during open enrollment are irreversible.

“Millions of people are going to be absolutely affected,” said Ann Kayrish, senior program manager for Medicare at the National Council on Aging. “And you hate to think about millions of people having the wrong plan. That’s kind of crazy.”

“It’s not like there’s one consistent problem that you can fix and then be addressed,” said David Lipschutz, associate director of the Center for Medicare Advocacy. “It’s really like a game of whack-a-mole.”

The Centers for Medicare and Medicaid Services, the government agency that administers federal health programs, acknowledged in a statement that some problems have been reported but said development of the redesigned tool, which cost about $11 million, was an “iterative” process. The statement said CMS did “extensive consumer testing … to ensure that the information that is displayed is complete, streamlined, understandable, and is in plain language.”

CMS said it previewed the new plan finder in June and began “road-testing” it in July, saying “stakeholder feedback led to enhancements” that were implemented before a public launch on Aug. 27. Updates have continued as users report issues, according to the statement.

CMS said this is the first time since the Medicare Plan Finder was developed in 2005 that the tool has been redesigned. The complete rebuild was necessary, it said, because legacy technology and “proprietary software” couldn’t keep pace with “the needs of today’s digital audience” and because the explosion in options available to seniors under various plans.

Millions of Medicare beneficiaries use the plan finder to reevaluate their insurance choices during open enrollment. The tool drives traffic to Medicare.gov, where users may filter, sort and compare choices. A recent Kaiser Family Foundation analysis found that beneficiaries will have an average of 28 Medicare Advantage plans and 28 prescription drug plans to choose from in 2020. As of Friday, there was no alert on the federal website warning of possible inaccuracies because of the redesign.

Choosing from myriad options can be so befuddling to the typical Medicare participant that states offer help through the federally supported Senior Health Insurance Program, which offers free one-on-one counseling to those eligible for Medicare, their families and caregivers. In Nebraska, for example, seniors choose from 30 Medicare Advantage and 29 Part D prescription drug plans, and the plan finder tool offers a way to quickly sort through dense information.

“You put in the drugs and you put in the pharmacies they want to go to, and it prices them all out,” said Alicia Jones, administrator of the Senior Health Insurance Information Program in the Nebraska Department of Insurance and chair of the national SHIP Steering Committee. Without the plan finder, she said, “the only way to do that is go to 29 different websites and do them individually.”

Jones said she has flagged about 100 errors with the new tool since Oct. 1, some small but others more significant. She noticed it wasn’t importing the correct quantity of drugs, specifically with tubes of medication. It was, she said, giving people way more medicine than needed, “so if you didn’t look at it close, it would give very high prices.”

One day, she said, the numbers for someone she was helping enroll in a prescription drug plan were off by $2,000. The inconsistencies grew so worrisome that Jones said they “just completely stopped” on Oct. 28 to better assess the situation.

Omdahl, the Medicare consultant in Wisconsin, said it recently took her 32 clicks — she counted — to return to the main page after trying to figure out what limits apply for a specific Medicare Advantage plan’s referral and prior authorization requirements.

Every day, each of the 21 Senior LinkAge Line specialists working with Metropolitan Area Agency on Aging, serving the Minneapolis-St. Paul area, have come across at least one error, said Hannah Fox, a Senior LinkAge Line specialist.

“The inaccuracies we’re seeing and are concerned about have to do with the medication copays,” Fox said. “The plan finder is telling us that a medication is on a plan formulary — or not on the plan formulary — and, again, we’re contacting the plan and getting the reverse.”

Julie Roles, spokeswoman for the Minnesota agency, said “specialists know what to be looking for, so they can ID something that might be wrong. But an ordinary person who is just looking by themselves really won’t be alerted to an issue.”

The agency advised beneficiaries who have already enrolled in a plan to verify pricing and other details with the plan provider. It’s also asking CMS “to immediately remedy this situation and to extend the open enrollment period for at least three months once the Plan Finder tool is fixed.”

In New York, Judith Esterquest said she had previously grown so frustrated using the tool to reevaluate her Medicare options that she gave up. This year, she tried again, finding a prescription drug plan that would save her money. Then she decided to double-check with a friend who also happens to be a SHIP adviser, asking, “Am I reading this right?”

“I’m an educated person. I have a PhD. I should be able to figure it out,” she said, adding that she spent “an enormous amount of time” on the National Institutes of Health’s website looking at research protocols when her husband was diagnosed with a rare form of leukemia. “I am comfortable with technology, and I had trouble with the Medicare website.”

According to federal documents used during a national Medicare education meeting, more than 20 million users visited Medicare.gov to use the plan finder in 2017, accounting for about a third of all traffic to the site.

“Plan Finder users often find the process daunting,” the documents said. “Research has shown that users want Plan Finder to provide the following: Simpler results - only information they really need to make a good decision; Personalized information - based on their individual situation. Clarity on out-of-pocket costs.”

The U.S. Government Accountability Office laid out a number of issues with the old plan finder in a July report, including “cumbersome” navigation, confusing instructions, difficult filter and sort functions, and incomplete cost estimates for traditional Medicare that makes it difficult to compare with Medicare Advantage.

“So many of these systems are held together by baling wire and chewing gum,” said Lisa Bari, an independent consultant who previously worked on health IT, interoperability and artificial intelligence at CMS’ innovation center.

The contract for completely redesigning the Medicare Plan Finder was awarded to the software design and engineering company Ad Hoc, which describes itself as being born from “the successful effort to rescue HealthCare.gov after its disastrous initial launch.” The federal online insurance marketplace, a major component of the Affordable Care Act, crashed within hours of launching in 2013, and additional technical problems kept it from functioning properly for weeks.

In a July 2018 blog post, Greg Gershman, Ad Hoc’s chief executive officer and co-founder, announced that his company had won the contract for developing a replacement plan finder as part of the Medicare Coverage Tools initiative. He said “Ad Hoc proposed a product development process that centers around understanding beneficiaries desires when trying to make care decisions, tests our understanding early and often via a minimum viable product, and adds value to the experience with iterative product enhancements.”

Gershman did not respond to requests for comment.

Bari, who did not work on the project, said the issues described to her suggest logic, calculation and coding errors that stem from poor user acceptance testing. That kind of testing, typically the final phase before new software is released, ensures the software can handle real-world tasks and perform up to development specifications. It can be difficult, she said, to find enough people with the necessary skills to do appropriate testing with something as complex as drug coverage.

With a large-scale project that will be used by millions, such as the Medicare Plan Finder, there should be about a month of testing to identify problems, fix them and get everything signed off on, Bari said. Each cohort of features, she added, should go through this process.

“It sounds like they’re just patching stuff now,” she said, “trying to catch the holes that weren’t caught or specified.”