Makary’s hand-picked panels lack diversity of opinion, robust evidence reviews, and credibility
The recent controversies about the safety of antidepressants during pregnancy and hormone therapy for menopause were heightened by how decisions are being made. Earlier this month, HHS officials announced that boxed warnings would be removed from hormone therapy drugs to treat symptoms of menopause, several months after a group of 12 experts handpicked by FDA Commissioner Marty Makary, MD, MPH, recommended removal of the warnings.
Makary asserted that traditional FDA advisory committees — which have historically been convened before making such a change — are outdated because they are “bureaucratic, long, often conflicted, and very expensive.” He said he plans to instead rely more on his expert panels, which he described as more “spontaneous” but still “robust.”
There are many differences between advisory committees and the “expert panels” held this year. As someone who has participated in hundreds of FDA advisory committee meetings and watched several of these expert panels, I believe Makary is missing the point and undermining the credibility of FDA decisions.
What Is the Purpose of Advisory Committees?
FDA advisory committee meetings are scheduled for drugs, devices, or tobacco products when there is conflicting evidence or a contentious difference of opinion within the agency or between the agency’s point of view and other interested parties (such as the company that makes the product).
Typically, the meeting focuses on whether a new drug or device merits FDA approval, whether there is persuasive evidence that an approved product is less safe or effective than FDA previously concluded, or if a type of product may require more/less rigorous regulatory standards than currently required.
Most meetings are around 8 hours long (although this varies): approximately 2 hours for the company to provide evidence on safety and efficacy; 1-2 hours for FDA scientists to express their views of the evidence; 1 hour for an “open public hearing” for patients, family members, or health professionals or organizations to express their views; and 3 hours of advisory committee questions, discussion, and voting. A full meeting transcript is made publicly available afterwards and typically the meetings are all livestreamed now as well.
The members typically include about a dozen medical experts (mostly clinicians) who do not work for a federal agency; at least one patient or consumer representative; and one industry representative (non-voting). Many serve on the committee for several years after being nominated and selected based on their medical specialty; at each meeting, a few additional experts are added because of their relevant expertise. FDA staff are not on the committee, but NIH or other agencies may have one or two non-voting members with relevant expertise.
Weeks before the meeting, advisory committee members are provided with the company’s description of their evidence (often 100+ pages long) and the FDA’s detailed critique of that evidence. These documents, the agenda, names and affiliations of committee members, and information about possible conflicts of interest are made publicly available on FDA.gov 48 hours in advance and stay online for years afterwards.
How Are Expert Panels Different?
Compare that to a 2-hour expert panel meeting, comprised of approximately 8 to 12 experts selected by the FDA, each speaking for a few minutes and answering softball questions asked by the FDA commissioner or other official. The list of speakers is not necessarily released in advance (perhaps in part due to the “spontaneous” nature of the panels), no written documentation regarding the evidence has been provided to the public, and the panel members’ presentations appear mostly consistent with the commissioner’s opinion. The public can attend in person or remotely but can’t ask questions or express their views.
One unfortunate trait that the advisory committees and expert panels have in common is that some members are less “expert” than others. Many are clinicians with limited understanding of statistics or the implications of the FDA regulatory process. Meanwhile, the academic physicians participating do not acknowledge that many clinicians who write these prescriptions may rely on drug and device representatives for information rather than regularly reading studies in medical journals or closely studying very long drug labels.
Another unfortunate trait they have in common is conflicts of interest. Financial ties to industry are discouraged but still exist and are not always disclosed. Both panels include clinicians whose quoted statements at these meetings may influence their salaries by generating positive or negative publicity. An additional possible conflict unique to researchers on expert panels is the assumption that pleasing HHS officials could improve their chances of future federal funding or recognition.
Shortcomings of Advisory Committees
To be fair, during advisory committee meetings, some committee members appear to have not read most of the meeting materials provided in advance, and it sometimes seems the FDA has “stacked the deck” by cherry-picking committee members. Also, the votes may at times be unanimous even when numerous members express great concerns about the product. For example, I once heard a committee member share substantial doubts and then say, “I certainly would not want my mother to take it” — but he voted in favor of the product anyway.
In addition to companies’ hours-long presentations, many recruit and train public comment speakers. Most are patients desperate for a cure who may not realize the product performed no better than placebo. Their stories are often tragic and compelling, and numerous advisory committee members have told me they voted in favor of a product of questionable efficacy because they didn’t want to seem indifferent to the patients’ plight.
As President Biden’s FDA commissioner, Robert Califf, MD, openly expressed concerns about FDA advisory committee meetings. In a meeting with several nonprofit leaders, he made it clear to us that he did not like it when FDA was criticized for approving a product that the advisory committee had voted against. At the end of the day, the advisory committees serve as just that: advisors. They have not been a rubber-stamp; although in the future, FDA may try to take them in a different direction. But the value of this public forum for discussion and transparency cannot be overstated.
Don’t Let Perfect Be the Enemy of Good
Advisory committees are not perfect, but I’ve never seen one that lacks a diversity of opinion. FDA scientists almost always raise important concerns that help committee members consider the risks and benefits of products and strengths and weaknesses of the research evidence.
In contrast, during the expert panel on hormones for menopause, the commissioner and the panel cited studies that, in several cases, were outdated, based on rodents rather than women, or included too few patients or were too short-term to provide meaningful evidence. There were no public speakers and no panel members willing to challenge those flawed research citations, so reporters under tight deadlines had little opportunity to include expert opinions that conflicted with the preordained message of the meeting.
Important FDA decisions deserve scientific scrutiny, challenging questions, differences of opinion, and nuanced summaries. These have, to date, been lacking in the FDA expert panels, making critical information less available to patients and their physicians.
Diana Zuckerman, PhD, is president of the National Center for Health Research in Washington, D.C. She is a former Congressional committee staffer who investigated FDA policies.
You can read the original MedPage Today article here.