April 7, 2025
[Docket No. FDA-2024-D-4245]
Thank you for the opportunity to comment on the FDA’s Draft Guidance for the Study of Sex Differences in the Clinical Evaluation of Medical Products. The National Center for Health Research has conducted research on this issue for decades and our comments today build on the findings of that research.
We strongly support most of the FDA’s proposed guidance, but we note that the agency has attempted to improve the representation of women, older Americans, and racial and ethnic minorities for many years but has fallen short. We have scrutinized this proposed guidance and are providing several suggestions that we respectfully encourage the FDA to implement.
Meaningful Representation of Males and Females and Major Demographic Subgroups
First and foremost, the agency should not focus entirely on increasing females but rather focus on ensuring males and females be substantially represented in all studies of medical products that will be used by males and females. Just as women have historically been underrepresented in studies of treatments for heart disease and several other illnesses, males have been underrepresented in studies of weight loss products and implants that are used by both males and females, for example. We therefore agree with the statement that “For diseases or conditions that can occur in both females and males but rarely occur in one of the sexes in actuality, avoid arbitrary exclusion criteria that prohibit participation based on sex.” It is not sufficient that males and females in the studies be represented in proportion to their likely use of the product; instead, meaningful representation requires that sufficient numbers of patients of the underrepresented sex (or demographic group) be included so that they can be statistically analyzed separately, and the statisticians can draw conclusions about safety and effectiveness for members of each sex.
We agree that sex differences should be the focus, rather than gender differences. We also agree with the importance of enrolling females and males of different ages, races, ethnicities, co-morbidities, and hormonal statuses. For example, when males or females are taking any form of sex hormones for whatever reason (birth control, menopausal symptoms, low testosterone, or due to gender preference), it may be necessary to analyze those groups separately to see if the hormonal treatments affect sex differences. If there are too few to analyze these subgroups separately, the indication on the product label should make it clear that the data may not apply to patients in those subgroups.
We agree with the proposed guidance that companies should analyze and interpret sex-specific data to hormonal and other changes with age, for products used by adults of all ages. Ideally, researchers should first analyze the data separately by sex to see if there are age differences in safety or effectiveness.
Encouraging Rather Than Requiring Meaningful Representation
Unlike other U.S. public health agencies, the FDA merely “encourages representation of females (or other demographic groups) in clinical trials submitted to the FDA” rather than requiring representation. As a result, major demographic groups have often been under-represented in clinical trials submitted to the FDA (Fox-Rawlings et al., 2018). The FDA has justified this lower standard by stating that the studies submitted to the FDA are paid for industry, rather than the U.S taxpayer. However, the U.S. taxpayer pays for the treatments that the FDA approves, even when those treatments are not studied on patients that meaningfully represent the U.S. population. Therefore, encouraging rather than requiring representation is potentially misleading and unsafe for the patients who are not meaningfully represented. At the very least, the FDA should improve the incentives for companies to comply with the recommendations by ensuring that labeling indicates which types of patients were not reliably studied, and limiting the indication to the sex (and age groups, etc.) for whom safety and effectiveness data are reliably provided.
We support the FDA Draft Guidance statement that sponsors “must submit a diversity action plan with goals for study enrollment, disaggregated by sex, among other demographic characteristics.” Unfortunately, diversity action plans do not always result in achieving the goals intended. We agree with the FDA’s list of ways “to improve the recruitment, enrollment, and retention of females in clinical trials” but believe they are also suitable for improving recruitment of men of all ages as well.
Trial Design
We agree with the initial statement of the draft guidance regarding trial design: “For most drugs and devices, males and females should be included in clinical trials in numbers adequate to allow for reliable benefit-risk assessments and to understand any potential sex related differences in medical product response.”
However, we strongly disagree with the Draft Guidance’s strong focus on determining if a product is safer or more effective for one sex than the other, because that does not matter to patients. What matters to patients is whether the product is statistically significantly safe and effective compared to placebo or other treatments. In other words, there is no reason why a woman would care if a product is more effective for men that for women as long as it is beneficial in a clinically meaningful way for females. Similarly, why would any man care if a product is safer for women than men, if its benefits outweigh the risks for both? Rather than analyze men and women together and separately and then determine if it is safe and effective for both, as the guidance suggests, it would introduce less bias to start with the smaller number of study participants in each sex and if each has statistically significant benefits that outweigh the risks, then it is not necessary to combine males and females. If those benefits are not statistically significant compared to placebo but suggest meaningful benefits, then the men and women can be combined to see if together the results are statistically significant or not.
We disagree with the draft suggestion that only where sex differences “are anticipated, there should be sufficient numbers to inform reliable benefit-risk assessments in males and females.” Since, it is often unanticipated or unknown whether there will be sex differences, there should always be sufficient numbers of males and females to provide meaningful data.
Statistical Concepts
We agree that “Analyzing sex differences in medical product performance is an important component of assessing product safety and effectiveness and can inform what goes in the product labeling to improve patient care.” We also agree that “Sex is [only] one of many potential demographic characteristics typically evaluated in subgroup analyses of a clinical trial or non-interventional study.’ We agree that when many subgroup analyses are performed, some will be statistically significant by chance, and that different effects between males and females may be due to other factors associated with sex, such as age and weight. These must be taken into consideration when multiple statistical analyses are performed.
Reporting Results of Analyses
We support the requirement that sponsors “must include in their annual reports for drug and biological products conducted under an IND, the number of participants entered into the study to date tabulated by age group, sex, and race, and sponsors must present safety and effectiveness data in the clinical data section of an NDA by sex, age, and racial subgroups. Because the enrollment demographics of the clinical study may impact the generalizability of the conclusions, for clinical studies of devices, the FDA recommends that sponsors report the number and proportion of study participants by sex.” We also agree that “Any potential difference by sex should be investigated, explained, and discussed with the Agency.”
We agree that the labeling should specify the safety and effectiveness of any product separately for males and females of different demographic subgroups when available, and we add that the labeling should specify when those subgroup analyses are not available.
Other General Considerations
We reiterate that if a product is not proven safe and/or effective compared to placebo or a different treatment, that information should influence the indication. The Draft Guidance suggests that such differences “may potentially be further explored in a study after approval” and that the FDA can require a postmarketing study when applicable criteria are met.” We strongly disagree with this laissez-faire approach, because it can result in years of inappropriate or ineffective treatment for males or females when other treatments might be more beneficial. Therefore, when there is evidence that a product is not safe or not effective for females, or for males, additional research should be conducted before approval.
For example, in our study of high-risk medical devices, we examined publicly available documents for all 22 medical devices that the FDA designated “highest risk” or “novel,” that were reviewed through the premarket approval (PMA) pathway, and were scrutinized at the FDA public meetings from 2014 to 2017 (Fox-Rawlings et al., 2018).1 We evaluated patient demographics and subgroup analyses for all pivotal trials. Although 20 were intended for men and women, the number of patients in the minority gender was only 1 in one study and half the studies included less than 35% of the minority gender. Only 6 (33%) of the devices included subgroup analysis by sex for safety and 13 (72%) included subgroup analysis by sex for effectiveness. One of the devices, the Lutonix drug-coated balloon catheter used to reduce blockage in the leg, was effective for the total patient sample, but that was because it was effective for men. Women assigned to the Lutonix study arm had slightly worse outcomes than women in the control arm of the study; the artery remained sufficiently dilated a year after the procedure for just over half the women. Nevertheless, the FDA approved this high-risk device for women as well as men. The FDA required a post-market randomized study, but the sponsor said it was unable to enroll a sufficient number of patients. Subsequent data continued to indicate female Lutonix patients did not do as well as male patients, but no publicly available studies indicate if non-drug eluting catheters are superior to Lutonix using real life data. Lutonix continues to be approved for female as well as male patients.
Our study indicated that the frequent lack of subgroup analyses makes it impossible to inform patients or physicians as to whether many newly approved medical devices are safe and effective for specific demographic subgroups defined by gender, race, and age. However, even when the analyses indicated that the women did not benefit from a high-risk device, it was approved by the FDA for women and men anyway.
References
1 Fox-Rawlings, S. R., Gottschalk, L. B., Doamekpor, L. A., & Zuckerman, D. M. (2018). Diversity in Medical Device Clinical Trials: Do We Know What Works for Which Patients?. The Milbank quarterly, 96(3), 499–529. https://doi.org/10.1111/1468-0009.12344
2 U.S. Food and Drug Administration. (n.d.). PAS 1 (Extended Follow-up Study) [Post-Approval Studies Database]. https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfpma/pma_pas.cfm?c_id=&t_id=524729
3 Becton, Dickinson and Company. (n.d.). Lutonix™ Drug Coated Balloon PTA Catheters: Clinical Data. BD. Retrieved April 7, 2025, from https://www.bd.com/en-us/products-and-solutions/products/product-families/lutonix-drug-coated-balloon-pta-catheters#clinicaldata