Construction and evaluation of machine learning-based prediction model for live birth following fresh embryo transfer in IVF/ICSI patients with polycystic ovary syndrome

Abstract

Objective

To investigate the determinants affecting live birth outcomes in fresh embryo transfer among polycystic ovary syndrome (PCOS) patients using various machine learning (ML) algorithms and to construct predictive models, offering novel insights for enhancing live birth rates in this specific group.

Methods

A sum of 1,062 fresh embryo transfer cycles involving PCOS patients were analyzed, with 466 resulting in live births. The dataset was split randomly into training and testing subsets at a 7:3 ratio. Least absolute shrinkage and selection operator and recursive feature elimination methods were utilized for feature selection within the training data. A grid search strategy identified the optimal parameters for seven ML models: decision tree (DT), K-nearest neighbors (KNN), light gradient boosting machine (LightGBM), naive Bayes model(NBM), random forest (RF), support vector machine (SVM) and extreme gradient boosting (XGBoost). The evaluation of model effectiveness incorporated diverse metrics, encompassing area under the curve (AUC), accuracy, positive predictive value, negative predictive value, F1 score, and Brier score. Calibration curves and decision curve analysis were employed to ascertain the optimal model. Furthermore, Shapley additive explanations were applied to elucidate the importance of predictor variables in the top-performing model.

Results

The AUC values of DT, KNN, LightGBM, NBM, RF, SVM and XGBoost models in the training set were 0.813, 1.000, 0.724, 0.791, 1.000, 0.819 and 0.853, respectively. Corresponding values in the testing set were 0.773, 0.719, 0.705, 0.764, 0.794, 0.806 and 0.822. XGBoost emerged as the most effective ML model. SHAP analysis revealed that variables encompassing embryo transfer count, embryo type, maternal age, infertility duration, body mass index, serum testosterone (T) levels, and progesterone (P) levels on the day of human chorionic gonadotropin administration were pivotal predictors of live birth outcomes in individuals with PCOS receiving fresh embryo transfer.

Conclusion

This study developed a live birth prediction model tailored for PCOS fresh embryo transfer cycles, leveraging ML algorithms to compare the efficacy of multiple models. The XGBoost model demonstrated superior predictive capacity, enabling prompt and precise identification of critical risk factors influencing live birth outcomes in PCOS patients. These findings offer actionable insights for clinical intervention, guiding strategies to improve pregnancy outcomes in this population.

Clinical trial number

Not applicable.

Polycystic ovary syndrome (PCOS) is widely recognized as a prevalent reproductive, endocrine, and metabolic disorder impacting women of childbearing age. Beyond pathophysiological irregularities—such as ovulatory dysfunction, infertility, hyperandrogenism (HA), excessive body weight, hyperinsulinemia, and metabolic syndrome—psychological challenges, including anxiety and depression, also influence reproductive well-being. These individuals not only encounter difficulties in achieving conception but are also exposed to heightened risks of miscarriage and preterm delivery during pregnancy. Additionally, pregnancy-related complications and comorbidities, such as gestational diabetes mellitus (GDM) and hypertensive disorders of pregnancy, exhibit a marked increase [1, 2], with potential long-term implications for the metabolic health and development of offspring [1, 3].

In vitro fertilization-embryo transfer (IVF-ET) technology, along with its derivative method, intracytoplasmic sperm injection (ICSI), has been extensively applied in the treatment of infertility, with the proportion of PCOS patients steadily rising over the years [4]. The advancement of machine learning (ML) algorithms has facilitated the development of numerous ML-based models within the domain of assisted reproduction [5]. These algorithms offer remarkable adaptability in handling intricate nonlinear relationships among variables and identifying complex data patterns and interactions, which allows for more precise predictions and enhanced model performance [6]. Moreover, in comparison with logistic regression, ML approaches exhibit superior robustness against outliers, reduced sensitivity to extreme values, and more effective data-processing capabilities, whereas the susceptibility of logistic regression to outliers may result in skewed outcomes [7].

With the advancement of ML applications in the medical field, considerable strides have been achieved in the early diagnosis and management of PCOS [8, 9]. Artificial intelligence-driven techniques are capable of recognizing patterns within medical data, such as hormonal profiles, facilitating the differentiation between PCOS and non-PCOS individuals. This enhanced precision allows for earlier and more accurate diagnoses, ultimately contributing to improved pregnancy outcomes for patients with PCOS [10]. Although previous studies have investigated factors influencing live births following IVF treatment among PCOS patients [11–12], these studies are based on the frozen-thawed embryo transfer cycles, with a deficiency in the analysis of the influencing factors of the outcomes of fresh embryo transfer cycles in PCOS. In recent years, through effective improve the body metabolism and endocrine abnormalities, and accept low dose gonadotropin (GN) stimulate ovulation treatment, more and more patients with PCOS were offered the opportunity to fresh embryo transfer. For this group of people, our study is useful for the targeted interventions and the development of personalized treatment. Moreover, research focusing on the use of ML for predicting live births within this context remains scarce. The purpose of this investigation is to explore influential factors contributing to live births after fresh embryo transfer in individuals with PCOS, determine reliable predictive indicators, and construct prediction models for live birth outcomes after fresh embryo transfer. Seven ML models were developed and assessed using demographic, biochemical, and embryological data, and their predictive performance characteristics were compared. Furthermore, Shapley additive explanations (SHAP) were utilized to analyze and interpret each predictor’s contribution, addressing the challenges associated with the interpretability of ML models and providing clinical guidance for early intervention in this population.

Methodologies and materials

General information

This retrospective cohort study investigated assisted reproductive populations, focusing specifically on female PCOS patients who underwent the antagonist protocol followed by fresh embryo transfer at the Fujian Provincial Maternal and Child Health Hospital between January 2019 and December 2023.

Inclusion criteria: ① PCOS patients meeting the Rotterdam diagnostic criteria [13] or Chinese guidelines for PCOS diagnosis and treatment [14]; ② ovarian stimulation with antagonist protocol; ③ fresh embryo transfer cycles.

Exclusion criteria: ① uterine abnormalities including uterine malformation, adenomyosis, and submucosal fibroids; ② endometriosis; ③ hydrosalpinx; ④ chromosomal abnormalities in either partner; ⑤ severe oligoasthenozoospermia in male partners; ⑥ loss to follow-up or missing outcome data. This investigation per the principles of the Declaration of Helsinki.

Ovarian stimulation and in vitro fertilization (IVF)

All patients underwent ultrasound assessments and hormone assays on the second or third day of their menstrual cycle. Ovarian stimulation was initiated with Gn [Gonal-F, recombinant FSH (rFSH), Merck Serono; recombinant follitropin beta injection (rFSH), MSD (China); or urofollitropin for injection, Livzon Pharmaceutical Group] at an initial dosage ranging from 112.5 to 225 U. GnRH antagonists (Ganirelix, 0.25 mg, Organon; or Cetrotide, 0.25 mg, Merck Serono) were administered once follicles exceeded 12 mm in diameter and/or serum estradiol concentrations reached at least 1,000 pmol/L. Dosage adjustments were made based on each patient’s response. When two follicles measured 18 mm or more, or three follicles reached at least 17 mm in diameter, final oocyte maturation was triggered with either 250 µg recombinant human chorionic gonadotropin (Ovidrel, Merck Serono, Switzerland) or a dual trigger involving 0.2 mg triptorelin acetate (Decapeptyl, Ferring, Germany) administered subcutaneously, combined with 2,000 IU human chorionic gonadotropin (HCG, Livzon, China) administered intramuscularly.

Oocyte retrieval was conducted under the guidance of transvaginal ultrasound 36–38 h following the trigger. Transfer of either a fresh cleavage-stage embryo or a blastocyst was performed on the third or fifth-day post-retrieval. Standard luteal phase support was initiated after oocyte retrieval, comprising oral dydrogesterone tablets (Duphaston, Abbott, Netherlands), administered every eight hours, along with an progesterone vaginal sustained-release gel (Crinone, Merck Serono, Switzerland) at a dosage of 90 mg per day via vaginal administration. A positive hCG result was defined by serum hCG levels exceeding 5 U/L on the 14th day after cleavage-stage embryo transfer or the 12th day following blastocyst transfer. Clinical pregnancy was confirmed by the presence of at least one gestational sac, with or without a fetal pole, located within or outside the uterus, as verified through transvaginal ultrasound.

Data collection and preprocessing

For eligible patients, basic clinical data were collected, including: ① demographic information (such as age, body mass index (BMI), duration of infertility, and number of treatment cycles); ② laboratory test results (e.g., basal follicle-stimulating hormone (FSH), luteinizing hormone, and estradiol (E2) levels); ③ treatment procedures (e.g., GN dosage, number of transferred embryos, and types of transferred embryos). Data cleaning was conducted in accordance with patterns of missing values. Initially, rows containing more than 20% missing data across all samples were excluded. Subsequently, columns with over 20% missing values across all rows were also eliminated. Any remaining missing values were imputed using the missForest function in R.

Data feature definition

A live birth was characterized as a pregnancy reaching 28 weeks or more, resulting in the presence of at least one of the four vital signs following delivery: heartbeat, respiration, umbilical cord pulsation, or muscle tone.

Statistical analysis

Python 3.9, R 4.3.2, and SPSS 26.0 were utilized to perform data analysis. Continuous variables were evaluated utilizing either the t-test or the Mann-Whitney U test, while categorical variables were analyzed with the chi-square test. Continuous data were expressed as median values, whereas categorical data were described as frequencies or percentages. The significance level (α) was established at 0.05. The dataset was split arbitrarily into training and testing subsets at a ratio of 7:3. Least absolute shrinkage and selection operator (LASSO) regression, in combination with recursive feature elimination (RFE), was utilized to identify predictive factors, thereby enhancing both predictive performance and model interpretability. Optimal parameters for multiple ML models were ascertained utilizing five-fold cross-validation and grid search. Decision tree, K-nearest neighbors, light gradient boosting machine, naive Bayes model, random forest, support vector machine, and extreme gradient boosting (XGBoost) models were developed using five-fold cross-validation. The discriminative ability of these models in predicting live birth events was evaluated through metrics encompassing area under the curve (AUC), accuracy, precision, positive predictive value (PPV), negative predictive value (NPV), F1 score, and Brier score. Calibration curves were employed to assess the alignment between forecasted probabilities and actual outcomes in the optimal model. SHAP was applied to explain the optimal model. The statistical analysis and model development workflow are illustrated in Fig. 1.

Results

Patient characteristics

A sum of 1,062 cycles was analyzed in this investigation, with 466 resulting in live births. The modeling group comprised 743 cycles, while the validation group included 319 cycles. Furthermore, no statistically significant differences were detected in the baseline data and clinical characteristics between these two groups (all P > 0.05), affirming their comparability (Table 1). Table 2 depicts a comparison of patient characteristics between the live birth group and the control group. Significant differences (P < 0.05) were identified for maternal age, BMI, infertility duration, treatment frequency, serum testosterone (T) levels, initial GN dosage, FSH levels on the day of HCG administration, progesterone (P) levels on the HCG day, the number of high-quality cleavage-stage embryos, the type of transferred embryos, and the number of transferred embryos between the groups.

Fig. 1

Flow chart of the study

Table 1 Basis characteristics of training group and validation group

Table 2 Basis characteristics of live birth group and control group

Selection of predictive factors

LASSO regression, in combination with RFE, was employed to identify predictive factors. Figure 2ABand 2 C illustrate the variable selection process based on LASSO regression, yielding a feature set of 9 through ten-fold cross-validation, with Lambda 1se serving as the selection criterion. The impact of increasing feature numbers on model accuracy, determined using the RFE approach, is depicted in Fig. 2D and E. The model attained its highest accuracy when the feature count reached 10. The final feature set is presented in Fig. 2F, where the left circle displays features identified by LASSO regression and the right circle highlights features selected through RFE. The intersection of these two sets comprises 7 features, representing the final feature set considered for inclusion in the model.

Fig. 2

Features selected by LASSO and RFE. (A) The Lasso regression coefficient profiles of all baseline characteristics. (B) The optimal lambda selection in the Lasso regression with 10-fold cross-validation. Misclassification errors of different variables against log(lambda) are revealed. The two vertical dashed lines represent the optimal value under the minimum criterion and 1-SE criterion, respectively. The “lambda”is the tuning parameter. (C) A total of 9 predictors with non-zero coefficients are identified. (D) Features selected by RFE, When the number of features is 10, the RMSE is the lowest. (E) The top ten significant predictors identified by RFE. (F) The Venn diagram of features selected by LASSO and RFE. The intersection results of two methods yield 7 predictors. LASSO, Least Absolute Shrinkage and Selection Operator; RFE, Recursive Feature Elimination; RMSE, Root Mean Square Error

Development and validation of prediction models

The selected predictive factors were integrated into seven distinct ML models. To optimize the predictive capabilities of each model, hyperparameters underwent further refinement, and 5-fold cross-validation was utilized for performance assessment. The outcomes of these models on the validation set are presented in Table 3. As depicted in Fig. 3, the XGBoost model achieved superior performance compared to the other six models, attaining an AUC of 0.822 (95% confidence interval [CI] = 0.777–0.867), an accuracy rate of 0.752, specificity of 0.732, sensitivity of 0.772, PPV of 0.682, NPV of 0.812, an F1 score of 0.724, and a Brier score of 0.172 in the validation set. Furthermore, calibration curves were employed to ascertain the predictive effectiveness of the models. The calibration curve for the validation set in Fig. 4A reflects a high level of alignment between the forecasted live birth rates and the actual outcomes when the XGBoost model is used. Figure 4B displays the decision curve, suggesting that the XGBoost model provides enhanced net benefits when the anticipated live birth rates span from 10 to 90%.

Table 3 Performance of seven machine learning-based models for predicting live birth in the testing set

Fig. 3

Comparison of receiver operator characteristic curves (ROCs) for the machine learning models. (A) The ROCs of training models. (B) The ROCs of validation models. AUC, area under the ROC; DT, decision tree; KNN, k-nearest neighbors; LGBM: light gradient boosting machine; NBM, naïve bayes model; RF, random forest; XGBoost, eXtreme gradient boosting

Fig. 4

Discriminative power and accuracy of XGBoost model. A. The calibration curves of the validation group in XGBoost model. B The clinical decision curves of the validation group in XGBoost model

Model interpretability using SHAP

The XGBoost model exhibited excellent predictive capability, leading to the adoption of the SHAP framework for further model interpretation. Figure 5A displays the seven most influential factors, ranked by their mean absolute SHAP values, which include, in descending order: number of transferred embryos, blastocyst transfer, female age, duration of infertility ≥ 3, BMI ≥ 28, testosterone (T) level and P level on HCG day. Figure 5B visualizes the effects of these factors on live birth outcomes, with the y-axis representing the factor values and the x-axis reflecting their influence on the likelihood of live birth. A higher female age, elevated serum T levels, increased P levels on HCG day, duration of infertility ≥ 3 and BMI ≥ 28 were linked with a lower probability of live birth after fresh embryo transfer in PCOS patients. Two representative cases are presented to illustrate personalized feature attributions and demonstrate the application of SHAP in explaining individual model predictions. Specifically, Fig. 5C depicts a PCOS patient who achieved live birth, while Fig. 5D illustrates a PCOS patient who did not achieve live birth. The explanation process begins with the base value, which represents the average prediction across all instances. Subsequently, each input feature at varying levels can either increase or decrease the predicted probability of the outcome. The length of the arrows in the force plots reflects the magnitude of the SHAP values for these features. Ultimately, the model’s predicted output for a specific patient is derived.

Fig. 5

SHAP plots. (A) SHAP summary plot shows feature importance for each predictor of the XGBoost model in descending order. The upper predictors are more important to the model’s predictive outcome. A dot is created for each feature attribution value for the XGBoost model of each patient. The further away a dot is from the baseline SHAP value of zero, the stronger it effects the model output. Dots are colored according to the values of features. Yellow represents higher feature values and red represents lower feature values. (B) Bar chart of the mean absolute SHAP value for each predictor of the XGBoost model in descending order. C and D. The force plots provide personalized feature attributions using two examples

Discussion

In recent years, the correction of metabolic and endocrine abnormalities, combined with low-dose GN ovarian stimulation in IVF, has enabled a growing number of PCOS patients to undergo fresh embryo transfer, thereby reducing the time required to achieve live birth. Utilizing ML methods, this study was among the first to explore the determinants affecting live birth outcomes in PCOS patients after fresh embryo transfer. Seven ML prediction models were developed, each exhibiting strong predictive ability in differentiating live birth outcomes within this group, with the XGBoost model delivering the most favorable performance. This model supports clinicians in initiating early diagnostic interventions for these patients and offers valuable insights for enhancing pregnancy outcomes in PCOS cases in the future.

ML techniques exhibit enhanced performance over traditional statistical methods when managing complex relationships among numerous features [15–16], as they can identify influencing factors that might be overlooked by conventional approaches based on experience [17]. For predictor selection, LASSO regression and RFE were employed, and their intersection was used to construct predictive models. Among the seven ML models developed, the XGBoost model demonstrated the highest performance, achieving an AUC of 0.822 (95% CI = 0.777–0.867). To further interpret the model and assess the contributions of individual predictors, SHAP analysis was applied to the top-performing XGBoost model. Each SHAP value quantifies the positive or negative impact of a feature on live birth outcomes after fresh embryo transfer in PCOS patients. Among the predictors, blastocyst transfer provided the most substantial contribution to model predictions, showing higher live birth rates in contrast to cleavage-stage embryo transfer [18]. Nevertheless, considering the heightened possibility of ovarian hyperstimulation syndrome in fresh embryo transfers for PCOS patients, some researchers have proposed adopting a freeze-all strategy [19–20]. A randomized controlled trial involving 1,650 patients compared fresh blastocyst transfer cycles with freeze-all and thawed blastocyst transfer groups, focusing on primary outcomes such as singleton live birth rates and secondary outcomes including pregnancy complications, neonatal birth weight, birth defects, and perinatal complications. The findings revealed that the freeze-all strategy significantly enhanced blastocyst implantation rates, live birth rates, and singleton newborn birth weights, contributing to improved maternal-fetal safety and clinical outcomes. However, the study also noted that frozen-thawed single blastocyst transfers were linked to an elevated risk of maternal preeclampsia, raising critical considerations for clinical application [21]. In a recent retrospective cohort study of 10,964 single blastocyst transfer cycles, it was observed that transferring single low-grade blastocysts yielded approximately 30% lower live birth rates relative to 44% for high-quality single blastocysts (with very low-grade blastocysts achieving 14%) without negatively affecting perinatal outcomes [22]. SHAP analysis further demonstrated that transferring two embryos increased the probability of live birth. Although twin embryo transfer yields higher pregnancy rates compared to single embryo transfer [23], existing research and consensus emphasize that it also elevates the risk of multiple pregnancies, along with associated pregnancy complications and adverse perinatal outcomes [24–25]. Consequently, single blastocyst transfer is recommended for PCOS patients, as it balances embryo implantation and live birth success with a reduced risk of multiple pregnancies.

As women age, fertility gradually declines. Once women surpass 35 years of age, the likelihood of spontaneous abortion increases substantially, while pregnancy and live birth rates decrease, accompanied by a heightened risk of various pregnancy and perinatal complications [26]. The connection between maternal age and clinical outcomes following embryo transfer in assisted reproductive technology (ART) has been well-established in numerous studies, with a general consensus highlighting significantly reduced clinical pregnancy and live birth rates in women older than 35 years undergoing ART [27–28]. Consistent findings were observed in this study. Advanced maternal age elevates the risk of early miscarriage, primarily due to a decline in oocyte quality associated with aging, which results in an increased likelihood of embryonic aneuploidy [29]. Preimplantation genetic screening prior to embryo transfer may help mitigate the risk of aneuploidy and reduce the incidence of early miscarriage, although the safety and long-term risks of this approach remain a topic of debate [30]. This study also found that increased progesterone levels on HCG day negatively impacted live birth rates in individuals with PCOS undergoing fresh embryo transfer. Some research suggests that progesterone levels on HCG day may indirectly influence endometrial receptivity and embryo attachment through alterations in gene expression during the implantation window [31]. However, other studies report no association between progesterone levels on HCG day and clinical pregnancy or miscarriage following IVF protocol [32]. As a result, the effect of progesterone measurements on HCG day regarding clinical results post-embryo transfer remains a debated topic.

A high BMI contributes not only to cardiovascular and metabolic disorders but also impairs fertility [33]. Multiple studies have established that a BMI ≥ 28 is correlated with reduced ovarian responsiveness and unfavorable pregnancy outcomes [34–35]. In this investigation, BMI ≥ 28 was ascertained as a prominent risk factor for the failure to achieve live birth in individuals with PCOS undergoing fresh embryo transfer. The foundational treatment for patients with PCOS involves lifestyle interventions, which encompass a diversified approach including appropriate exercise, dietary control, and behavioral modification. For obese PCOS patients, weight loss has been demonstrated to significantly improve treatment outcomes. Specifically, reducing body weight by 5–10% can lead to notable improvements in ovulation, menstrual cycle regulation, and insulin sensitivity. It is important to emphasize that weight loss should be gradual and sustained over time. HA is recognized as a defining feature of PCOS [36]. However, prior investigations into the influence of HA on reproductive outcomes in PCOS patients have been limited, concentrating mainly on early reproductive stages. Research on subsequent maternal and neonatal outcomes in individuals who achieved clinical pregnancy remains scarce, and the existing literature presents some inconsistencies. In animal experiments, Diao et al. [37] found that high-dose androgen exposure could disrupt endometrial development and interfere with the prostaglandin system, potentially causing early pregnancy loss. Similarly, De Vos et al. [38] reported significantly lower cumulative live birth rates among PCOS patients with HA compared to those without HA. Accordingly, lowering serum testosterone levels in PCOS patients positively influences live birth rates.

Anti-müllerian hormone (AMH) and antral follicle count (AFC) serve as crucial markers for evaluating ovarian reserve function in ART-assisted pregnancy populations. These markers are often utilized as predictors of IVF/ICSI-ET success rates and are regarded as useful metrics for forecasting live birth outcomes. Fertility counseling provided by clinicians frequently relies on changes in these indicators throughout ovarian stimulation to offer personalized guidance [39]. Nevertheless, the findings of this study suggest that AMH and AFC do not exhibit marked predictive value for live birth rates after embryo transfer in PCOS patients, highlighting the necessity for further investigations into the diagnosis and management of PCOS patients undergoing assisted reproductive treatments.

Certain constraints need to be recognized in this investigation. First, the data utilized were obtained from a single center, which may restrict the model’s applicability to patients from other institutions. Second, certain parameters, including insulin and glucose metabolism, were absent from the available electronic medical records. Furthermore, external validation of the constructed prediction model has not been performed, raising concerns regarding its generalizability and highlighting the need for further verification. Moving forward, comprehensive external validation datasets will be gathered to enhance the model’s robustness.

Seven ML models were developed in this study to predict live birth following fresh embryo transfer in patients with PCOS, demonstrating strong evaluation accuracy. These models provide critical support for identifying high-risk cases within this population that do not result in live birth, facilitate informed treatment decisions, and enable effective monitoring of patient progression.