Venetoclax and Sequential Busulfan, Cladribine, and Fludarabine Phosphate Before Donor Stem Cell Transplant in Treating Patients With Acute Myelogenous Leukemia or Myelodysplastic Syndrome

Inclusion Criteria: * Patients with biopsy-proven acute myeloid leukemia or myelodysplastic syndrome with persistent disease or in remission * Human leukocyte antigen (HLA)-identical sibling or 8/8 matched unrelated donor transplant * Patients age 18 to 70 years old; eligibility for pediatric patients will be determined in conjunction with an MD Anderson Cancer Center (MDACC) pediatrician; patients age 2-17 years may be enrolled after at least 10 adults (ages 18-70 years old) have been assessed for safety at day 30 * Direct bilirubin \< 1 mg/dl * Alanine aminotransferase (ALT) \< 3 times upper limit of normal * Creatinine clearance \> 50 ml/min (calculated creatinine clearance is permitted) * Forced expiratory volume in 1 second (FEV1) \>= 50% of expected corrected for hemoglobin and/or volume * Forced vital capacity (FVC) \>= 50% of expected corrected for hemoglobin and/or volume * Diffusing capacity of the lungs for carbon monoxide (DLCO) \>= 50% of expected corrected for hemoglobin and/or volume * Children unable to perform pulmonary function tests (e.g., less than 7 years old) pulse oximetry of \>= 92% on room air * Left ventricular ejection fraction (LVEF) \>= 40% * Patient, legally authorized representative (LAR), or parent able to sign informed consent; able to give assent for patients age 7-17 * Negative beta human chorionic gonadotropin (HCG) test in a woman with child bearing potential, defined as not post-menopausal for 12 months or no previous surgical sterilizat…