Venetoclax and Decitabine in Treating Participants With Relapsed/Refractory Acute Myeloid Leukemia or Relapsed High-Risk Myelodysplastic Syndrome

Inclusion Criteria: * Patients with AML, BPDCN, biphenotypic or bilineage leukemia (including a myeloid component) or mixed phenotype acute leukemia (MPAL) who have failed prior therapy; patients with AML should have failed prior therapy or have relapsed after prior therapy; patients with isolated extramedullary AML are eligible * Elderly (\> 60 year old) patients with newly diagnosed AML, BPDCN, or mixed phenotype acute leukemia (MPAL) not eligible for intensive chemotherapy * Patients with newly diagnosed AML with poor risk complex karyotype and/or TP53 deletions/mutations equal or younger than 60 year old * AML or BPDCN patients with prior history of MDS or CMML who received any therapy or no therapy for the MDS or CMML and progressed to AML, are eligible at the time of diagnosis of AML regardless of any prior therapy for MDS; the World Health Organization (WHO) classification will be used for AML * Patients with high-risk MDS with bone marrow blasts between 10% and 20%, relapsed or refractory to prior hypomethylating agent (HMA) therapy, defined as prior receipt of 4 cycles of HMA therapy with failure to attain a response, or relapse after prior response to HMA therapy; patients with high risk chronic myelomonocytic leukemia (CMML) with bone marrow blasts \>= 10% regardless of prior therapy * Eastern Cooperative Oncology Group (ECOG) performance status =\< 3 * White blood cell count =\< 10,000 * Adequate renal function including creatinine \< 2 unless related to the dise…