Brentuximab Vedotin and Nivolumab With or Without Ipilimumab in Treating Patients With Relapsed or Refractory Hodgkin Lymphoma

Inclusion Criteria: * PHASE I (ARMS A, B, C, D, E, F, G, H, I, X, Y, Z) * Age \>= 18 years * Patients must have pathologically confirmed relapsed or refractory classical Hodgkin lymphoma (cHL); a biopsy at any relapse is acceptable; other histologies including lymphocyte predominant (LP) HL are not permitted * Patients must have relapsed after first line chemotherapy; may have relapsed after autologous or allogeneic stem cell transplant, or have primary refractory disease; no upper limit for number of prior therapies; if status post allogeneic stem cell transplant, no active graft versus host disease * Patients may have received prior brentuximab vedotin, but must not have received brentuximab vedotin within 6 months prior to registration, and must not have relapsed within 6 months of receiving previous brentuximab vedotin; patients may not have received prior nivolumab or PD1/PDL1 axis agents; patients in the nivolumab/brentuximab cohorts ONLY (D, E, F, Y) may have received prior ipilimumab * Patients may have received other prior activating immunotherapies (i.e. checkpoint inhibitors), but must not have received them within 6 months prior to registration, and there must be no serious unresolved complication of therapy at the time of registration; for the purposes of this study monoclonal antibodies and antibody drug conjugates are not considered to be activating immunotherapies and there are no additional time restrictions on prior exposure to these agents (except prior br…