Nivolumab With or Without Ipilimumab in Treating Patients With Refractory Metastatic Anal Canal Cancer

Inclusion Criteria: * Patients must have histologically or cytologically confirmed previously treated metastatic squamous cell carcinoma of the anal canal * Patients must have measurable disease according to the standard Response Evaluation Criteria in Solid Tumors (RECIST) version 1.1. CT scans or MRIs used to assess the measurable disease must have been completed within 28 days prior to study drug initiation * Patients must have been treated with at least one prior systemic treatment for incurable advanced or metastatic SCCA of the anal canal; prior treatment for metastatic disease is not required for patients who develop new metastatic lesions during or within 6 months of completion of chemoradiation for limited-stage disease; patients who receive chemotherapy for incurable advanced or metastatic SCCA of the anal canal must wait a minimum \>= 28 days (6 weeks for nitrosoureas or mitomycin C) after the date of completion of chemotherapy prior to initiating treatment with nivolumab on this study; patients who undergo palliative radiotherapy to a site of tumor must wait a minimum \>= 28 days from the date of completion of radiotherapy prior to initiating treatment with nivolumab (Part A and B) or nivolumab +/- Ipilimumab (Part B) on this study * Patients must be of age \>= 18 years at the time of study registration; because no dosing or adverse event data are currently available on the use of nivolumab in patients \< 18 years of age, children are excluded from this study, bu…