Testing Nivolumab and Ipilimumab With Short-Course Radiation in Locally Advanced Rectal Cancer

Inclusion Criteria: * Patient must be \>= 18 years of age * Patient must have histologically confirmed adenocarcinoma of the rectum with the inferior margin within 15 cm from the anal verge based on colonoscopy and/or flexible sigmoidoscopy * Patient must have T3-4Nx or TxN+ disease (stage II or III) based on magnetic resonance imaging of the pelvis and computed tomography of the chest and abdomen. These baseline scans must be done within 28 days prior to registration * Patient must have MSI-H (microsatellite instability-high) or dMMR (deficient mismatch repair) tumors based on immunohistochemistry or PCR (polymerase chain reaction) * Patient must have Eastern Cooperative Oncology Group (ECOG) Performance status 0-2 * Patient must not have previously received chemotherapy or immunotherapy for rectal cancer * Patient must not have previously received radiotherapy to the pelvis * Patient must have the ability to understand and the willingness to sign a written informed consent document. Patients with impaired decision-making capacity (IDMC) who have a legally authorized representative (LAR) or caregiver and/or family member available will also be considered eligible * Patient must not have had major surgery performed within 28 days prior to registration * Patient must not have a history of interstitial lung disease (e.g., pneumonitis or pulmonary fibrosis) or evidence of interstitial lung disease on baseline chest CT scan * Patient must not have a serious active infection requ…