Nivolumab and Radiation Therapy With or Without Ipilimumab in Treating Patients With Brain Metastases From Non-small Cell Lung Cancer

Inclusion Criteria: * Pathologically confirmed non-small lung cancer * Stage IV metastatic disease with intracranial disease visible with magnetic resonance image (MRI) * At least one brain lesion size \>= 0.3 cm in the longest axis amenable to radiation therapy (either via SRS or WBRT) * Be willing and able to provide written informed consent/assent for the trial * Have a performance status of 0 or 1 on the Eastern Cooperative Oncology Group (ECOG) performance scale * Absolute neutrophil count (ANC) \>= 1,000 /mcL (performed 28 days prior to study registration up to the first dose of study drug) * Platelets \>= 100,000 /mcL (performed 28 days prior to study registration up to the first dose of study drug) * Hemoglobin \>= 9 g/dL or \>= 5.6 mmol/L (performed 28 days prior to study registration up to the first dose of study drug) * Serum creatinine or measured or calculated creatinine clearance (glomerular filtration rate \[GFR\] can also be used in place of creatinine or creatinine clearance \[CrCl\]) =\< 1.5 X upper limit of normal (ULN) or \>= 40 mL/min CrCl using the Cockroft-Gault formula (performed 28 days prior to study registration up to the first dose of study drug) * Serum total bilirubin =\< 1.5 X ULN (except for subjects with Gilbert syndrome, who may have total bilirubin \< 3.0 mg/dl) or direct bilirubin =\< ULN for subjects with total bilirubin levels \> 1.5 x ULN (performed 28 days prior to study registration up to the first dose of study drug) * Aspartate amin…