Nivolumab With or Without Stereotactic Radiosurgery in Treating Patients With Recurrent, Advanced, or Metastatic Chordoma

Inclusion Criteria: * Patients must have histologically confirmed diagnosis of chordoma; the pathologic confirmation may be from another metastatic site * Patients may have metastases, with newly identified peripheral metastases * Cross-sectional imaging evidence of progression of recurrent or metastatic disease * Previous treatment information (name of agent, treatment starting date, and date of progression) must be available for review * Measurable disease in one or more site. (Per RECIST criteria: Measurable lesions are defined as those that can be accurately measured in at least one dimension (longest diameter to be recorded) as \>= 10 mm with CT scan, MRI, or calipers by clinical exam. All tumor measurements must be recorded in millimeters (or decimal fractions of centimeters)). * Karnofsky performance scale \>= 70% * White blood cells (WBC) \>= 3,000/mcL * Absolute neutrophil count \>= 1,500/mcL * Platelets \>= 100,000/mcL * Total bilirubin =\< 2 x institutional upper limit of normal * Aspartate aminotransferase (AST) (serum glutamic oxaloacetic transaminase \[SGOT\])/alanine aminotransferase (ALT) (serum glutamic pyruvic transaminase \[SGPT\]) =\< 2.5 x institutional upper limit of normal * Creatinine within normal institutional limits OR according to Johns Hopkins magnetic resonance imaging (MRI) policy * Women of child bearing potential (WOCBP) should use an adequate method to avoid pregnancy for 5 months plus the time required for nivolumab to undergo approximately…