Pomalidomide and Dexamethasone in Treating Patients With Relapsed or Refractory Primary Central Nervous System Lymphoma or Newly Diagnosed or Relapsed or Refractory Intraocular Lymphoma

Inclusion Criteria: * Relapsed or refractory primary central nervous system (CNS) diffuse large B cell lymphoma (PCNSDLBCL) with a CNS lesion, with cerebrospinal fluid (CSF) relapse with positive CSF cytology, or with ocular relapse with positive ocular tissue biopsy; NOTE: tissue biopsy is not absolutely necessary for CNS tumor unless clinical and radiologic findings strongly suggest other etiologies as per treating physician; initial diagnosis must be made by tissue biopsy; NOTE: patients with B-cell lymphoma with features intermediate between diffuse large B-cell lymphoma and Burkitt lymphoma are also eligible for the protocol as long as they meet other criteria; patients with typical Burkitt lymphoma are not eligible * Relapsed/refractory primary vitreoretinal diffuse large B cell lymphoma (DLBCL) with a CNS lesion, with CSF relapse with positive CSF cytology, or with ocular relapse with positive ocular tissue biopsy; NOTE: tissue biopsy requirement of the CNS lesion is as outlined in bullet above * Eastern Cooperative Oncology Group (ECOG) performance status (PS) 0, 1, 2 or 3 * Absolute neutrophil count (ANC) \>= 1000/uL * Platelets (PLT) \>= 100,000/uL * Total bilirubin =\< 1.5 x upper limit of normal (ULN) or if total bilirubin is \> 1.5 x ULN the direct bilirubin must be =\< 1.5 x ULN (=\< 0.45 mg/dL) * Aspartate aminotransferase (AST) =\< 3 x ULN * Creatinine =\< 2.5 x ULN * Females of reproductive potential must be willing to adhere to the scheduled pregnancy testi…