Pembrolizumab Alone or With Idelalisib or Ibrutinib in Treating Patients With Relapsed or Refractory Chronic Lymphocytic Leukemia or Other Low-Grade B-Cell Non-Hodgkin Lymphomas

Inclusion Criteria: * CLL/SLL PATIENTS (ARM A) ONLY * Diagnosis of CLL according to the National Cancer Institute (NCI) criteria or SLL according to the World Health Organization (WHO) criteria; this includes previous documentation of: * Biopsy-proven small lymphocytic lymphoma or * Diagnosis of CLL according to NCI working group criteria as evidenced by all of the following: * Peripheral blood B cell count of \> 5 x 10\^9/L consisting of small to moderate size lymphocytes * Immunophenotyping consistent with CLL defined as: * The predominant population of lymphocytes share both B-cell antigens (cluster of differentiation \[CD\]19, CD20 \[typically dim expression\] or CD23) as well as CD5 in the absence of other pan-T-cell markers (CD3, CD2, etc.) * Clonality as evidenced by kappa or lambda light chain expression (typically dim immunoglobulin expression) or other genetic method (e.g. immunoglobulin heavy chain variable \[IGHV\] analysis) * NOTE: splenomegaly, hepatomegaly, or lymphadenopathy are not required for the diagnosis of CLL * Before diagnosing CLL or SLL, mantle cell lymphoma must be excluded by demonstrating a negative fluorescent in situ hybridization (FISH) analysis for t(11;14) (immunoglobulin H \[IgH\]/cyclin D1 \[CCND1\]) on peripheral blood or tissue biopsy or negative immunohistochemical stains for cyclin D1 on involved tissue biopsy * Patients must be previously treated with at least one prior line of therapy; EXCEPTION: …