Testing A New Combination of Anti-cancer Immune Therapies, Atezolizumab and CDX-1127 (Varlilumab) With or Without the Addition of a Third Anti-cancer Drug, Cobimetinib, for Advanced-Stage Biliary Tract Cancer

Inclusion Criteria: * Pathologically confirmed biliary tract cancer, having received at least 1 prior line of systemic therapy, and received no more than 2 prior lines of therapy in the metastatic setting (disease recurrence =\< 6 months from the last dose of perioperative therapy/day of surgery \[whichever is more recent\] in resected patients will be considered the first line of therapy) * Includes intrahepatic cholangiocarcinoma (IHC), extrahepatic cholangiocarcinoma (EHC), and gallbladder carcinoma (GBC), but not Ampulla of Vater cancers * Patients must have measurable disease by Response Evaluation Criteria in Solid Tumors (RECIST) version (v)1.1 * Age \>= 18 years. Because no dosing or adverse event data are currently available on the use of atezolizumab, cobimetinib, and CDX-1127 (varlilumab) in patients \< 18 years of age, children are excluded from this study * Eastern Cooperative Oncology Group (ECOG) performance status =\< 1 (Karnofsky \>= 80%) * Absolute neutrophil count \>= 1,500/mcL * Hemoglobin \>= 9.0 g/dl * Platelets \>= 100,000/mcL * Total bilirubin =\< 1.5 x institutional upper limit of normal (ULN) (Patients with known Gilbert disease who have serum bilirubin level =\< 3 x ULN may be enrolled) * Aspartate aminotransferase (AST) (serum glutamic oxaloacetic transaminase \[SGOT\])/alanine aminotransferase (ALT) (serum glutamic pyruvic transaminase \[SGPT\]) =\< 3 x institutional ULN * Serum creatinine =\< 1.5 x institutional ULN OR * Creatinine clearance …