Testing the Safety of the Anti-Cancer Drugs Durvalumab and Olaparib During Radiation Therapy for Locally Advanced Unresectable Pancreatic Cancer

Inclusion Criteria: * Patients must have histologically confirmed pancreatic cancer (excluding islets) (not otherwise specified \[NOS\]) (Medical Dictionary for Regulatory Activities \[MEDDRA\] code: 10033612). * Patients must have unresectable locally advanced pancreatic cancer as determined by a multidisciplinary tumor board applying National Comprehensive Cancer Network (NCCN) version (v)2.2021 criteria or as surgically determined during failed resection attempt. * Patients must have had prior first-line chemotherapy for this cancer for at least 16 weeks without clinical, biochemical, or radiologic progression. There should be a washout of at least 2 weeks from first-line chemotherapy and start of therapy on clinical trial. * Age \>= 18 years. Because no dosing or adverse event data are currently available on the use of durvalumab and olaparib in combination with radiation in patients \< 18 years of age, children are excluded from this study. * Body weight \> 30 kg. * Eastern Cooperative Oncology Group (ECOG) performance status =\< 1 (Karnofsky \>= 70%). * Hemoglobin \>= 9.0 g/dL without blood transfusion in last 4 weeks (within 2 weeks of enrollment). * Absolute neutrophil count \>= 1,500/mcL (within 2 weeks of enrollment). * Platelets \>= 100,000/mcL (within 2 weeks of enrollment). * Total bilirubin =\< 1.5 x institutional upper limit of normal (ULN) (within 2 weeks of enrollment). * Aspartate aminotransferase (AST) (serum glutamic oxaloacetic transaminase \[SGOT\])/ala…