ESK981 and Nivolumab for the Treatment of Metastatic Castration Resistant Prostate Cancer

Inclusion Criteria: * Eastern Cooperative Group (ECOG) performance status =\< 1 * Recovery to baseline or =\< grade 1 Common Terminology Criteria for Adverse Events (CTCAE) version (v.)5 from toxicities related to any prior treatments, unless adverse event (AE)(s) are clinically non-significant and/or stable on supportive therapy * Absolute neutrophil count (ANC) \>= 1.5 K/mm\^3 * Hemoglobin (Hgb) \>= 9 g/dL * Platelets (Plt) \>= 100,000/mm\^3 * Serum creatinine =\< 1.5 times the upper limit of normal OR creatinine clearance \> 30 mL/min by Cockcroft-Gault formula * Total bilirubin =\< 1.5 x upper limit of normal (ULN) * Aspartate aminotransferase (AST) =\< 2.5 x ULN (=\< 5 x ULN with known hepatic metastases) * Alanine aminotransferase (ALT) =\< 2.5 x ULN (=\< 5 x ULN with known hepatic metastases) * Prothrombin time (PT) and activated partial thromboplastin time (aPTT) levels =\< 1.5 x ULN (If patient is receiving anticoagulation that is expected to alter these levels, should be in targeted therapeutic range for that agent) * Patient must have progressive disease while receiving androgen deprivation therapy (ADT) defined by any one of the following as per the PCWG3 criteria for PSA, measurable or non-measurable (bone) disease and must have a castrate serum testosterone level (i.e. =\< 50 ng/dL) at screening: * PSA: At least two consecutive rises in serum PSA, obtained at a minimum of 1-week intervals, with the final value \>= 2.0 ng/mL * Measurable disease (by Respons…