177Lu-PSMA-617 and Pembrolizumab in Treating Patients With Metastatic Castration-Resistant Prostate Cancer

Inclusion Criteria: * The subject is able and willing to comply with study procedures and provide signed and dated informed consent * Histologically confirmed prostate adenocarcinoma. De novo small cell neuroendocrine prostate cancer will not be allowed due to putative lower PSMA expression in this tumor subtype. Treatment-emergent small cell neuroendocrine prostate cancer detected in metastatic tumor biopsy is not an exclusion * A minimum of three PSMA-avid lesions on baseline 68Ga-PSMA-11 PET, with positive lesions defined as those with maximum standardized uptake value (SUVmax) values greater than liver. * Progressive metastatic castration-resistant prostate cancer by Prostate Cancer Working Group (PCWG)3 criteria at the time of study entry * Castrate level of serum testosterone at study entry (\< 50 ng/dL). Participants without prior bilateral orchiectomy are required to remain on luteinizing hormone-releasing hormone (LHRH) analogue treatment for duration of study * Prior progression on at least one second generation androgen signaling inhibitor including abiraterone, apalutamide, darolutamide, and/or enzalutamide * Absolute neutrophil count \> 1.5 x 10\^9/L * Hemoglobin \> 9.0 g/dL * Platelet count \> 100,000/microliter * Serum creatinine =\< 1.5 x upper limit of normal (ULN) or estimated glomerular filtration rate (GFR) \> 50 ml/min by Cockcroft-Gault or 24 hour urine collection * Total bilirubin =\< 1.5 x ULN. In participants with known or suspected Gilbert's disease…