Ipilimumab and Sargramostim in Treating Patients With Metastatic Prostate Cancer

Inclusion Criteria: * Histologically confirmed prostate cancer * Metastatic disease * Progressive disease after prior androgen deprivation as defined by at least 1 of the following criteria: * Patients with measurable disease must have an increase of at least 20% in the sum of the longest diameter of target lesions OR the appearance of 1 or more new lesions * Patients with nonmeasurable disease must have a positive bone scan and a prostate-specific antigen (PSA) level of at least 5 ng/mL, which has risen on at least 2 successive occasions at least 2 weeks apart\* * At least 1 PSA level must be obtained at least 4 weeks after flutamide (6 weeks after bicalutamide or nilutamide) * Testosterone no greater than 50 ng/dL * Patients with no prior orchiectomy must continue luteinizing hormone-releasing hormone agonist therapy * No history or radiologic evidence of CNS metastases * Performance status - ECOG 0-2 * At least 12 weeks * Absolute neutrophil count greater than 1,500/mm\^3 * Platelet count at least 100,000/mm\^3 * Hemoglobin at least 8 g/dL * Bilirubin less than 1.5 times upper limit of normal (ULN) * SGOT and SGPT no greater than 2.5 times ULN * Creatinine no greater than 1.5 times ULN * No significant cardiovascular disease * No New York Heart Association class III or IV congestive heart failure * No active angina pectoris * No myocardial infarction within the past 6 months * Not pregnant or nursing * Fertile patients must use effective contraception prio…