Erlotinib Hydrochloride in Reducing Duodenal Polyp Burden in Patients With Familial Adenomatous Polyposis at Risk of Developing Colon Cancer

Inclusion Criteria: * PRE-REGISTRATION INCLUSION * Diagnosis of familial adenomatous polyposis (FAP) or attenuated familial adenomatous polyposis (AFAP), defined as at least one of the following: * Genetic diagnosis with confirmed APC mutation (Clinical Laboratory Improvement Act \[CLIA\] certified lab or research testing) * Obligate carrier * Clinical diagnosis of classic FAP with \>= 100 colorectal adenomas status post colectomy and a family history of FAP * Clinical diagnosis of FAP, based on personal and family history; Note: This criterion requires documented review and agreement from either the study chair or the Cancer Prevention Network (CPN) lead investigator * Ability to understand and the willingness to sign a written informed consent document * Willing to discontinue taking nonsteroidal anti-inflammatory drugs (NSAIDS) for 30 days prior to initiation of and during intervention; exception: use of =\< 81 mg daily or =\< 650 mg weekly aspirin is allowed * Willing to discontinue smoking for the duration of study intervention * Willing to provide mandatory biospecimens as specified in the protocol * REGISTRATION INCLUSION * Eastern Cooperative Oncology Group (ECOG) performance status =\< 1 * Leukocytes (white blood cells \[WBC\]) \>= 3,000/uL (\>= 2,500/uL for African-American participants) * Platelet count \>= 100 x 10\^9/L * Hemoglobin \>= 11.5 g/dL * Total bilirubin =\< 1.5 x institutional upper limit of normal (ULN) * Alkaline phosphatase =\< 1.5 x instit…