The use of recycled concrete aggregates (RCAs) in lieu of natural aggregates improves the sustainability of the built environment. Barriers to the use of RCA include its variable composition, including the residual mortar content (RMC), chemical composition, and its potential to contain contaminants, which can negatively affect the properties of concrete or present environmental concerns. In this study, a rapid, economical method to estimate the RMC and provide the chemical characterization of RCA was developed using a portable handheld X-ray fluorescence (PHXRF) device. Models were developed using reference tests (RMC test based on the thermal shock method and chemical composition from whole-rock analysis) to correlate PHXRF results to measured values. The PHXRF shows strong potential for estimating the RMC and chemical composition of RCA. Paired with locally calibrated reference samples, the test method could be used in laboratory or field applications to characterize RCA and increase its use in bound and unbound applications.