Construction workers are exposed to heat stress risks due to the combined effects of hot and humid weather conditions (HHWCs) and physically demanding work. A real-time activity intensity identification (AII) is required to measure the impact of HHWCs using a nonintrusive approach. This research developed a real-time AII system based on computer vision analysis (CVA). It then combined the CVA system with real-time video recordings to approximate workers’ activity intensity (AI) levels alongside HHWC records. A fundamental activities matrix was developed to build a list of measurable and identifiable features of site activities. These features were used to identify and link different postures to a crew’s AI and safety status within a given context. In real-site conditions, the AII system instantly and unobtrusively approximated workers’ AI and safety status under HHWCs. The system showed high detection performance with competitive deployment time, cost, and effort, outperforming previous related models. The results showed that formwork and steelwork are mostly moderate activities; however, moderate AI and HHWCs can create heat stress and fatigue and significantly affect workers’ safety, resulting in heat-related injuries and accidents. This research gives researchers and practitioners insight into the challenges associated with measurement methods and solving practical site measurement issues. This research promotes innovative methods for real-site measurements and contributes to knowledge in the field of safety and productivity in the construction industry by employing new, innovative CVA technology. This technology has applications in the industry by deploying a practical tool that could support aligned improvement in the safety and productivity of construction workers working under HHWCs.