A novel temporal adaptive fuzzy neural network for facial feature based fatigue assessment

When engaging in activities such as using video display terminals, driving, or sports, effective fatigue monitoring is crucial. Nevertheless, obtaining biological information through contact devices may interfere with normal activities. Deterministic expressions in mainstream machine learning have limitations in reflecting the continuous, dynamic, and fuzzy process of fatigue status. Additionally, fatigue is a cumulative process where the previous state impacts the assessment of the current one. To address these needs and challenges, this paper proposes a novel model based on facial videos called the temporal adaptive fuzzy neural network (TAFNN) for fatigue assessment. TAFNN utilizes an adaptive fuzzy neural network as its foundation and employs causal and dilated convolutions for the time information processing method to achieve time series extraction and fatigue assessment. It leverages facial physiological and motion features to minimize interference during assessment. Furthermore, TAFNN introduces a new calculation method for rule antecedents to enhance stability. Experimental results demonstrate that TAFNN effectively captures the cumulation and fuzziness of state changes, outperforming other widely adopted methods in both assessment ability and runtime performance. The improved rule antecedent calculation method successfully mitigates the issue of multiple memberships rapidly approaching zero after combination. Through 1000 repeated experiments, the enhanced method reduces TAFNN’s instability by 81.58%.