Deep learning-based bimodal speech and facial expression recognition of miners' unsafe emotions
Clinical Snapshot
PICO Framework
| P — Population | Miners working in underground or surface mining operations |
| I — Intervention | Deep learning-based bimodal emotion recognition system using speech and facial expression analysis |
| C — Comparator | Not clearly specified - appears to be comparison against baseline emotion recognition methods |
| O — Outcomes | Recognition accuracy of unsafe emotions (85.56% mean accuracy), potential for real-world mining safety applications |
Bottom Line
This technology development study presents a bimodal emotion recognition system for mining safety applications, achieving 85.56% accuracy in recognizing unsafe emotions through speech and facial expression analysis. While the technical approach appears sound, the study lacks crucial methodological details, statistical validation, and clinical outcome measures. The absence of confidence intervals, comparison groups, and safety outcome data significantly limits the evidence quality. For Australian mining operations, this technology would require extensive local validation, regulatory approval, and integration with existing safety frameworks before implementation. The preliminary nature of field testing and unclear generalizability to different mining contexts and populations further limit immediate clinical application. Healthcare professionals should view this as early-stage technology development rather than validated safety intervention.
Key Findings
P Value: Not reported
Effect Size: 85.56% accuracy
Primary Outcome: Mean recognition accuracy of unsafe emotions
Nnt Or Sensitivity: Sensitivity and specificity for individual emotion categories not reported
Confidence Interval: Not reported
Clinical Application
Requires significant technological infrastructure and may face implementation challenges in harsh mining environments Limited direct applicability - would require validation in Australian mining contexts, consideration of WorkSafe regulations, and integration with existing safety management systems Mining workforce in underground and surface operations, particularly relevant for high-risk environments
Abstract
Under the influence of unsafe emotions, miners' ability to perceive risks is hindered, which can easily lead to decision-making errors and safety accidents. To recognize unsafe emotions exhibited by miners during operations, this study proposes a deep learning-based bimodal framework that integrates speech and facial expression features. A convolutional neural network (CNN) combined with a bidirectional long short-term memory (Bi-LSTM) network is employed to model local spectral patterns and temporal dependencies in speech signals, and ShuffleNet-V2 is used to capture deep facial features. In addition, three feature enhancement strategies are proposed to improve the generalization ability of the model. By constructing a dataset containing five categories of miners' unsafe emotions for network training, the model achieves a mean recognition accuracy of 85.56%. Furthermore, we conducted a preliminary field test of the bimodal model in a real mining environment. The results provide preliminary evidence of its potential applicability in real-world mining conditions.
References
- 1.Lu, Y., Zhao, Z., Yan, L., & Shi, X. (2026). Deep learning-based bimodal speech and facial expression recognition of miners' unsafe emotions. PLoS One. https://doi.org/10.1371/journal.pone.0348906
Related Research
JMIR mHealth and uHealth
Machine Learning Frameworks for Wearable-Based Stress Modeling in Naturalistic Settings: Scoping Review
1 Aug 2026
Lasers in medical science
Effect of photobiomodulation on trapezius myofascial pain: a randomized, sham-controlled, double-blind clinical trial.
28 July 2026
Journal of neural engineering
Multimodal quantification of cognitive load using a printed wearable facial bio-potential system
23 July 2026
This content is for educational purposes for healthcare professionals only and does not constitute clinical advice. Clinical decisions should be based on individual patient assessment, current guidelines, and appropriate specialist consultation. Editorial Standards · Privacy Policy · Terms of Service