fNIRS-STCT: A Novel Hybrid Spatial and Temporal CNN and Transformer Network for fNIRS Signal Classification

Clinical Snapshot

Key Findings

• P Value: Not reported

• Effect Size: Accuracy improvements of 1-3% over second-best performing model

• Primary Outcome: Classification accuracy across three evaluation schemes

• Nnt Or Sensitivity: Subject-dependent: 83.29%, Subject-semi-dependent: 81.73%, Subject-independent: 79.60%

• Confidence Interval: Not reported

Clinical Application

Requires fNIRS equipment and computational resources for model implementation, but non-invasive nature supports clinical adoption Relevant for Australian neurology and rehabilitation services; fNIRS technology available through TGA-approved devices; potential applications in NDIS-funded rehabilitation programs Patients requiring brain-computer interface applications, motor rehabilitation assessment, or neural activity monitoring

Abstract

Functional near-infrared spectroscopy (fNIRS) is a non-invasive brain imaging method that detects neural activity by monitoring blood oxygen levels and circulation changes in the brain. Some current fNIRS classification models are simply adapted from deep learning models such as Transformer and fail to effectively leverage both temporal and spatial information from fNIRS. In this paper, we develop a hybrid deep learning model for fNIRS classification, fusing spatial convolutional neural network (CNN), a temporal CNN, and Transformer, named as fNIRS-STCT. Primarily, the architecture employs dual large convolutional kernels - one for temporal feature extraction and another for spatial feature extraction. Second, spatial and temporal CNNs are respectively integrated with Transformer by feeding CNN-extracted features as input to Transformer and then concatenating CNN and Transformer features. Third, spatial and temporal features are fused by element-wise addition. Finally, we exploit cross-entropy as the loss function, which integrates both online label smoothing and flooding strategies to avoid overfitting. The model's effectiveness is demonstrated through testing on two public fNIRS datasets. Across subject-dependent, subject-semi-dependent, and subject-independent evaluation schemes, fNIRS-STCT model consistently outperforms other compared models for all given performance metrics. Specifically, on the unilateral finger- and foot-tapping dataset, our model achieves accuracies of 83.29%, 81.73%, and 79.60% across three evaluation schemes, surpassing the second-best model by margins of 3%, 2%, and 1%, respectively. The findings underscore fNIRS-STCT's effectiveness in cross-subject mental state and brain activity interpretation.

References

1. 1.Kuang, L.-D., Wang, Y.-X., Li, W., & Gui, Y. (2026). fNIRS-STCT: A Novel Hybrid Spatial and Temporal CNN and Transformer Network for fNIRS Signal Classification. Brain Topography. https://doi.org/10.1109/TIP.2021.3089942