A hybrid deep learning model for user story effort estimation
Clinical Snapshot
PICO Framework
| P — Population | Agile software development projects requiring effort estimation of user stories (textual natural language requirements expressed as Scrum story points) |
| I — Intervention | Hybrid deep learning model combining BERT (Bidirectional Encoder Representations from Transformers) encoder with Long Short-Term Memory (LSTM) networks for automated effort estimation |
| C — Comparator | Multiple conventional machine learning baseline models (unspecified in abstract; implied to include traditional ML classifiers and regression approaches) |
| O — Outcomes | Prediction accuracy of story point effort estimation measured by Mean Absolute Error (MAE), Root Mean Square Error (RMSE), R² coefficient of determination, and Fibonacci-mapped classification accuracy |
Bottom Line
This paper presents a hybrid BERT-LSTM deep learning model for automated effort estimation of agile user stories, evaluated on a curated dataset of 4,079 instances. The model achieves an MAE of 0.6481 story points, R² of 0.6581, and 72% accuracy when outputs are mapped to Fibonacci story point values — statistically superior to conventional ML baselines by Wilcoxon test. The study is methodologically sound in its design science approach and multi-metric evaluation, but has notable limitations: no confidence intervals are reported, confounding by team and domain context is unaddressed, and the ~41% data attrition during preprocessing raises selection bias concerns. The model has not been validated on independent real-world industrial datasets, and the absence of explainability features limits responsible deployment. For software engineering practitioners, this represents a promising proof-of-concept rather than a deployment-ready solution. Independent external validation, XAI integration, and user acceptance testing are essential next steps before operational adoption. This work is not directly applicable to clinical practice but may interest health informatics and digital health project management teams operating in agile delivery environments.
Key Findings
P Value: p < 0.05 (Wilcoxon Signed-Rank Test for pairwise comparisons between proposed model and baselines)
Effect Size: MAE = 0.6481 story points; RMSE = 1.4559 story points; R² = 0.6581 (explaining ~65.8% of variance in effort). Described as 'huge improvement' over conventional methods — magnitude of improvement over baselines not quantified in abstract
Primary Outcome: Effort estimation accuracy of the hybrid BERT-LSTM model on a curated dataset of 4,079 agile user stories, evaluated against conventional ML baselines
Nnt Or Sensitivity: Fibonacci-mapped classification accuracy: 72% (mapping continuous model outputs to discrete Scrum story point values on the Fibonacci scale). Equivalent metric type: multi-class classification accuracy for a predictive/decision-support model
Confidence Interval: Not reported for any metric — a significant methodological gap
Clinical Application
The web-based decision support system operationalisation suggests moderate implementation feasibility. However, practical adoption requires integration with existing project management tools (e.g., Jira, Azure DevOps), team training, and validation within the specific organisational context. The absence of explainability features may limit acceptance among experienced practitioners who rely on transparent reasoning. This study has no direct relevance to Australian clinical healthcare practice, PBS listings, TGA regulatory frameworks, or RACGP guidelines. It is a software engineering / information systems study. In the Australian context, potential indirect relevance exists for: (1) health informatics teams managing agile development of clinical software systems; (2) digital health project managers in state and federal health departments; (3) Australian Digital Health Agency initiatives involving agile delivery of health IT infrastructure. The model has not been validated in Australian software development contexts and would require local validation before adoption. Agile software development teams using Scrum methodology who require automated or decision-supported effort estimation for user stories. Most relevant to teams managing large backlogs, distributed teams, or organisations seeking to standardise estimation practices across projects.
Abstract
Accurate effort estimation of user stories is a key challenge in agile software development due to both subjectivity and the complexity of natural language requirements. This paper proposes a hybrid Deep Learning (DL) model for data driven effort estimation using large scale textual data and advanced semantic modeling. One of the significant contributions is the development of a dataset of 6,956 user stories which was collected from many heterogeneous sources and then meticulously cleaned and refined to 4,079 high quality instances, by using a systematic preprocessing and expert validation. Following a Design Science Research (DSR) methodology, a hybrid model integrating a pre-trained Bidirectional Encoder Representations from Transformers (BERT) encoder with a Long Short-Term Memory (LSTM) is developed to capture both contextual semantics and sequential dependencies in user story description. The DL model is evaluated against multiple Machine Learning (ML) baselines using a robust multi-metric framework. Experimental findings show the superior performance of the proposed model with a Mean Absolute Error (MAE) = 0.6481, Root Mean Square Error (RMSE) = 1.4559 and = 0.6581, which is a huge improvement over the conventional methods. To ensure practical relevance in discrete Scrum planning, the continuous model outputs were mapped to the standard Fibonacci sequence, achieving a classification accuracy of 72%. To guarantee the consistency of performance improvements, the statistical validation is done with the Wilcoxon Signed-Rank Test to show that the improvements are significant (p < 0.05). Moreover, the model is operationalized as web based decision support system to enable real time estimation in agile software development. The results reveal the efficacy of using large scale curated datasets alongside hybrid DL model to reduce estimation bias and boost predictive quality to deliver a scalable and strong solution for intelligent agile project management. Future work will focus on integrating eXplainable AI (XAI) techniques and validating the model across real world industrial datasets to further enhance transparency and generalizability.
References
- 1.Malik, S., Hamid, M., & Saleem, M. (2026). A hybrid deep learning model for user story effort estimation. PLOS ONE. https://doi.org/10.1371/journal.pone.0353348
Related Research
Journal of medical Internet research
Advancing Human-Centered AI in Clinical Decision Support: Sociocognitive Human-in-the-Loop Study in HIV Care
2 Aug 2026
Neurological research
Federated deep learning model for epilepsy seizure detection using electroencephalogram (EEG) signal
2 Aug 2026
Neural networks : the official journal of the International Neural Network Society
MFBD: Model-free backdoor defense based on vision-language pre-trained models
2 Aug 2026
This content is for educational purposes for healthcare professionals only and does not constitute clinical advice. Clinical decisions should be based on individual patient assessment, current guidelines, and appropriate specialist consultation. Editorial Standards · Privacy Policy · Terms of Service