A machine learning approach to support the identification of architectural technical debt
Published
:Issue
:Section
:License
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
How to Cite
Dimensions Metrics
solution architecture, heterogeneous artifacts, technical debt in architecture, machine learning
Decisions made by architects to favor short-term objectives and to the possible detriment of long-term software quality are known as Architectural Technical Debt. This type of technical debt is difficult to identify because it is related to quality attributes not visible to the customer, such as the maintainability and evolvability of the system. Objective: For this reason, in this paper, a supervised machine learning model is proposed to support the identification of technical debt in architecture that is located in the design stage of the architecture. Methods: This proposal relies on the information collected from the artifacts produced during the architecture design to build a dataset to evaluate different supervised learning algorithms and thus establish the one that offers the best accuracy. The identification of the technical debt, within the framework of this proposal and unlike those proposed in the literature, does not consider the source code. Results: The performance of the model was evaluated through a real industry case and allowed discovering that both accuracy and recall present acceptable levels. Conclusions: The data used to train the model, while appropriate, has room for further improvement. This approach will allow architects to support the identification of conscious and unconscious ATD injected into their architectures.
A. Martini, E. Sikander y N. Madlani, “A semi-automated framework for the identification and estimation of Architectural Technical Debt”, Information and Software Technology, vol. 93, pp. 264–279, January 2018. https://doi.org/10.1016/j.infsof.2017.08.005. DOI: https://doi.org/10.1016/j.infsof.2017.08.005
N. Rios, R. Spínola, M. Mendonça y C. Seaman, “The practitioners’ point of view on the concept of technical debt and its causes and consequences: a design for a global family of industrial surveys and its first results from Brazil”, Empirical Software Engineering, vol. 25, ed. 5, pp. 3216-3287, September 2020. https://doi.org/10.1007/s10664-020-09832-9. DOI: https://doi.org/10.1007/s10664-020-09832-9
A. Martini, J. Bosch y M. Chaudron, “Architecture Technical Debt: Understanding Causes and a Qualitative Model”, in Proceedings of the 40th EUROMICRO Conference on Software Engineering and Advanced Applications, pp. 85-92, 2014. https://doi.org/10.1109/SEAA.2014.65. DOI: https://doi.org/10.1109/SEAA.2014.65
N. Ernst, S. Bellomo, I. Ozkaya, R. Nord y I. Gorton, “Measure it? Manage it? Ignore it? software practitioners and technical debt”, in Proceedings of the 10th Joint Meeting on Foundations of Software Engineering (ESEC/FSE 2015), pp. 50-60, 2015. https://doi.org/10.1145/2786805.2786848. DOI: https://doi.org/10.1145/2786805.2786848
Z. Li, P. Liang y P. Avgeriou, “Architectural Technical Debt Identification Based on Architecture Decisions and Change Scenarios”, in Proceedings of the 12th Working IEEE/IFIP Conference on Software Architecture, pp. 65-74, 2015. http://doi.org/10.1109/WICSA.2015.19 DOI: https://doi.org/10.1109/WICSA.2015.19
A. Martini, T. Besker y J. Bosch, “The Introduction of Technical Debt Tracking in Large Companies”, in Proceedings of the 23rd Asia-Pacific Software Engineering Conference (APSEC), pp. 161-168, 2016. http://doi.org/10.1109/APSEC.2016.032. DOI: https://doi.org/10.1109/APSEC.2016.032
R. L. Nord, I. Ozkaya, P. Kruchten and M. Gonzalez-Rojas, “In Search of a Metric for Managing Architectural Technical Debt,” in Proceedings of the Joint Working IEEE/IFIP Conference on Software Architecture and European Conference on Software Architecture, pp. 91-100, 2012 http://doi.org/10.1109/WICSA-ECSA.212.17. DOI: https://doi.org/10.1109/WICSA-ECSA.212.17
Zengyang Li, Paris Avgeriou y Peng Liang, “A systematic mapping study on technical debt and its management”, Journal of Systems and Software, vol. 101, pp. 193-220, 2015. https://doi.org/10.1016/j.jss.2014.12.027 DOI: https://doi.org/10.1016/j.jss.2014.12.027
J. A. Díaz-Pace, A. Tommasel y D. Godoy, “Towards Anticipation of Architectural Smells Using Link Prediction Techniques”, in Proceedings of the IEEE 18th International Working Conference on Source Code Analysis and Manipulation (SCAM), pp. 62-71, 2018. http://goi.org/10.1109/SCAM.2018.00015. DOI: https://doi.org/10.1109/SCAM.2018.00015
M. A. de Freitas Farias, M. G. de Mendonça Neto, A. B. d. Silva y R. O. Spínola, “A Contextualized Vocabulary Model for identifying technical debt on code comments”, in Proceedings of the IEEE 7th International Workshop on Managing Technical Debt (MTD), pp. 25-32, 2015. http://doi.org/10.1109/MTD.2015.7332621. DOI: https://doi.org/10.1109/MTD.2015.7332621
D. Tsoukalas, D. Kehagias, M. Siavvas y A. Chatzigeorgiou, “Technical debt forecasting: An empirical study on open-source repositories”, Journal of Systems and Software, vol. 170, pp. 1-35, 2020. https://doi.org/10.1016/j.jss.2020.110777 DOI: https://doi.org/10.1016/j.jss.2020.110777
K. Dai y P. Kruchten, “Detecting Technical Debt through Issue Trackers”, in Proceedings of the 5th International Workshop on Quantitative Approaches to Software Quality, pp. 59-65, 2017. https://dx.doi.org/10.14288/1.0374920.
A. Tsintzira, E. M. Arvanitou, A. Ampatzoglou, A. Chatzigeorgiou, “Applying Machine Learning in Technical Debt Management: Future Opportunities and Challenges”, Communications in Computer and Information Science, vol. 1266, pp. 53-67, 2020. https://doi.org/10.1007/978-3-030-58793-2_5 DOI: https://doi.org/10.1007/978-3-030-58793-2_5
J. Musil et al., “Continuous Architectural Knowledge Integration: Making Heterogeneous Architectural Knowledge Available in Large-Scale Organizations”, in Proceedings of the IEEE International Conference on Software Architecture (ICSA), pp. 189-192, 2017. https://doi.org/10.1109/ICSA.2017.28 DOI: https://doi.org/10.1109/ICSA.2017.28
C. Atkinson y T. Kuhne, “Model-driven development: a metamodeling foundation”, IEEE Software, vol. 20, no. 5, pp. 36-41, September 2003. https://doi.org/10.1109/MS.2003.1231149. DOI: https://doi.org/10.1109/MS.2003.1231149
H. Jonkers, E. Proper, M. M. Lankhorst, D. A. C. Quartel y M. Iacob, “ArchiMate(R) for Integrated Modelling Throughout the Architecture Development and Implementation Cycle”, in Proceedings of the IEEE 13th Conference on Commerce and Enterprise Computing, pp. 294-301, 2011. https://doi.org/10.1109/CEC.2011.52. DOI: https://doi.org/10.1109/CEC.2011.52
M. Nygard, “Documenting Architecture Decisions”, November 2011. [Online]. Disponible en: https://cognitect.com/blog/2011/11/15/documenting-architecture-decisions.html
R. N. Taylor, N. Medvidovic y E. M. Dashofy. Software Architecture: Foundations, Theory, and Practice. Wiley India Pvt. Limited. 2010. DOI: https://doi.org/10.1145/1810295.1810435
R. Verdecchia, I. Malavolta y P. Lago, “Architectural Technical Debt Identification: The Research Landscape”, in Proceedings of the IEEE/ACM International Conference on Technical Debt (TechDebt), pp. 11-20, 2018. https://doi.org/10.1145/3194164.3194176 DOI: https://doi.org/10.1145/3194164.3194176
T. Besker, A. Martini y J. Bosch, “Managing architectural technical debt: A unified model and systematic literature review”, Journal of Systems and Software, vol. 135, pp. 1-16, 2018. https://doi.org/10.1016/j.jss.2017.09.025 DOI: https://doi.org/10.1016/j.jss.2017.09.025
H. C. Wu, R. W. Pong Luk, K. F. Wong y K. L. Kwok, “Interpreting TF-IDF term weights as making relevance decisions”, ACM Transactions on Information Systems, vol. 26, pp. 1-37, June 2008. https://doi.org/10.1145/1361684.1361686 DOI: https://doi.org/10.1145/1361684.1361686
Decisions made by architects to favor short-term objectives and to the possible detriment of long-term software quality are known as Architectural Technical Debt. This type of technical debt is difficult to identify because it is related to quality attributes not visible to the customer, such as the maintainability and evolvability of the system. Objective: For this reason, in this paper, a supervised machine learning model is proposed to support the identification of technical debt in architecture that is located in the design stage of the architecture. Methods: This proposal relies on the information collected from the artifacts produced during the architecture design to build a dataset to evaluate different supervised learning algorithms and thus establish the one that offers the best accuracy. The identification of the technical debt, within the framework of this proposal and unlike those proposed in the literature, does not consider the source code. Results: The performance of the model was evaluated through a real industry case and allowed discovering that both accuracy and recall present acceptable levels. Conclusions: The data used to train the model, while appropriate, has room for further improvement. This approach will allow architects to support the identification of conscious and unconscious ATD injected into their architectures.
A. Martini, E. Sikander y N. Madlani, “A semi-automated framework for the identification and estimation of Architectural Technical Debt”, Information and Software Technology, vol. 93, pp. 264–279, January 2018. https://doi.org/10.1016/j.infsof.2017.08.005. DOI: https://doi.org/10.1016/j.infsof.2017.08.005
N. Rios, R. Spínola, M. Mendonça y C. Seaman, “The practitioners’ point of view on the concept of technical debt and its causes and consequences: a design for a global family of industrial surveys and its first results from Brazil”, Empirical Software Engineering, vol. 25, ed. 5, pp. 3216-3287, September 2020. https://doi.org/10.1007/s10664-020-09832-9. DOI: https://doi.org/10.1007/s10664-020-09832-9
A. Martini, J. Bosch y M. Chaudron, “Architecture Technical Debt: Understanding Causes and a Qualitative Model”, in Proceedings of the 40th EUROMICRO Conference on Software Engineering and Advanced Applications, pp. 85-92, 2014. https://doi.org/10.1109/SEAA.2014.65. DOI: https://doi.org/10.1109/SEAA.2014.65
N. Ernst, S. Bellomo, I. Ozkaya, R. Nord y I. Gorton, “Measure it? Manage it? Ignore it? software practitioners and technical debt”, in Proceedings of the 10th Joint Meeting on Foundations of Software Engineering (ESEC/FSE 2015), pp. 50-60, 2015. https://doi.org/10.1145/2786805.2786848. DOI: https://doi.org/10.1145/2786805.2786848
Z. Li, P. Liang y P. Avgeriou, “Architectural Technical Debt Identification Based on Architecture Decisions and Change Scenarios”, in Proceedings of the 12th Working IEEE/IFIP Conference on Software Architecture, pp. 65-74, 2015. http://doi.org/10.1109/WICSA.2015.19 DOI: https://doi.org/10.1109/WICSA.2015.19
A. Martini, T. Besker y J. Bosch, “The Introduction of Technical Debt Tracking in Large Companies”, in Proceedings of the 23rd Asia-Pacific Software Engineering Conference (APSEC), pp. 161-168, 2016. http://doi.org/10.1109/APSEC.2016.032. DOI: https://doi.org/10.1109/APSEC.2016.032
R. L. Nord, I. Ozkaya, P. Kruchten and M. Gonzalez-Rojas, “In Search of a Metric for Managing Architectural Technical Debt,” in Proceedings of the Joint Working IEEE/IFIP Conference on Software Architecture and European Conference on Software Architecture, pp. 91-100, 2012 http://doi.org/10.1109/WICSA-ECSA.212.17. DOI: https://doi.org/10.1109/WICSA-ECSA.212.17
Zengyang Li, Paris Avgeriou y Peng Liang, “A systematic mapping study on technical debt and its management”, Journal of Systems and Software, vol. 101, pp. 193-220, 2015. https://doi.org/10.1016/j.jss.2014.12.027 DOI: https://doi.org/10.1016/j.jss.2014.12.027
J. A. Díaz-Pace, A. Tommasel y D. Godoy, “Towards Anticipation of Architectural Smells Using Link Prediction Techniques”, in Proceedings of the IEEE 18th International Working Conference on Source Code Analysis and Manipulation (SCAM), pp. 62-71, 2018. http://goi.org/10.1109/SCAM.2018.00015. DOI: https://doi.org/10.1109/SCAM.2018.00015
M. A. de Freitas Farias, M. G. de Mendonça Neto, A. B. d. Silva y R. O. Spínola, “A Contextualized Vocabulary Model for identifying technical debt on code comments”, in Proceedings of the IEEE 7th International Workshop on Managing Technical Debt (MTD), pp. 25-32, 2015. http://doi.org/10.1109/MTD.2015.7332621. DOI: https://doi.org/10.1109/MTD.2015.7332621
D. Tsoukalas, D. Kehagias, M. Siavvas y A. Chatzigeorgiou, “Technical debt forecasting: An empirical study on open-source repositories”, Journal of Systems and Software, vol. 170, pp. 1-35, 2020. https://doi.org/10.1016/j.jss.2020.110777 DOI: https://doi.org/10.1016/j.jss.2020.110777
K. Dai y P. Kruchten, “Detecting Technical Debt through Issue Trackers”, in Proceedings of the 5th International Workshop on Quantitative Approaches to Software Quality, pp. 59-65, 2017. https://dx.doi.org/10.14288/1.0374920.
A. Tsintzira, E. M. Arvanitou, A. Ampatzoglou, A. Chatzigeorgiou, “Applying Machine Learning in Technical Debt Management: Future Opportunities and Challenges”, Communications in Computer and Information Science, vol. 1266, pp. 53-67, 2020. https://doi.org/10.1007/978-3-030-58793-2_5 DOI: https://doi.org/10.1007/978-3-030-58793-2_5
J. Musil et al., “Continuous Architectural Knowledge Integration: Making Heterogeneous Architectural Knowledge Available in Large-Scale Organizations”, in Proceedings of the IEEE International Conference on Software Architecture (ICSA), pp. 189-192, 2017. https://doi.org/10.1109/ICSA.2017.28 DOI: https://doi.org/10.1109/ICSA.2017.28
C. Atkinson y T. Kuhne, “Model-driven development: a metamodeling foundation”, IEEE Software, vol. 20, no. 5, pp. 36-41, September 2003. https://doi.org/10.1109/MS.2003.1231149. DOI: https://doi.org/10.1109/MS.2003.1231149
H. Jonkers, E. Proper, M. M. Lankhorst, D. A. C. Quartel y M. Iacob, “ArchiMate(R) for Integrated Modelling Throughout the Architecture Development and Implementation Cycle”, in Proceedings of the IEEE 13th Conference on Commerce and Enterprise Computing, pp. 294-301, 2011. https://doi.org/10.1109/CEC.2011.52. DOI: https://doi.org/10.1109/CEC.2011.52
M. Nygard, “Documenting Architecture Decisions”, November 2011. [Online]. Disponible en: https://cognitect.com/blog/2011/11/15/documenting-architecture-decisions.html
R. N. Taylor, N. Medvidovic y E. M. Dashofy. Software Architecture: Foundations, Theory, and Practice. Wiley India Pvt. Limited. 2010. DOI: https://doi.org/10.1145/1810295.1810435
R. Verdecchia, I. Malavolta y P. Lago, “Architectural Technical Debt Identification: The Research Landscape”, in Proceedings of the IEEE/ACM International Conference on Technical Debt (TechDebt), pp. 11-20, 2018. https://doi.org/10.1145/3194164.3194176 DOI: https://doi.org/10.1145/3194164.3194176
T. Besker, A. Martini y J. Bosch, “Managing architectural technical debt: A unified model and systematic literature review”, Journal of Systems and Software, vol. 135, pp. 1-16, 2018. https://doi.org/10.1016/j.jss.2017.09.025 DOI: https://doi.org/10.1016/j.jss.2017.09.025
H. C. Wu, R. W. Pong Luk, K. F. Wong y K. L. Kwok, “Interpreting TF-IDF term weights as making relevance decisions”, ACM Transactions on Information Systems, vol. 26, pp. 1-37, June 2008. https://doi.org/10.1145/1361684.1361686 DOI: https://doi.org/10.1145/1361684.1361686