| 3/2023 - 6 |
ROSMutation: Mutation Based Automated Testing for ROS Compatible Robotic SoftwareYAYAN, U.   |
Extra paper information in    |
| Click to see author's profile in SCOPUS, IEEE Xplore, Web of Science |
Download PDF  (1,383 KB) | Citation | Downloads: 520 | Views: 1,053 |
Author keywords
robot operating system, mutation testing, software robustness, fault injection, automated software testing
References keywords
software(16), testing(12), mutation(8), test(7), systems(6), robot(6), robotics(4), robotic(4), analysis(4)
Blue keywords are present in both the references section and the paper title.
About this article
Date of Publication: 2023-08-31
Volume 23, Issue 3, Year 2023, On page(s): 47 - 56
ISSN: 1582-7445, e-ISSN: 1844-7600
Digital Object Identifier: 10.4316/AECE.2023.03006
Web of Science Accession Number: 001062641900006
SCOPUS ID: 85172316027
Abstract
Ensuring the safety, security, robustness, and fault tolerance of advanced robotic systems is essential for various fields, including healthcare, industry, and space. To address these issues, it is necessary to use software testing techniques and standards, similar to those applied in other safety-critical applications. The Robot Operating System (ROS) is a popular choice for developing robotic systems, so it is important to have specialized testing libraries and methods for it. In this study, a novel mutation testing library for ROS was developed and integrated it into the automated/tailored mutation-based software fault injection tool (IM-FIT). IM-FIT is an open-source automated software testing tool that is used to evaluate the software robustness of safety-critical systems using mutation-based tests. The proposed ROSMutation library mutates ROS-specific code snippets (publisher, subscriber, params, services, etc.) in the Python code within ROS-compatible software packages using IM-FIT. We evaluated the effectiveness of the ROSMutation library in two scenarios (basic and advanced), applying it to ROS-compatible code through IM-FIT and measuring its ability to assess the software robustness of ROS-compatible and Python-based software packages. The results showed that the ROSMutation library is effective in evaluating software robustness criteria for ROS-compatible and Python based software. |
| References | | | Cited By «-- Click to see who has cited this paper |
| [1] J. Guiochet, M. Machin, H. Waeselynck, "Safety-critical advanced robots: A survey," Robotics and Autonomous Systems, 94, pp. 43-52 2017. [CrossRef] [Web of Science Times Cited 142] [SCOPUS Times Cited 176] [2] F. Tambon, G. Laberge, L. An, A. Nikanjam, P. S. N. Mindom, Y. Pequignot, F. Laviolette, "How to certify machine learning based safety-critical systems? A systematic literature review," Automated Software Engineering, 29(2), pp. 1-74, 2022. [CrossRef] [Web of Science Times Cited 23] [SCOPUS Times Cited 33] [3] A. Afzal, C. Le Goues, M. Hilton, C. S. Timperley, "A study on challenges of testing robotic systems," In 2020 IEEE 13th International Conference on Software Testing, Validation and Verification (ICST) pp. 96-107, IEEE, October 2020. [CrossRef] [Web of Science Times Cited 44] [SCOPUS Times Cited 63] [4] Z. M. Bi, Z. Miao, B. Zhang, C. W. Zhang, "The state of the art of testing standards for integrated robotic systems," Robotics and Computer-Integrated Manufacturing, 63, 101893. 2020. [CrossRef] [Web of Science Times Cited 21] [SCOPUS Times Cited 23] [5] R. Bouziane, L. S. Terrissa, S. Ayad, "Semantic web services for ROS: A Robot as a Service approach," Automated Software Engineering, 29(2), pp. 1-26, 2022. [CrossRef] [Web of Science Times Cited 3] [SCOPUS Times Cited 3] [6] A. Alami, Y. Dittrich, and A. Wasowski, "Influencers of quality assurance in an open source community," In Proceedings of the 11th International Workshop on Cooperative and Human Aspects of Software Engineering, pp. 61-68, May 2018. [CrossRef] [Web of Science Times Cited 11] [SCOPUS Times Cited 13] [7] M. Lauer, M. Amy, J. C. Fabre, M. Roy, W. Excoffon, and M. Stoicescu, "Resilient computing on ROS using adaptive fault tolerance," Journal of Software: Evolution and Process, 30(3), e1917, 2018. [CrossRef] [Web of Science Times Cited 6] [SCOPUS Times Cited 9] [8] S. C. Lee and O. S. Ee, "rosTest: Universal test framework for real-time operating system," 2016 IEEE 25th Asian Test Symposium (ATS), Hiroshima, Japan, 2016, pp. 129-129, 2016. [CrossRef] [Web of Science Times Cited 2] [SCOPUS Times Cited 3] [9] T. Durieux, R. Abreu, M. Monperrus, T. F. Bissyande and L. Cruz, "An analysis of 35+ million jobs of travis CI," 2019 IEEE International Conference on Software Maintenance and Evolution (ICSME), Cleveland, OH, USA, pp. 291-295, 2019. [CrossRef] [Web of Science Times Cited 14] [SCOPUS Times Cited 15] [10] R. Halder, J. Proenca, N. Macedo, A. Santos, "Formal verification of ROS-based robotic applications using timed-automata," In 2017 IEEE/ACM 5th International FME Workshop on Formal Methods in Software Engineering (FormaliSE), pp. 44-50, IEEE, May 2017. [CrossRef] [Web of Science Times Cited 40] [SCOPUS Times Cited 60] [11] R. Baker, I. Habli, "An empirical evaluation of mutation testing for improving the test quality of safety-critical software," IEEE Transactions on Software Engineering, 39(6), pp. 787-805, 2012. [CrossRef] [Web of Science Times Cited 48] [SCOPUS Times Cited 60] [12] R. Delgado, M. Campusano, A. Bergel, "Fuzz testing in behavior-based robotics," In 2021 IEEE International Conference on Robotics and Automation (ICRA), pp. 9375-9381, IEEE, May 2021. [CrossRef] [Web of Science Times Cited 4] [SCOPUS Times Cited 6] [13] A. Santos, A. Cunha, N. Macedo, "Property-based testing for the robot operating system," In Proceedings of the 9th ACM SIGSOFT International Workshop on Automating TEST Case Design, Selection, and Evaluation, pp. 56-62, November 2018. [CrossRef] [Web of Science Times Cited 14] [SCOPUS Times Cited 18] [14] S. O. Sohail, A. Mitrevski, N. Hochgeschwender, P. G. Ploger, "Property-based testing in simulation for verifying robot action execution in tabletop manipulation," In 2021 European Conference on Mobile Robots (ECMR), pp. 1-7, IEEE, 2021. [CrossRef] [Web of Science Times Cited 1] [SCOPUS Times Cited 2] [15] M. A. Brito, S. R. Souza, P. S. Souza, "Integration testing for robotic systems," Software Quality Journal, pp. 1-33, 2020. [CrossRef] [Web of Science Times Cited 5] [SCOPUS Times Cited 4] [16] J. S. Kang, H. S. Park, "Web-based automated black-box testing framework for component based robot software," In Proceedings of the 2012 ACM Conference on Ubiquitous Computing, pp. 852-859, September 2012. [CrossRef] [SCOPUS Times Cited 5] [17] T. A. Budd, "Mutation analysis of program test data," (Order No. 8025191), 1980. https://www.proquest.com/dissertations-theses/mutation-analysis-program-test-data/docview/288372646/se-2 [18] R. A. DeMillo, "Completely validated software: Test adequacy and program mutation," In Proceedings of the 11th International Conference on Software engineering, pp. 355-356, May 1989. [CrossRef] [19] A. J. Offutt, R. H. Untch, "Mutation 2000: Uniting the orthogonal," Mutation testing for the new century, pp. 34-44, 2001. [CrossRef] [20] Y. Jia, M. Harman, "An analysis and survey of the development of mutation testing," IEEE transactions on software engineering, 37(5), pp. 649-678, 2010. [CrossRef] [Web of Science Times Cited 685] [SCOPUS Times Cited 1270] [21] U. Yayan, C. Baglum, "Tailored mutation-based software fault injection tool (IM-FIT)," SoftwareX, 23, pp. 1-4, 2023. [CrossRef] [Web of Science Times Cited 3] [SCOPUS Times Cited 4] [22] ROSMutation Repository. https://github.com/ESOGU-SRLAB/IM-FIT/blob/main/rosFaultLib.json. Accessed 13 Jan. 2023 [23] U. Yayan, A. T. Arslan, H. Yucel, H. Gencturk, "Robot programming with ROS," September 2020. ISBN:9789750261466 [24] A. K. Erdogmus, U. Yayan, "Industrial robot motion planning algorithms performance benchmarking," Journal of Scientific Technology and Engineering Research. 2(2), pp. 31-45, 2021. [CrossRef] [25] B. Latif, T. A. Rana, "Preliminary survey on software testing practices in Khyber Pakhtun Khwa region of Pakistan," Turkish Journal of Electrical Engineering & Computer Sciences, 28(1), pp. 575-589, 2020. [CrossRef] [Web of Science Times Cited 1] [SCOPUS Times Cited 1] [26] E. Jenn, J. Arlat, M. Rimen, J. Ohlsson, J. Karlsson, "Fault injection into VHDL models: the MEFISTO tool," In Predictably Dependable Computing Systems, pp. 329-346. Springer, Berlin, Heidelberg, 1995. [CrossRef] Web of Science® Citations for all references: 1,067 TCR SCOPUS® Citations for all references: 1,768 TCR Web of Science® Average Citations per reference: 40 ACR SCOPUS® Average Citations per reference: 65 ACR TCR = Total Citations for References / ACR = Average Citations per Reference We introduced in 2010 - for the first time in scientific publishing, the term "References Weight", as a quantitative indication of the quality ... Read more Citations for references updated on 2024-11-26 19:00 in 173 seconds. Note1: Web of Science® is a registered trademark of Clarivate Analytics. Note2: SCOPUS® is a registered trademark of Elsevier B.V. Disclaimer: All queries to the respective databases were made by using the DOI record of every reference (where available). Due to technical problems beyond our control, the information is not always accurate. Please use the CrossRef link to visit the respective publisher site. |
Copyright ©2001-2024
Faculty of Electrical Engineering and Computer Science
Stefan cel Mare University of Suceava, Romania
All rights reserved: Advances in Electrical and Computer Engineering is a registered trademark of the Stefan cel Mare University of Suceava. No part of this publication may be reproduced, stored in a retrieval system, photocopied, recorded or archived, without the written permission from the Editor. When authors submit their papers for publication, they agree that the copyright for their article be transferred to the Faculty of Electrical Engineering and Computer Science, Stefan cel Mare University of Suceava, Romania, if and only if the articles are accepted for publication. The copyright covers the exclusive rights to reproduce and distribute the article, including reprints and translations.
Permission for other use: The copyright owner's consent does not extend to copying for general distribution, for promotion, for creating new works, or for resale. Specific written permission must be obtained from the Editor for such copying. Direct linking to files hosted on this website is strictly prohibited.
Disclaimer: Whilst every effort is made by the publishers and editorial board to see that no inaccurate or misleading data, opinions or statements appear in this journal, they wish to make it clear that all information and opinions formulated in the articles, as well as linguistic accuracy, are the sole responsibility of the author.
Faculty of Electrical Engineering and Computer Science
Stefan cel Mare University of Suceava, Romania
All rights reserved: Advances in Electrical and Computer Engineering is a registered trademark of the Stefan cel Mare University of Suceava. No part of this publication may be reproduced, stored in a retrieval system, photocopied, recorded or archived, without the written permission from the Editor. When authors submit their papers for publication, they agree that the copyright for their article be transferred to the Faculty of Electrical Engineering and Computer Science, Stefan cel Mare University of Suceava, Romania, if and only if the articles are accepted for publication. The copyright covers the exclusive rights to reproduce and distribute the article, including reprints and translations.
Permission for other use: The copyright owner's consent does not extend to copying for general distribution, for promotion, for creating new works, or for resale. Specific written permission must be obtained from the Editor for such copying. Direct linking to files hosted on this website is strictly prohibited.
Disclaimer: Whilst every effort is made by the publishers and editorial board to see that no inaccurate or misleading data, opinions or statements appear in this journal, they wish to make it clear that all information and opinions formulated in the articles, as well as linguistic accuracy, are the sole responsibility of the author.
