What's in this paper?
We used a staged evidence-based process that includes case study, systematic literature review, and value stream analysis to study the challenges and to create a map of solution proposals for our case company.
Combining the above mentioned approaches is a good way to do industry academia collaboration -> it allows studying industrial problems with rigorous academic methods and produces a result that is relevant to the case company.
- It was challenging to perform a systematic literature review of a broad topic, the problems of testing process, as solutions may range from knowledge management, requirements engineering or test environment, etc.
- We solved the problem by limiting the literature review to studies about an automotive domain
- Existing literature reviews are often topic specific and targeted for researchers, e.g. what do we know about pair-programing. Problem specific literature reviews would be more helpful for industry, e.g. what evidence is available of requirements traceability solutions.
Abstract
Context: Evidence-based software engineering (EBSE) provides a process for solving practical problems based on a rigorous research approach. The primary focus so far was on mapping and aggregating evidence through systematic reviews.
Objectives: We extend existing work on evidence-based software engineering by using the EBSE process in an industrial case to help an organization to improve its automotive testing process. With this we contribute in (1) providing experiences on using evidence based processes to analyze a real world automotive test process; and (2) provide evidence of challenges and related solutions for automotive software testing processes.
Methods: In this study we perform an in-depth investigation of an automotive test process using an extended EBSE process including case study research (gain an understanding of practical questions to define a research scope), systematic literature review (identify solutions through systematic literature), and value stream mapping (map out an improved automotive test process based on the current situation and improvement suggestions identified). These are followed by reflections on the EBSE process used. Results: In the first step of the EBSE process we identified 10 challenge areas with a total of 26 individual challenges. For 15 out of those 26 challenges our domain specific systematic literature review identified solutions. Based on the input from the challenges and the solutions, we created a value stream map of the current and future process.
Conclusions: Overall, we found that the evidence-based process as presented in this study helps in technology transfer of research results to industry, but at the same time some challenges lie ahead (e.g. scoping systematic reviews to focus more on concrete industry problems, and understanding strategies of conducting EBSE with respect to effort and quality of the evidence).
Ref
Kasojua, A, Petersen, K., Mäntylä M. V., "Analyzing an Automotive Testing Process with Evidence-Based Software Engineering", Information and Software Technology, vol. 55, issue 7, July 2013, pp. 1237-1259
