Reasons for bottlenecks in very large-scale system of systems development
Introduction
System of systems development is considered a complex task [1]. Multiple definitions of system of systems exist (cf. [1], [2]). All agree on that a system in a system of systems shall be possible to be distributed alone. However, a set of systems also shall have the capability to operate together in a system of systems. Different strategies of how to develop the system of systems are characterized in [1], namely virtual, collaborative, acknowledged, and directed. The system of systems investigated in this study is directed. According to the definition in [1] a directed SoS is characterized by a well specified purpose, the development is centrally managed, and that “component systems maintain an ability to operate independently, but their normal operational mode is subordinated to the central managed purpose [1].” Given the centrally managed purpose, it is common that requirements may affect multiple systems, which require coordination. In contrast, in single stand-alone system development individual systems are developed and not built with the intention to operate in a compound of multiple other systems.
A vast amount of research has been done on system of systems in different disciplines. In particular military systems have been researched intensively [1], but not with a specific focus on software intensive system of systems. Research acknowledges that developing system of systems is a very complex task [1], [3], [4]. Already known reasons that complicate the development are the high number of parties and stakeholders involved, and the challenge of continuously integrating the systems [3], [4], [5].
In large-scale software intensive system of systems, a large amount of requirements are transformed (e.g. specified in detail, designed, coded, and tested). That is, they flow through the development life-cycle. It was found that the flow should be continuous and bottlenecks as well as inventory should be avoided (see [6], [7], [8]). In past research projects we provided solutions for analyzing the flow of requirements to identify bottlenecks [7]. We used those solutions to identify bottlenecks in single system development and SoS development, and observed more severe bottlenecks in SoS development (see Section 3.3). Bottlenecks have negative consequences, such as work overload [6], [9], delays, finding problems late in the process [10], requirements becoming obsolete [11], and creation of stress in the organization [12].
Given the severance of bottlenecks in complex SoS development, and the negative consequences associated with them, there is a need to identify the reasons for bottlenecks in SoS development. In order to identify the reasons we conducted a case study at Ericsson AB, the company develops single stand-alone systems and system of systems. In the case study we elicited reasons from two views, the system of systems view (responsible for technical oversight and making sure that individual systems can be integrated) and the system view (developing a system that is part of the system of systems in sets of projects). The elicited reasons were prioritized from both views. The following contributions are made by this study:
- •
Identification of reasons for bottlenecks from the system of systems and system views in the flow of developing software requirements.
- •
Prioritization of reasons from both views.
- •
Comparative analysis with respect to different roles (e.g. testers, requirements engineers, and so forth).
- •
Comparative analysis of the system of systems view and system view with respect to reasons identified and their priorities.
The practical relevance of the research is given as there is a need to understand the most critical reasons in order to prioritize process improvements addressing them.
The remainder of the paper is structured as follows: In Section 3 we provide a description of the SoS types identified in literature, and relate the types to the studied SoS. Furthermore, Section 3 reviews work on requirements flow and bottlenecks, followed by a comparison of bottlenecks between system of systems and single systems. Section 2 represents the related work, focusing on system of systems as well as existing studies on flow and bottleneck analysis in the software engineering context. Section 4 presents the research method (case study), describing the case context, units of analysis, data collection and data analysis. Furthermore, threats to validity are presented. Section 5 provides the results of the study. The results are described from the SoS view, and the system view. Both views are compared with each other. Section 6 discusses the results, and Section 7 concludes the paper.
Section snippets
Systems of systems
Although relatively new concept, there have been numerous publications in the area of SoS design and development. For example, Ncube [13] discussed key challenges related to interoperability, requirements engineering processes and management of emergent behavior. Ncube also stated initiatives that aim to provide strategic research agendas and road-maps for addressing some of these key challenges. Lewis et al. [4] identified a set of characteristics of SoS, proposed an SoS life-cycle, and
System of systems context
The system of systems term has been used in different disciplines (e.g. health [26] and military [1]).
The system of systems engineering guide of the department of defense [1] describes four types of system of systems, i.e. virtual, collaborative, acknowledged and directed. They differ with respect to the purpose, management and maintenance/change mechanism, which are summarized in Table 1. Based on these aspects, the studied SoS falls under directed SoS. Moreover, in practice there is usually
Research methodology
As a research methodology a single case study with multiple units of analysis (cf. [27]) has been used.
Results
The results are split based on the definition of the units of analysis (i.e. SoS level and system view). For each of the two views we provide an overview of the reasons, elaborate on their priorities, and identify the most significant differences between reasons with respect to their prioritization, followed by an analysis of agreement between practitioners conducting the prioritization.
Discussion
In the discussion, we highlight interesting observations and relate them to existing theories and literature. Based on the observations we provide implications and action points for researchers and practitioners.
Conclusion
We made four contributions in this paper. First, we have identified the reasons that make it challenging to achieve a continuous flow in SoS development from (a) system of systems view and (b) individual system view. These reasons were identified in workshops and after the workshops clustered under six different areas. We highlighted reasons that explicitly refer to the SoS, while other reasons are more general, but amplified due to the SoS context.
Secondly, we have reported the most
Acknowledgements
We would like to thank all the participants in the study who provided valuable input. Thanks also to Ronald Jabangwe for proof-reading the paper. This work has been partially funded by ELLIIT, The Linköping-Lund Initiative on IT and Mobile Communication, www.elliit.liu.se, and the BESQ + research project funded by the Knowledge Foundation (Sweden) (Grant: 20100311).
References (49)
- et al.
Software process improvement through the lean measurement (spi-leam) method
J. Syst. Softw.
(2010) - et al.
Exploring bottlenecks in market-driven requirements management processes with discrete event simulation
J. Syst. Softw.
(2001) - et al.
Empirical extension of a classification framework for addressing consistency in model based development
Inform. Softw. Technol.
(2011) - et al.
A comparative study of architecture knowledge management tools
J. Syst. Softw.
(2010) - DoD, Systems and Software Engineering, Systems Engineering Guide for Systems of Systems, Version 1.0., Tech. Rep....
- et al.
Synthesizing SoS concepts for use in cost estimation
Syst. Eng.
(2007) - et al.
Interoperability test and evaluation: a system of systems field study
J. Defense Softw. Eng.
(2008) - G. Lewis, E. Morris, P. Place, S. Simanta, D. Smith, Requirements engineering for systems of systems, in: Proceedings...
Architecting principles for systems-of-systems
Syst. Eng.
(1998)- et al.
Measuring the flow in lean software development
Softw. Pract. Exper.
(2011)
Agile Management for Software Engineering: Applying the Theory of Constraints for Business Results
Lean software development: two case studies
Softw. Qual. J.
Agile and Iterative Development: A Manager’s Guide
Extreme Programming Explained: Embrace Change
Benchmarking performance measurement and lean manufacturing in the rough mill
Forest Prod. J.
Cited by (13)
Determining a core view of research quality in empirical software engineering
2023, Computer Standards and InterfacesCitation Excerpt :Reliability: There is a potential threat related to the misinterpretation of the collected data. To mitigate such threat, both data analysis and interpretation procedures were designed in advance, based on the works of Kuzniarz & Angelis [36], and Petersen, Khurum & Angelis [39]. Mainly, we normalized the importance values, given the dimensions were not equally distributed across the four main areas.
FLOW-assisted value stream mapping in the early phases of large-scale software development
2016, Journal of Systems and SoftwareCitation Excerpt :The work by Staron and Meding (2011) acknowledges the underlying contribution of VSM but that was not the object of their study. While the earlier work on identification of bottlenecks by Petersen et al. (2014a) and Petersen et al. (2014b) uses the typical visualizations used in a value stream analysis, but it does not use the VSM as a framework to guide the improvement activity in the organization. Rother and Shook (1999) presented the guidelines to conduct VSM for manufacturing processes where both material and information flow are mapped.
Evaluation of simulation-assisted value stream mapping for software product development: Two industrial cases
2015, Information and Software TechnologyCitation Excerpt :Continuous flow means that work items flow through the process at a continuous pace. A detailed account of definitions of continuous flow and reasons for the lack thereof is presented in [47,48]. A lack of continuous flow, in combination with the waiting times, is a first indication of waste in the process, and should be further investigated [8].
Temporal Complexity in Information Systems Development Flow: Challenges and Recommendations
2024, Communications of the Association for Information SystemsBottleneck identification and ranking model for mine operations
2020, Production Planning and Control