On Challenges and Opportunities of Using Continuous Experimentation in the Engineering of Contemporary Software Systems

Context: Modern Information Systems require the use of contemporary software systems such as Cyber-Physical Systems, Embedded Systems, and Smart Cities-based Systems, eventually built under the paradigm of the Internet of Things. These Contemporary Software Systems (CSS) add new challenges for their construction, maintainability, and evolution, including the involvement of many actors with the software project and the necessary management of dependencies among hardware/things, software systems, and people. Problem: These technological challenges jeopardize the final quality of modern information systems due to the lack of adequate mechanisms supporting the engineering of CSS. Solution: Continuous Experimentation (CE) deserves some investigation regarding its suitability to mitigate and reduce engineering CSS risks. IS Theory: This research is under the General Systems Theory and is consistent with the Systems Information challenges regarding building smart cities-based systems. Method: To undertake a Structured Literature Review (StLR) supported with snowballing to reveal CE's empirical studies. Results: The StLR identified seven primary studies on CE adoption to support CSS building. Many studies are in the domain of embedded systems and CPS. Besides, the findings allowed us to conjecture a set of challenges and opportunities regarding using CE in CSS engineering. Conclusion: There are emergent technologies to support CE's execution in the context of web-based systems. However, several challenges and gaps surround CE's use for engineering CSS. Furthermore, the lack of software technologies, blueprints, or concrete guidance to promote CE in these software systems can motivate further investigations into its use in engineering the important parts of modern information systems.