Assessing software product line potential: an exploratory industrial case study

Abstract

Corporate organizations sometimes offer similar software products in certain domains due to former company mergers or due to the complexity of the organization. The functional overlap of such products is an opportunity for future systematic reuse to reduce software development and maintenance costs. Therefore, we have tailored existing domain analysis methods to our organization to identify commonalities and variabilities among such products and to assess the potential for software product line (SPL) approaches. As an exploratory case study, we report on our experiences and lessons learned from conducting the domain analysis in four application cases with large-scale software products. We learned that the outcome of a domain analysis was often a smaller integration scenario instead of an SPL and that business case calculations were less relevant for the stakeholders and managers from the business units during this phase. We also learned that architecture reconstruction using a simple block diagram notation aids domain analysis and that large parts of our approach were reusable across application cases.

Appendix: Application Cases

This section provides a detailed description of the application of our Software Product Line potential assessment approach and is intended to facilitate executing similar approaches.

1.1 A.1 Application Case 1: Industrial Control Systems

Industrial control systems monitor and control automation processes in a variety of different industrial sectors (e.g., power generation, chemical plants, or substations). ABB has several such systems in its portfolio, which address specific industrial subdomains and have been incorporated in former company mergers.

For step 1 of our approach we conducted an internal survey of existing ABB products, which resulted in a list of products, from which we removed such products that are no longer in active development. In step 2 we decided to use almost all of the generic reuse criteria formerly defined. We also decided to focus on high-level features only. Since these are large-scale systems, there is no fine-granular break-down of the features as the analysis rather targets the systematic reuse of whole subsystems. According to the selected reuse criteria, we started to create the questionnaire for the later interviews.

In step 3, we found that for some systems extensive architectural documentation was available. We used this information to come up with a sketch of the architecture in the FMC notation. We created an initial list of 30 features in step 4, which we based on our own domain knowledge from working with similar systems (Fig. 2). The list was reviewed by multiple ABB-internal domain experts, who did not work on the products themselves.

Fig. 2

Anonymized excerpt of the feature support matrix for the control systems case study

We conducted on-site interviews for each product ranging from half a day to three days (step 5) depending on the product’s size. As the last part of each product interview, we created and extended our architectural sketches. Due to the different sizes of the systems and the varying depth of the interviews, the architecture illustrations created during the interviews had different granularity. In subsequent document analyses and clarifications via phone the architecture illustrations of all systems were brought to a comparable level (example in Fig. 3. Based on the architecture illustrations we were capable of visualizing the data flows through the components, which are omitted here for confidentiality. For example, one data flow included a sensor value read from an industrial device and propagated through the system to be shown on the operator screen.

Fig. 3

Architecture map for three of the analyzed systems in application case 1 (simplified)

The next step 6 was to identify reusable features and subsystems based on the information from system documentation and the interviews transcripts. We executed this step offsite after all interviews had been finished. We used a product feature support matrix (Fig. 2). The matrix shows for each product whether a feature from the feature list is supported. We also indicated whether a new implementation of a feature or a replacement of an existing subsystem is currently realistic. In this case, we marked the product as a potential reuse consumer for that feature. Products that have a high-quality and generic implementation of a feature are highlighted as potential reuse providers.

Furthermore, we analyzed the cohesion and coupling of the subsystems that implement a feature to find out whether subsystems could be easily extracted from the implementations to serve as core assets. We also colored subsystems in different products that implement similar features. Using the matrix we identified several realistic new reuse opportunities among the products. We documented the technical feasibility for each reuse opportunity based on the information from the interviews and the available documentation. The organizational feasibility of each reuse opportunity was also analyzed similarly.

In parallel, we attempted to create a economic analysis (step 7), which proved to be too complicated in this case. Besides the development and maintenance cost of each product, also the migration costs for existing installations need to be considered if a subsystem is replaced. This is especially critical in the industrial control system domain, since these systems include numerous hardware devices and engineering work to customize the system. We were thus not able to create a reasonable business calculation in this domain, because the inputs for such a calculation were difficult to gather.

In summary, in this application case, our approach enabled us to create a thorough, albeit high-level, assessment of reuse opportunities. It showed to scale to complex systems and still provides useful results despite the relatively short feature list. In one product, the high-level architecture overviews provided by our architecture illustrations made the respective business unit rethink architectural trade-offs. The architecture illustrations also were instrumental in stimulating discussions among the product architects and visualize the functional overlap.

Our initial intuition about high functional overlap between the various systems was confirmed by our analysis. However, an actual re-engineering of the systems towards reusing subsystems still seems difficult due to the expected high migration costs. These costs are a result of the high configurability of the products, which would require high efforts to update customer configurations. Therefore, the focus of reuse opportunities in this domain lies more in guiding future development towards a better alignment and systematic reuse of subsystems.

1.2 A.2 Application Case 2: Automation PC Tools

In the second application case, we analyzed a set of PC-based commissioning and monitoring tools for industrial automation systems. Commissioning tools load software applications onto a device and set application parameters. Monitoring tools observe an automation system, check and predict its operating conditions, and raise alarm events. During their development history, the developers of the tools have changed. The product management was aware of the tools in the families as well as of the potential functional overlaps, which could provide a potential for a software product line and for reducing future development and maintenance efforts.

The time frame for this SPL potential analysis was three working weeks. We used one week for preparation (step 1-4), one week on site for interviews with the developers (step 5), and one week for potential identification and economic analysis (step 6-7). For step 1 the developers provided an initial list of all tools. We excluded tools, for which a low reuse potential could be assumed or which are no longer maintained. From the generic reuse criteria we excluded organizational criteria, since the tools were owned by only two units, which were already collaborating successfully step 2.

We gathered user manuals and architecture documents in step 3, which were instrumental to build up an initial feature list (step 4). Nevertheless, due to our limited former domain knowledge, we had to significantly adapt the list in step 5. During this step, we interviewed three project managers and developers, who provided a business and technical perspective. There was no explicit architecture role defined for these products. We used a feature map (Schmid 2002) immediately in the interviews (Fig. 4). Compared to Schmid’s approach (Schmid 2002), we only performed product line mapping and a short domain potential assessment, but no quantitative reuse infrastructure scoping, due to effort limitations.

Fig. 4

Example feature map

The last session of each interview, was the architecture reconstruction. We started from a black box view of a tool with its interfaces, refined this into a view showing clients and servers, for example, and modeled the next levels of components and internal data flow. The data-flow oriented view of FMC was a new notation for the developers. After a short learning phase, modeling worked well and supported an active discussion about the architecture and its concepts, such as used patterns and styles.

After completing the interviews, we finalized the assessment of the reuse and SPL potential for each domain and subdomain in the feature map (step 6). We added rationale for each potential rating of low, medium, and high. In the end, the feature map consisted of more than 300 features grouped into 50 subdomains and 19 domains. The potential for systematic reuse of ten subdomains, e.g., data management, are ranked high. The architecture illustration was then used to cross-check the technical feasibility of creating shared components for the identified subdomains. In the end, we identified three short-term reuse scenarios involving smaller subsets of features that can be implemented in the next releases. Additionally, six feature domains and subdomains are long-term candidates for being integrated into a common platform, but require deeper technical investigations.

To round up our investigation with an economic analysis, we performed a business case calculation step 7. In parallel to the preceding steps of the analysis, we collected numbers for the current development and maintenance costs of the tools, which required several weeks of calendar time, as the information had to be gathered from different sources. Then we calculated the return on investment for each reuse scenario. As originally defined, the formulas (Boeckle et al. 2004) were only applicable to one reuse scenario without changes. In this scenario, two components are merged for being reused in both tool families. In this case, we estimated C _{c a b} as 50 percent higher than building one of the individual products. F _{c a b} was 10 percent and C _{r e u s e} was approx. 12 percent of the costs of building the whole products from scratch. We have omitted the other costs for confidentiality reasons. The calculation showed a positive ROI after three years.

The stakeholders were particularly interested in the three short-term reuse scenarios and their business cases. They did not require the exact numbers, but were satisfied by the positive trend expected for implementing the scenarios. Thus, they planned to discuss the scenarios with the other relevant stakeholders of the tools for implementation.

1.3 A.3 Application Case 3: Device Engineering Tools

In the following application case, we focus on three different ABB device engineering tools for configuration, commissioning, and maintenance within a single industrial sector. In step 1 of our approach these tools had been identified by management and a contact person had been named for each tool. Specific for this case we considered a new product with only a technical and market requirement specification available at the time of analysis. This is to ensure that this new product can directly benefit from and support the systematic software reuse opportunities that are identified during our analysis.

While establishing the reuse criteria in step 2 management asked us to put specific emphasis on lessons learned from the existing tools and future developments. Finally, as management appeared to be open minded towards reuse for future developments, reuse culture was also integrated into the questionnaire. Processes and market fit were excluded in this case.

During step 3 we collected documentation on the different engineering tools as well as the devices to be engineered, e.g., product manuals, specification sheets, and getting started guides. For the most powerful and complex tool, we also received detailed internal technical documentation, such as an architecture review of a previous tool version, from which we created a first architectural sketch. In step 4, we created an extensive feature list with roughly 200 entries categorized in 10 domains (Fig. 4).

For step 5, we scheduled one and a half days for the interviews with lead architect, product manager, and team manager for each tool. We started each interview with a short tool and device demonstration, which proved to provide a good common understanding. During the interviews we spent most time for discussing and refining the feature list. To reconstruct the architecture in the interviews we additionally relied on the CrossModel Architecture Discovery Pattern Language (Fairbanks 2010) for guidance, which was not yet known to us in the other application cases. In addition, the FMC notation was used to document the architecture.

After the interviews, we identified reuse opportunities on the feature and system level (step 6). The regarded tools partially have similar functionality as well as similar technology roadmaps for future releases. This finding resulted in the creation of a new project to design and implement a common underlying platform. Especially, collected lessons learned and identified high level requirements covering all tools will be useful. The lessons learned were categorized into organizational, technical, and known challenges with reuse. This allows for avoiding known issues but also to benefit from known opportunities and good decisions.

We did not create a business case (step 7) in this application case, as the management’s priority was on technical aspects of future reuse opportunities. Nevertheless the involved stakeholders confirmed their perception of an expected positive return on investment by starting a new platform design and development project. Based on the positive analysis results, one additional ABB device engineering tool was added to the potential scope of a future common platform and will be analyzed soon.

1.4 A.4 Application Case 4: Enterprise Information Systems

In this application case, we analyzed a set of ABB enterprise information systems, which are used to store large data sets and to provide them to different kinds of users via corresponding views, filters, and search functions. The analysis comprised a family of five products and two individual products, which have grown independently of each other. Today, the two individual products (A and B) and one product from the family (C) are used to support similar business processes, i.e., some users require all three systems to manage related sets of data for particular tasks. The four remaining products of the family have different users and handle independent data.

To better understand the relationships of the considered enterprise information systems, we extended the domain analysis: we used a comparison of the database schemata to analyze the relation and overlap of the data managed by the products. Additionally, we conducted an evaluation of the underlying business processes to assess the tool overlap on this level as well as the impact of merging the tools on the business processes.

In step 1, the product managers provided the list of products to analyze as well as the contact persons. To establish the reuse criteria in step 2 we reused the criteria prioritization from application case 3, which was agreed upon by the contact persons. The provided documentation for step 3 included product overviews such as powerpoint presentations, user manuals, database schemata, use case descriptions in UML, and high-level architectural descriptions.

From the collected documentation, we were able to prepare around 70 percent of the final feature table in step 4 and major parts of the architecture maps before the interviews. Similar to the feature table, we prepared a table that mapped the data model entities (in the rows) to products (in the columns). We mainly used the product overviews and user manuals of the products as input for the data entity map, because the database schemata were too detailed and complex for such an high level mapping. Based on a use case documentation, we were able to create a draft version of a business process description of one of the products.

The on-site interviews in step 5 required one and a half day for each product. Showing the feature map for all products to the developers of a single tool created discussions. Some interviewees disputed the features supported by other products or argued that despite missing support in their own product, a work around would be possible. These discussions were valuable, as they allowed to cross check the feature table and achieve a better understanding of the differences of the products. The final feature map contained more than 250 features grouped into subdomains and domains.

Afterwards, we reviewed the prepared FMC architecture maps and completed them together with the interviewees. The original architecture documentations used proprietary notations and described the architecture on different levels of abstraction for each product. Redrawing the architectures in FMC unified the notation and the abstraction level for better comparability. All interviewees understood the notation immediately and were able to provide corrections and additions.

In step 6, we compared the business process, features, architectures, and data schemata. For comparing the business processes, we mapped them onto a related reference business process specification. As a result, we identified a large overlap between the individual product A and product C from the family, although they might be instantiated differently. This fact was determined in three business processes which are supported by both systems but handled in a different way. For the feature map, we assessed the overlap with a subjective of rating high, medium, and low and added a rationale statement. As for the business processes, the individual product A and product C from the family showed a large feature overlap. The functional overlap inside product B and the product family was lower and mainly covers basic tool functionality such as open, edit, and store data sets.

From the architecture maps, we compared logical structuring and used technologies. The two individual products and the tool family had been implemented in different technologies, which hinders direct reuse of components between the products. Also the logical structures of the products were different, but the cores of the individual product A and the product family were similar and provided potential for a common platform. The data schemata included different names and types, but semantically similar elements. We observed the largest overlap between product A’s and C’s data model, while product B had a low data model overlap.

Based on these results, we elaborated several scenarios for integrating the products: 1) data integration of A and C via corresponding interfaces, 2) merging A and C into a single product, 3) building A and C based on a common platform, and 4) building a software product line for implementing product A, product B, and the family (including C). For all scenarios, we listed the most important advantages and disadvantages such as usability, complexity, maintainability, time to market, and investment cost. We also sketched migration roadmaps between the scenarios.

In step 7, we only performed a return on investment calculation for reuse scenarios 3) and 4). The business units provided the former development and maintenance costs for the calculations, and we estimated a 50 percent higher effort for building generic, shared parts instead of specific, separated parts (C _{c a b} ). Although the feature map indicated a high functional overlap, we conservatively assumed the overlap to be 50 percent after consulting the architects. With these and other assumptions, we expected a positive return on investment for scenario 3) after seven years, latest. Due to the larger number of products, the return on investment scenario 4) will likely pay off earlier. However, in both cases not the exact numbers, but the positive trend even under conservative assumptions was important for the stakeholders.

The business unit will first implement scenario 1) (data integration) for better usability of products A and C in common business processes and will align their business processes for preparing a more deep integration scenario. After the alignment of the business processes, the proposed integration scenarios well be reassessed for selecting the final solution.