Exploratory analysis on the halo effect of strategic goals on IOS effectiveness evaluation
Introduction
While the impact of IT use on organizational performance has been a major research focus, past studies “have provided findings that tend to be either mixed or inconclusive …” because “the effects of IT are indirect …” and “IT is a moderator of organizational characteristics and processes” [23]. For example, one may find a direct link between office IT use and office work productivity, but the IT use may not translate into business unit success in terms of operational cost reductions, revenue increases, or business goal accomplishments.
The strategic value of interorganizational systems (IOS) has been well recognized for order entry and distribution [18], just-in-time (JIT) manufacturing [56], reservation systems [43], quick response [29], and vendor managed inventory (VMI) [30].1 An interesting case is efficient consumer response (ECR)—an IT-enabled industry-wide supply-chain initiative that started in the early 1990s in US grocery distribution channels. ECR has received reports of “growing industry dissatisfaction in the [US] trade press” [9], though positive reports exist [31], [36]. ECR had “fallen far short of its promised efficiencies and value” primarily because the industry expectation exceeded the reality [25]. Then, how do firms assess IOS, the key enabler of initiatives like ECR, when their business expectations and business outcomes do not coincide?
The adoption of IOS generally requires substantial commitments of both financial and operational resources. While such commitments indicate a firm’s faith in the ability of IOS to help achieve its strategic goals, not all firms share the same level of “faith.” Some see the situation differently because their trading partners or market conditions have mandated such resource commitments. Sometimes the results from IOS do not make significant strategic contributions [19], [41]. Furthermore, unlike systems used only in one firm, IOS are used with multiple trading partners. How does a retailer evaluate its IOS when its benefits differ with various suppliers? It usually requires a judgment call on how much IOS contributes to the accomplishment of favorable retailing business results when considering the impact on one trading partner.
Thus, the business evaluation of IOS use is neither simple nor straightforward. There are criteria on effectiveness, such as the gains in operational efficiency from various retail distribution processes, the reduction of operational personnel and/or their time, and product sales increase due to more timely and accurate market information. There are also factors that can either limit IOS use and any resulting effectiveness; these include general market conditions, the number and quality of competitors, and the firm’s management skills.
Given such an “ambiguous” link between IOS use and business goal accomplishments, our study examines whether a halo effect exists (or even pervades) when evaluating IOS. A halo effect is a strong overall impression that blurs distinctions between dimensions or attributes [10]. For example, our first impression of a person is sometimes based on feeling about a stereotype (of the group) to which the person belongs [3].
Our question therefore is: Do firms evaluate IOS based on the status of their business goal accomplishments but not on how much IOS actually contributes to the accomplishments? To answer this question we examined the relationships between four factors—the strategic goals established for a trading relationship, the degree that each goal is achieved, IOS usage, and the perceptions of IOS effectiveness—in the context of trading relations between retailers and suppliers. By assessing these relationships, the study explored whether and how IOS are “fairly” evaluated by retailers and suppliers. Also the study explored whether and how their views were different.
At the outset, it should be noted that this is an exploratory study based on a cross-sectional survey with limited sample sizes. Also, the survey data were collected in late 1998 and this limits the generalization of the research findings. However the results of our research did not depend on the “currency” of the IOS technologies used.
Section snippets
Research background
Studies of IS effectiveness are rooted in those of IS success. Delone and McLean [22] summarized the fundamental measures of IS success as systems quality, information quality, use, user satisfaction, individual impact, and organizational impact. Yet while these measures are the direct results of IS use, we face ambiguities when we evaluate IS success against the degree that strategic goals are realized.
Dewett and Jones reviewed and summarized a broad array of IT impact literature, and their
Research framework and hypotheses
This study formulated hypotheses on the relationships between four factors: strategic goals, IOS use, goal accomplishments, and IOS effectiveness evaluation (see Fig. 1). In the model, the existence of the halo effects was evaluated in three different ways:
- (a)
Whether the path between goal accomplishments and IOS effectiveness evaluation was significant while the path between IOS use and goal accomplishments was not or was only marginal (a halo based on the level of goal accomplishments).
- (b)
Whether
Research method
We collected data from retailers and suppliers through a mail survey in late 1998. The hypotheses were then tested using path analyses, because they allow us to test not only the relationship between individual variables but also the pattern of these relationships. Thus, we could construct and test the various models of possible causal relationships based on theoretical considerations.
Results
The overall results of PLS path analyses are shown in Fig. 2, Fig. 3.
Implications
The research model tested three possible types of halo effects in evaluating IOS effectiveness: a halo from accomplishing strategic goals; a halo from having important strategic goals; and a halo from having IOS in place itself. The results were that the first two types of halo effects occurred for both retailers and suppliers. The presence of the third was not detected for either.
For retailers, path analysis showed only the second type of halo effect. They correctly evaluate IOS based on how
Limitations
This study is exploratory in nature because of convenience sampling, smaller sample sizes, limited supplier participation and IOS data prior to 2000. In addition, it had low response rates like those in other studies using unsolicited mail surveys.14 It should be noted, however, that F tests between the first and second half of the responses did not show significant non-response bias.
In this research,
Conclusion
This exploratory research study showed that firms do not necessarily evaluate the effectiveness of IOS by how much IOS use contributed to achieving strategic goals. Rather, firms gave more favorable opinions on IOS, regardless of its contributions. This is due to the two types of halo effects where firms’ IOS evaluation is overshadowed by the extent to which their strategic goals are accomplished and the extent to which those strategic goals have significance to their trading partners.
Makoto Nakayama is assistant professor at the School of Computer Science, Telecommunications and Information Systems in DePaul University. He holds a PhD from University of California, Los Angeles and an MBA from University of Texas at Austin. Prior to moving into academe, he served as a product marketing manager at Novell Japan. He also worked on business planning and operating system developments at Yokogawa Electric Corporation in Tokyo. His current research interests are in IT use in
References (59)
- et al.
Managing EDI for corporate advantage: a longitudinal study
Information & Management
(1997) - et al.
Electronic data interchange: a multi-industry investigation using grounded theory
Information & Management
(1998) - et al.
The role of information technology in the organization: a review, model, and assessment
Journal of Management
(2001) - et al.
Duality in consumer post-purchase attitude
Journal of Economic Psychology
(1991) - et al.
The CEO/CIO relationship revisited: an empirical assessment of satisfaction with IS
Information & Management
(1995) The impact of EDI on the component sector of the Australian automotive industry
Journal of Strategic Information Systems
(1993)Interorganizational information systems and the structure and climate of seller-buyer relationships
Information & Management
(1995)- et al.
Information technology use by managers in Greece to support decision making: amount, perceived value, and satisfaction
Information & Management
(1995) - et al.
The effect of EDI on marketing channel relationships in retailing
Information & Management
(1997) - et al.
Time flies when you’re having fun: cognitive absorption and beliefs about information technology usage
MIS Quarterly
(2000)
Studies in the principles of judgments and attitudes II: determination of judgments by group and by ego standards
Journal of Social Psychology
The organizational transformation process: the micropolitics of dissonance reduction and the alignment of logics of action
Administrative Science Quarterly
A resource-based perspective on information capability and firm performance
MIS Quarterly
ECR and grocery retailing: an exploratory financial statement analysis
Journal of Business Logistics
Removing the financial performance halo from fortune’s “most admired” companies
Academy of Management Journal
Cooptive corporate actor networks: a reconsideration of interlocking directorates involving American manufacturing
Administrative Science Quarterly
Issues and opinion on structural equation modeling
MIS Quarterly
On the use, usefulness, and ease of use of structural equation modeling in MIS research: a note of caution
MIS Quarterly
Research report: empirical test of an EDI adoption model
Information Systems Research
Information technology and industrial cooperation: the changing economics of coordination and ownership
Journal of Information Systems
McKesson drug company: a case study of economist—a strategic information system
Journal of Management Information Systems
Limits to interfirm coordination through information technology: results of a field study in consumer packaged goods distribution
Journal of Management Information Systems
Coefficient alpha and the internal consistency of tests
Psychometrika
Information systems success: the quest for the dependent variable
Information Systems Research
Cited by (0)
Makoto Nakayama is assistant professor at the School of Computer Science, Telecommunications and Information Systems in DePaul University. He holds a PhD from University of California, Los Angeles and an MBA from University of Texas at Austin. Prior to moving into academe, he served as a product marketing manager at Novell Japan. He also worked on business planning and operating system developments at Yokogawa Electric Corporation in Tokyo. His current research interests are in IT use in marketing channels and IT skills portfolio management. His papers appeared in Information & Management, Journal of Information Technology, and proceedings of national conferences.
Norma G. Sutcliffe is assistant professor at the School of Computer Science, Telecommunications and Information Systems in DePaul University. She holds a PhD as well as an MBA from University of California, Los Angeles. With extensive experience in industry, Dr. Sutcliffe has been a consultant for many Fortune 500 firms in evaluating IT needs, IT implementations, and IT strategies. Dr. Sutcliffe has worked as systems developer on mainframe and client/server systems as well. Her current research interests are in IT enabled organizational change, IT leadership behavior, and IT skills portfolio management. Her papers appeared in Information & Management and proceedings of national conferences.