The role of consultants in the implementation of enterprise resource planning systems.
Metrejean, Eddie ; Stocks, Morris H.
INTRODUCTION
Organizations implement enterprise resource planning (ERP) systems
to solve their accounting, scheduling, and production problems and
because existing business practices and procedures are inadequate to
meet their current or future strategic needs (Nah, et al. 2001;
Karakanian 1999; Davenport 1998). Details on the inner workings of ERP
systems can be found in resources such as O'Leary (2000) and Jacobs
and Whybark (2000).
Because ERP implementations are very complex, costly, and risky,
organizations typically use consultants to assist them in their
implementation. Many organizations lack the information technology (IT)
personnel with the required expertise and time to undertake implementing
an ERP system, and consultants often assist their clients throughout the
entire implementation process (Glover et al. 1999; Nah et al. 2001;
Thong et al. 1994; Yap et al. 1992).
Considerable research has been performed in the areas of ERP
systems and the presence of consultants during these implementations.
The use of consultants is often cited as a critical factor for the
success of an implementation (Nah et al. 2001; Thong et al. 1994; Yap et
al. 1992). However, little research, if any, has examined the phases of
an ERP implementation and how the consultants fit into an implementation
on a phase-by-phase basis. If consultants are not necessary and/or
effective in a particular phase, they can then be left out of the work
on that phase, potentially saving considerable money for the
implementing organization. The results give researchers a starting point
for examining how consultants fit into the specific parts of an ERP
implementation. Further, practitioners will have an idea about were to
best use their scare resources when implementing ERP systems.
In light of this lack of research, this study has a twofold
purpose. The first purpose is to examine whether consultants are
perceived to be more effective and necessary in certain phases of an ERP
implementation than in other phases. The results of this research should
provide researchers who study ERP systems with information about the
phases of an implementation in which organizations use consultants. If
consultants are not perceived to be effective in a particular phase,
then consultants may be left out of that phase and internal IT personnel
may be used. The results show that IT managers do perceive that
consultants are more effective and necessary in the configuration and
integration phase of an implementation and least effective and necessary
in the operation of the ERP system. Thus, organizations may be better
able to focus the use of their ERP consultants on those particular
phases, which should save resources.
The second purpose of this study is to examine whether certain
characteristics possessed by ERP consultants contribute more to their
effectiveness than other characteristics. Researchers who study ERP
implementations should benefit from this research because the results
will give them some idea of which characteristics and skills that real
users consider most important. The results show that IT managers do
perceive that four characteristics (technical skills, business context
skills, commitment to quality, and the ability to manage ERP
implementations) are related to ERP consultant effectiveness. ERP
consultants may be able to focus on these particular skills to enable
them to provide more value to their clients who are implementing ERP
systems. Organizations may be able to look for consultants with a
particular skill set because that skill set tends to improve the
consultants' effectiveness.
The remainder of this paper examines the existing literature
regarding the effectiveness and need for ERP system consultants and
describes the development of a multiple-phase ERP implementation model
used to structure the research instrument for this study. Next, the
research method is described and results are examined. The final section
includes a conclusion and a discussion of the limitations of the
research.
REVIEW OF PRIOR LITERATURE
ERP implementations are costly in terms of both dollars and time
spent by personnel. Estimates of a typical ERP implementation range from
a low of 14 months to as high as 23 months to actually implement the ERP
system and another 31 months to see the benefits of the effort (Bingi et
al. 1999; Chen 2001; Koch 2002). Estimates on the cost of a new ERP
system range from $2 to $4 million dollars for a small organization with
simple processes to over $1 billion for a large organization with
complex processes (Bingi et al. 1999; Chen 2001; Koch 2002). The fees of
ERP consultants are included in those costs.
Although many organizations have successfully implemented ERP
systems, studies have found that between 40 and 50 percent of ERP
implementations are only partially successful and 20 percent of ERP
implementations are scrapped as total failures (Davenport 1998; Chen
2001). Donovan (2002) suggests that 90 percent of the organizations that
have implemented ERP systems have not had a truly successful
implementation the first time. These implementations fail for many
reasons such as a failure to effectively reengineer business processes,
a lack of management support for the project, ineffective
pre-implementation activities, and inexperienced consultants or IT
personnel driving the implementation.
Because these systems are so costly and risky, most organizations
use consultants to assist in the implementations. Good consultants can
handle many tasks or projects that organizations are unwilling or unable
to handle. External consultants may bring an outside perspective to the
implementation team, which may contribute to the implementation's
success (Willcocks and Sykes 2000; Thong et al. 1994; Spacek 2000;
Mische 2000). In many instances, consultants have been an invaluable
resource for the organizations implementing ERP systems. Ifenedo (2008)
shows that the use of quality external experts is critical for the
success of ERP systems. The study also shows that the user of external
experts is more important that even management's support of the ERP
system.
Several studies have examined the factors that lead to a successful
ERP implementation. Bingi et al. (1999) outline several critical
factors, including the use of ERP consultants, for success in an ERP
implementation. Nah et al. (2001) determine, through a review of the
literature, that 11 factors are critical to ERP implementation success.
Previous studies in their literature reviews suggest that the ERP
implementation team should include a mix of consultants and internal
personnel. Yap et al. (1992) found that for organizations that used
consultants, the likelihood of a successful implementation was
associated with the use of an effective consultant. In other words, the
effectiveness of the consultant is important to the success of an
information system implementation. Loh and Koh (2004) found that for
small- and medium-sized organizations, consultants are key players in
the early phases of ERP implementations.
Several other studies or articles offer advice about the use of
consultants for IT or ERP implementations. Ko et al. (2005) examine the
relationship between consultants and their clients. The authors find
that it is important to create an environment in which the consultants
and client personnel can interact frequently and freely so that
knowledge can flow freely between the two parties. Spacek (2000)
indicates that external consultants should be used because they bring an
external and hopefully independent point of view to the implementation
team. Slater (1999) suggests that using consultants may lead to getting
things right the first time; however, many implementations in which
consultants were used still have not been completely successful. Based
on the arguments above, organizations should be encouraged to use
external consultants for their IT and ERP implementations.
Several studies have identified characteristics of effective
consultants. In their book, Arnoudse et al. (1989) identify four sets of
skills that lead to effective information system consulting. These sets
of skills are technical skills, human interaction skills, business
context skills, and consulting framework skills.
Yap et al. (1992) use three characteristics to associate the
effectiveness of consultants with information system implementation
success. These three characteristics are consultant experience,
consultant capability, and consultant effectiveness during the
feasibility study. Experience typically leads to capability.
Thong et al. (1994) identify four variables to assess a
consultant's performance during an information system
implementation. These variables are consultant effectiveness in
performing analysis of requirements of the business, consultant
effectiveness in recommending the proper solution, consultant
effectiveness in managing the implementation, and the relationship
between the consultant and the other parties involved with the project.
Mische (2000) proposes several characteristics that are desirable
in information systems consultants. These characteristics include
communication skills; the consultant's background, experience, and
familiarity with the issues in the implementation; the ability to
determine the organization's needs and expectations and assign
personnel to meet those needs and expectations; personality and
chemistry; objectivity; expertise with systems implementations; and
commitment to quality.
While many of the cited studies have examined characteristics of IT
consultants, those characteristics should also be present in ERP
consultants. ERP systems are quite different than other IT systems, but
without good consultants with appropriate characteristics, the chances
for failure of either IT or ERP systems increase. Knowing the
characteristics of an effective ERP consultant is quite important for
the organization to pick the best consultant for their ERP
implementation.
THE ERP IMPLEMENTATION MODEL
One area that has been addressed in the ERP literature is the
attempt to develop definitive models for ERP implementations. For
example, Parr and Shanks (2000) and Markus and Tanis (2000) have
proposed different ERP implementation models. ERP implementation models
allow a deeper understanding of what occurs during the implementation.
Parr and Shanks (2000) developed what they called the project phase
model (PPM) that has three major phases: planning, project, and
enhancement. To give more detail of the actual installation, the project
phase is further divided into five subphases: set-up, reengineering,
design, configuration and testing, and installation. The PPM includes
elements from two other models developed by Ross (1998) and Bancroft et
al. (1998).
In the PPM, the planning phase includes tasks such as selection of
the ERP system, assembly of the steering committee, and determination of
the scope of the implementation. The project phase includes tasks to get
the new ERP system up and running. These tasks include reengineering
business processes, installing the software, mapping the processes to
the software, detailed design, configuring the system to match the
company, testing the new ERP system, and installing the ERP interfaces
on the users' computers. Finally, the enhancement phase includes
monitoring the system and repairing, maintaining, and upgrading the ERP
system.
In their article, Parr and Shanks (2000) used the PPM and case
studies of two companies that had recently implemented ERP systems to
determine which critical success factors (CSFs) are necessary in the
particular phases and subphases of their model. However, the study makes
no mention of whether the PPM is actually used by organizations as a
model to be followed in the implementation of ERP systems.
Markus and Tanis (2000) developed a theoretical model called the
enterprise system experience model (ESEM) that has four phases:
chartering, project, shakedown, and onward and upward. In the ESEM, the
chartering phase includes activities leading up to the beginning of the
implementation, including selecting the ERP system, identifying a
project manager, and building a case for the implementation. The project
phase involves getting the system up and running and includes selection
of the project team, modeling and reengineering current processes,
software configuration, integration of "bolt on" software and
other systems that the organization wishes to retain, data conversion,
training, and rollout. The shakedown period includes time for employees
to get acclimated to new information systems and includes tasks such as
debugging and retraining users, among others. The onward and upward
phase includes tasks such as continuous business improvement, user
training, and a post-implementation assessment.
While both the PPM and ESEM are good models, neither was judged a
perfect fit for this study. For example, the PPM includes the subphases
of the project phase that allow for a more detailed look at the project
phase In the present study, more detail is needed to differentiate the
tasks done by the consultants in an ERP implementation. In addition, the
ESEM gives a good description of the shakedown phase. This phase is
important because users of ERP systems must have time to become
acclimated to the new system, but the PPM does not include a separate
phase in which users become acclimated to the system.
Since neither of the aforementioned models is a perfect fit for
this study, a model that combines elements of both the PPM and the ESEM
is developed. The new ERP implementation model consists of four phases:
planning, project, shakedown, and operation. The project phase is
further divided into five subphases: reengineering, design,
configuration and integration, testing, and installation and conversion.
The first phase is the planning phase. Activities involved in
planning are building a case for the implementation, selection of the
steering committee, ERP system, and project manager, and approval of a
budget, among others. The project phase includes activities involved in
configuring the system and integrating the new system with other
applications, such as reengineering, design, configuration of the system
to match the organization's processes, integration of existing
systems, testing the new system, installation of the interfaces on the
users' computers, and conversion of data to the new system. The
shakedown phase includes tasks such as debugging and fine-tuning the
system and training users. Finally, the operation phase includes
modifying the ERP system, debugging, repair and maintenance, and
post-implementation analyses. Table 1 summarizes the activities that are
included in each of the phases and subphase of this ERP implementation
model in greater detail.
HYPOTHESES
One aspect of this research addresses whether individuals feel that
consultants are effective in the tasks that they perform and are
necessary members of the implementation team on a phase-by-phase basis.
Effectiveness is commonly defined as producing a definite or desired
result and, in this study, refers to how well a consultant performed the
tasks that are involved in a particular phase of an ERP implementation.
Consultant effectiveness can be concretely measured by how well a task
is performed by the consultant. Conventional wisdom suggests that when a
consultant is effective in their tasks, their work should lead to a
successful ERP implementation. Following this discussion, the following
hypothesis is examined.
H1: Relative differences exist in the effectiveness of consultants
across the phases of an ERP implementation and overall.
Necessity is commonly defined as being indispensable, unavoidable,
or required and, in this study, refers to whether an individual
perceives that ERP consultants are an indispensible part of the
implementation team. Consultant necessity is more of a perception of
whether one feels that the consultants are needed on the implementation
team. Because of the complex nature of ERP implementations, consultants
are certainly thought to be necessary; however, no prior research has
examined the specific phases in which consultants are necessary or if
they are perceived as being more necessary in one phase versus other
phases. Following this discussion, the following hypothesis is examined.
H2: Relative differences exist in the necessity for consultants
across the phases of an ERP implementation and overall.
It is possible that a consultant could be very effective in a
particular phase, but not necessary to a particular organization in that
phase. Conversely, it is possible that an organization would have the
need for assistance in a particular phase, but their experience suggests
that consultants lack effectiveness in that particular phase.
Accordingly, the effectiveness and necessity measures are not expected
to be perfectly correlated.
As discussed above, prior studies have shown that certain
characteristics impact the effectiveness of information system
consultants. These prior studies included a total of 18 characteristics,
many of which are similar. An examination of these characteristics to
determine whether they were essentially the same as other
characteristics pared the 18 characteristics down to eight distinct
characteristics. These characteristics and their descriptions are:
Technical skills and knowledge--knowledge of a variety of
technologies, including ERP systems
Human interaction and communication skills--ability to communicate
with and relate to others
Business context skills--ability to gain an understanding of the
detailed operations of the client organization
Consulting skills and knowledge--ability to perform analysis of
information requirements and feasibility analyses (among other things)
and to make proper recommendations
Objectivity--freedom from bias regarding the organization and ERP
vendors
Experience with ERP implementations--whether the consultant has
been involved in ERP implementations and the similarity of previous
implementations
Commitment to quality--responsibility to perform the implementation
to the best of the consultant's ability and to produce the best
possible product
Management ability--ability to manage projects and implementations
and supervise personnel.
This study examines whether these characteristics have some impact
on the effectiveness of ERP consultants. Following this discussion, the
following hypothesis is examined.
H3: Individual characteristics possessed by ERP consultants
contribute to the perception of consultant effectiveness in ERP
implementations to varying degrees.
Prior studies (Yap et al. 1992; Thong et al. 1994) indicate that
all characteristics are rated similarly, and no single characteristic
stands out as having more of an impact on effectiveness. For this study,
the expectation is that the characteristics contribute to the
effectiveness of consultants to varying degrees; therefore, no
predictions are made regarding which characteristics have more of an
impact on effectiveness.
METHODOLOGY
A research instrument was administered via email and the Internet
to chief information officers (CIO), vice presidents of IT, and IT
directors and managers using rented lists of subscribers to the
magazines CIO and Enterprise Systems Journal and to individuals who had
indicated involvement with various ERP user groups. Subscribers to CIO
and Enterprise Systems Journal were used because the target audiences of
these magazines are IT executives who are looking for an information
source that focuses on their needs and that provides information clearly
and concisely. Since virtually all Fortune 1000 companies and many
smaller organizations have implemented ERP systems, many CIOs and IT
managers who subscribe to these magazines have probably been involved
with ERP implementations to some degree. Members of ERP user groups are
used because many of these individuals have been involved to some degree
with an ERP implementation.
Four thousand email addresses were rented from each of the two
magazines based on the respondents' positions in their
organizations (CIO, vice president of IT, or IT director). The
transmitted email included a description of the study and the purpose of
the research instrument. Prior research indicates that response rates
for emails surveys are typically higher than for traditional mail
surveys when the email is sent to computer savvy individuals (Sheehan
2001; Flicker and Schonlau 2002; Schuldt and Totten; Ali and Jones
2002). However, in an era of "spam" and the spread of
destructive programs via email, the response rates for this study were
quite low. Of these 8,000 individuals, only 43 (0.5 percent) completed
the research instrument. However, some overlap may have been present. In
other words, some subscribers chosen by CIO may have also been chosen by
Enterprise Systems Journal, thus decreasing the real number of
individuals in the population. A decrease in the number of individuals
in the population would, in turn, increase the response rate. Because
the two journals retained control over the emails and email addresses,
so there is no way of knowing if any overlap occurred.
Since so few responses were received via rented email lists, the
public domains of websites for the SAP, Oracle, and J.D. Edwards user
groups were searched to find as many names and email addresses as
possible. Other ERP user group websites could not be found or required
registration and membership in the user group to access information on
the website; therefore, no email addresses were found for ERP user
groups such as PeopleSoft and Baan. Also, since the data collection,
there have been mergers among ERP systems vendors, so these user group
websites may no longer exist. The user group website searches netted a
total of 1,675 viable email addresses. Of the individuals emailed using
this method, 143 (8.8 percent) responded to the research instrument. As
with the two magazines, there may have been some overlap in subjects. In
other words, some of the subjects chosen from the user groups may have
also been in the group of individuals chosen from one or both magazines.
Again, such a situation would reduce the number of individuals in the
population and thus increase the overall response rate. In total, 9,675
emails were sent, and 186 responses were received for an overall
response rate of 1.9 percent.
To test for non-response bias, early and late respondents were
compared on organization size, industry, and ERP consultant type. The
results of Chi-square tests showed no significant differences among
these key organizational characteristics.
While the response rate for the study is low, the demographics show
that the respondents are quite diverse. Several interesting results were
noted. Of the 112 respondents who answered the item regarding the type
of consultant used, 74 (66.1 percent) used consultants who were owned by
or involved with ERP vendors. One questions whether a consultant can be
(or should be) truly objective in a consulting engagement when such a
relationship exists. A large majority of the respondents (136, or 73.1
percent) indicated that their organizations use SAP. Only 23 respondents
(12.4 percent) indicated that their organizations had fewer than 500
employees, so most organizations were fairly large. Most respondents
(72, or 38.7 percent) work in organizations that they classify as
manufacturing companies.
One hundred thirty-two respondents (74.6 percent) felt that the
implementation was successful, while only 45 respondents (25.4 percent)
felt that the implementation was not successful or was only partially
successful. This rate of unsuccessful implementations is lower than
those found in other studies such as Davenport (1998), Chen (2001), and
Donovan (2002). Respondents gave varying reasons why they felt that
their implementations were not successful. Reasons given were: the
organization had not achieved its full return on investment (ROI); the
consultants "milk the clients;" no knowledge transfer took
place; cost overruns; inadequate time, financial resources, testing, or
training; resistance to change; lack of communication between management
and the consultants; long implementation time, and unrealistic
expectations. Each of these causes of unsuccessful implementations has
also been noted in the studies mentioned above.
THE RESEARCH INSTRUMENT
The research instrument included four parts. Part 1 was an
introductory section that explained the purposes of the study and
included definitions of consultant effectiveness and necessity to help
the individuals differentiate the two items. Instructions informed the
respondents of pop-up boxes that included descriptions of each of the
phases and subphases of the ERP implementation model and of the
characteristics of consultants. This information was provided to ensure
that the individuals understood what happened in each of the phases and
what each characteristic meant. The final item of Part 1 was a question
that asked whether the respondent had ever been involved in an ERP
implementation. Depending on the response to this question, the
respondent was redirected to one of two forms of the research instrument
tailored to an individual who either had experience with an ERP
implementation or who did not have such experience. Part 1 of the
research instrument is included in Appendix 1.
For individuals who had experience with ERP implementations (ERP
implementers), Part 2 of the research instrument included items
regarding both consultant effectiveness and necessity during an actual
ERP implementation. These items asked the respondents to rate their
consultants on effectiveness and necessity in the individual phases and
subphases of the ERP implementation model and overall in the
implementation using a Likert-type scale from one (Strongly Disagree) to
seven (Strongly Agree). For non-ERP implementers, Part 2 of the research
instrument included items regarding only consultant necessity because
they had no way of rating consultants on their effectiveness. Although
data were collected from these non-ERP implementers, their responses
have been omitted from most analyses because so few non-ERP implementer
responses were received (nine of 186). No significant differences were
observed in the responses between implementers and non-implementers.
Only Part 2 of the research instrument differed between ERP implementers
and non-ERP implementers. The remainder of the research instrument is
included in Appendix 2.
Part 3 of the research instrument was included to gather data on
which characteristics the respondents felt contribute to the
effectiveness of ERP consultants. All respondents completed Part 3
because even non-ERP implementers should have some perception of whether
a characteristic contributes to the effectiveness of a consultant. The
respondents were first asked to indicate the phase or subphase in which
they felt that their consultants were or would be (in the case of
non-ERP implementers) most effective. The respondents were then asked to
rate the characteristics' contributions to effectiveness using
Likert-type scales from one (Low Contribution) to seven (High
Contribution). Part 3 also included an open-ended item that asked the
respondents to indicate any additional characteristics that they felt
contributed to consultant effectiveness. Finally, respondents were asked
to indicate the phase or subphase in which they felt their consultants
were or would be (in the case of non-ERP implementers) least effective.
These last two items allowed for the consideration of characteristics
not included on the research instrument and also allowed confirmation of
the results of other parts of the study.
Part 4 of the research instrument included questions designed to
gather demographic data about the respondent. All respondents were asked
to complete Part 4.
A pretest of the research instrument was performed to determine if
changes were necessary. Four members of ERP implementation teams from
two organizations that had recently implemented ERP systems were asked
to complete the research instrument and provide any comments regarding
the items on the instrument and the overall understandability of the
instrument. These individuals were also asked whether they felt that
effectiveness and necessity should be more clearly defined to eliminate
confusion about whether they represented the same construct. All
individuals indicated that the instrument and instructions were
understandable and that they understood the difference between
effectiveness and necessity. Only insignificant changes were suggested.
The responses from the pretest are not included in the results.
RESULTS
CONSULTANT EFFECTIVENESS AND NECESSITY
Analyses of means and frequencies were performed on both consultant
effectiveness and necessity by phase. Higher means indicate that a
consultant was considered to be more effective in the tasks they
performed or more necessary to the implementation. Table 2 shows the
means for consultant effectiveness and necessity for each phase and
subphase of the implementation model. The highest mean rating for
consultant effectiveness is for the configuration and integration
subphase, with a mean of 5.21. Consultant effectiveness in the operation
phase received the lowest mean rating, with a mean of 3.95. Overall,
consultant effectiveness was rated 4.83, indicating that the respondents
feel that their consultants were effective overall but not
overwhelmingly so.
Part 3 of the research instrument asked the respondents to indicate
the phase or subphase in which they felt their consultants were most
effective and least effective. An examination of frequencies for these
responses shows that the vast majority of respondents indicated that
their consultants were most effective in the configuration and
integration phase. Most respondents indicated that their consultants
were least effective in the operation phase. These results are
consistent with the ratings in Table 2.
Table 2 also shows the means for consultant necessity for each
phase and subphase. The highest mean rating for necessity was in the
configuration and integration subphase, with a mean of 5.52. Consultant
necessity in the operation phase received the lowest mean rating, with a
mean of 3.60. The overall mean rating was 5.20, indicating that the
consultants were perceived to be necessary overall.
Table 2 also includes the rankings based on the mean ratings of the
phases and subphases. The mean ratings for consultant effectiveness and
consultant necessity are consistent. The only difference in the rankings
occurred between the reengineering and testing subphases. For consultant
effectiveness, the testing subphase was ranked fifth, and the
reengineering subphase was ranked sixth. Conversely, for consultant
necessity, the testing subphase was ranked sixth, and the reengineering
subphase was ranked fifth.
Although the rankings in the phases are quite similar,
effectiveness and necessity were not expected to be perfectly
correlated, as stated in an earlier section. Table 3 shows that for
effectiveness and necessity by phase, correlations range from .506 for
the testing subphase to .678 for the configuration and integration
subphase. Thus, while effectiveness and necessity appear to be related,
they are not perfectly correlated.
Hypothesis 1 states that relative differences exist between
perceptions of consultant effectiveness in the various phases and
subphases of an ERP implementation. Repeated measures analysis of
variance (ANOVA) is used in this study because the study examines the
respondents' ratings for successive phases within an ERP
implementation, i.e. repeated polling of the same subjects. According to
Stevens (1999), repeated measures ANOVA is appropriate when the same
subjects are "polled" about successive points during a
process. The results of the repeated measures ANOVA show that
significant differences exist in the perceptions of the effectiveness of
consultants across the implementation model (F=22.820, p<.001).
Therefore, Hypothesis 1 is supported.
To determine between which phases the differences occur, a post hoc
test using a Bonferroni adjustment was performed. Table 4 shows the
differences among the mean ratings for consultant effectiveness between
the individual phases. These differences are calculated as the mean for
the phase on the top of the table minus the mean for the phase on the
left of the table. Table 4 shows that, in general, respondents rated
consultant effectiveness significantly lower than all other phases and
lower than overall effectiveness. The respondents also rated consultant
effectiveness in the configuration phase significantly higher than all
other phases except planning. Finally, the respondents rated consultant
effectiveness in the shakedown phase significantly lower than in all
phases except reengineering, testing, and operations, which is rated
lower than shakedown.
These results suggest that ERP consultants must work on becoming
more effective in the phases that are rated significantly lower than
others. Clients might feel that resources are wasted when a consultant
is not perceived to be effective in the tasks performed in any given
phase of the ERP implementation. Reengineering is particularly important
in ERP implementations, and consultants were rated significantly lower
than in the design phase and in the configuration and integration
subphase.
Hypothesis 2 states that relative differences exist in the ratings
regarding consultant necessity in the phases and subphase of the
implementation model. The results of a repeated measures ANOVA show that
significant differences exist in the perceptions of the necessity for
consultants across the implementation model (F=43.782, p<.001).
Therefore, Hypothesis 2 is also supported.
Table 5 shows the results of a post hoc test using a Bonferroni
adjustment to determine where the differences between phases exist.
Consultant necessity in the configuration phase stands out as being more
highly rated than in all other phases, and the ratings for the shakedown
and operation phases are lower than in most other phases.
Perhaps consultants should "market" themselves more
aggressively in the phases in which their necessity is rated
significantly lower than others. Alternatively, ERP consultants may be,
in reality, less necessary to the typical user in certain stages of the
implementation. If so, a more effective strategy may be for consultants
to concentrate their efforts in the stages of the implementation in
which they are most effective. These results give consultants a starting
point for focusing their resources on making ERP implementers feel a
greater need for consultants' services throughout an
implementation, particularly in phases in which consultants are
perceived as necessary for ERP success.
Table 6 shows the mean ratings on consultant effectiveness and
necessity by phase and the differences between the mean ratings on a
phase-by-phase basis. To determine whether the differences were
significant, paired t-tests were performed on the mean ratings by phase.
If the respondents rated their consultants higher on effectiveness than
on necessity, the consultants appear to be outperforming the
expectations of their clients. Alternatively, if the respondents rated
their consultants higher on necessity than on effectiveness, the
consultants appear to be underperforming or not living up to
expectations.
Respondents rated consultants as more effective than necessary only
in the operation phase (at p<.01), indicating that the consultants
are outperforming their clients' expectations only in the operation
phase. On the other hand, the respondents feel that their consultants do
not meet expectations in the planning phase (at p<.01), in the design
(at p<.001) and configuration and integration (at p<.01)
subphases, or overall (at p<.001). Such results should be of concern
to the consulting industry and may also be a factor in the high rate of
unsuccessful ERP implementations that have occurred over the years.
Consultants must improve their skills and abilities and apply these
skills more effectively in these phases and subphases and in the overall
implementation. If clients continue to perceive that consultants are
ineffective in certain phases, they may see the consultants as
unnecessary and look elsewhere for assistance in their ERP
implementations.
CHARACTERISTICS OF CONSULTANTS
Table 7 shows the mean ratings for the individual characteristics
and their contributions to the effectiveness of ERP consultants. Higher
ratings for the characteristics indicate that the respondents feel that
the characteristic makes a high contribution to the effectiveness of ERP
consultants.
Technical skills and knowledge received the highest mean rating
with a mean of 5.59. Consultants' ability to manage ERP
implementations received the lowest mean rating, with a mean rating of
4.73. Table 7 includes the mean ratings and the relative rankings of all
characteristics included on the research instrument. All characteristics
have a mean rating above four (neutral rating), so all characteristics
appear to contribute to the effectiveness of ERP consultants. This
result is consistent with prior studies such as Yap et al. (1992), Thong
et al. (1994), and Mische (2000). Respondents who indicated that their
ERP implementations were successful consistently indicated that the
characteristics, except business context skills and objectivity,
contributed significantly more to consultant effectiveness (at p<.05)
than respondents who indicated that their ERP implementations were not
successful. These results are not unexpected given that these
respondents' implementations were successful and they were likely
happy with their consultants and their skills.
Hypothesis 3 states that characteristics possessed by ERP
consultants contribute to the effectiveness of the consultants to
varying degrees. To examine this hypothesis, two regression models are
estimated using two different dependent variables. For these models, the
characteristics of ERP consultants (independent variables) are used to
determine whether ERP consultants are effective (dependent variables).
Table 8 shows the eight independent variables used in the regression
analyses. Explanations of the characteristics are given in a previous
section. The general regression equation can be written as:
RATING FOR PHASE = [[beta].sub.0] + [[beta].sub.1]TECHNICAL +
[[beta].sub.2]COMM + [[beta].sub.3]BUSINESS + [[beta].sub.4]CONSULT +
[[beta].sub.5]OBJECT + [[beta].sub.6]CONSULTERPEXPER +
[[beta].sub.7]QUALITY + [[beta].sub.8]MANAGE.
The prediction for the regression coefficients is that all
variables have a positive effect on ERP consultant effectiveness because
previous studies have shown that these characteristics contribute to the
effectiveness of IS consultants.
The research instrument instructed the respondents to consider the
phase in which they felt their consultants were the most effective.
Therefore, for the first regression model, the dependent variable
includes the ratings given by of each respondent for the phase in which
their consultant was rated as most effective. In other words, if a
respondent indicated that the consultant was most effective in the
planning phase, that respondent's rating of effectiveness in the
planning phase was used as the dependent variable for that observation.
If a respondent rated the consultant as most effective in the
reengineering subphase, that respondent's rating on effectiveness
in the reengineering subphase was used as the dependent variable. This
method was used for all respondents' ratings.
Table 9 shows the results of the regression equation for the
characteristics of ERP consultants and their contributions to ERP
consultant effectiveness. The regression equation is significant at
p<.001, and the adjusted [R.sup.2] is .192. The TECHNICAL, BUSINESS,
and OBJECT variables are all significant at p<.10. These results
indicate that technical skills and knowledge, business context skills
and objectivity are all moderately related to the effectiveness of ERP
consultants. All of the regression coefficients are in the predicted
direction, except for the coefficient for the BUSINESS variable. The
negative coefficient on the BUSINESS variable indicates that a
consultant's ability to gain knowledge about and understand the
detailed operations of the client organization is inversely related to
effectiveness in the planning phase, although only moderately (p=.070).
Such a result could indicate that as a consultant learns more about a
business, he or she becomes too comfortable and fails to listen to the
needs of the client. This result could also mean that consultants
attempt to apply "cookie-cutter" solutions to their
clients' businesses or giving what they think the client needs
rather than what is really needed.
Very few respondents indicated that their consultants were most
effective in several phases. In particular, 10 or fewer respondents
indicated that their consultants were most effective in the
reengineering, testing, and installation and conversion subphases and
the shakedown and operation phases. Only the configuration and
integration subphase received a sizeable number of responses. The
results in Table 9 should be interpreted with this in mind.
A second regression model with a different dependent variable was
used to analyze the characteristics of ERP consultants and their
contributions to the overall effectiveness of ERP consultants. While the
respondents were asked to think about the phase in which their
consultants were most effective when rating the characteristics, an
analysis of the relationship of the characteristics and the overall
effectiveness of the consultants is a reasonable progression of the
examination of consultant effectiveness. The dependent variable is the
average of each respondent's ratings in all phases and subphases of
the ERP implementation model. While the respondents indicated a rating
on their consultants' effectiveness overall, the average of their
ratings for effectiveness in the individual phases is still a reliable
measure of overall effectiveness of consultants. The averages of the
respondents' ratings for all phases are highly correlated (r=.855)
to the respondents' ratings for the consultants overall. Such a
high correlation indicates that the respondents were consistent in their
ratings by phase and overall.
Table 10 shows the results of the regression on the overall rating
of ERP consultant effectiveness using the average of each
respondent's ratings in all phases. This regression equation is
significant at p<.001, and the adjusted [R.sup.2] for the equation is
.394. The TECHNICAL, QUALITY, and MANAGE variables are significant at
p<.05, and the coefficients of these variables are in the predicted
directions, so the respondents feel that these characteristics
contribute to effectiveness of consultants overall. The fact that only
three variables are significant is somewhat unexpected because all
characteristics in this study have previously been shown to contribute
to the effectiveness of IT consultants. Perhaps the respondents in this
study place more emphasis on other characteristics or assume that
consultants possess some of these characteristics by default. Two
variables, BUSINESS and CONSULTEXPER, although not significant, were not
in the predicted direction, while all others are in the predicted
direction.
The respondents were given room on the research instrument to
indicate any other characteristics that they felt might contribute to
ERP consultants' effectiveness. Examples of additional potential
characteristics that may contribute to consultant effectiveness are
product knowledge, experience in the client's specific industry,
the ability to transfer knowledge to the client's personnel, and
honesty.
CONTRIBUTIONS, LIMITATIONS, AND CONCLUSION
One purpose of this study is to determine whether a consultant is
perceived as being more effective and more necessary in certain phases
of ERP implementations. Respondents reported that their consultants were
both most effective and most necessary in the configuration and
integration subphase, while their consultants were least effective in
the operation phase, thus showing that the perception that consultants
are better in certain phases does exist. Results also show that
consultants are consistently rated higher on their necessity than on
their effectiveness in the phases of the ERP implementation model. Only
in the operation phase did the respondents indicate that their
consultants were more effective than necessary. These results indicate
that consultants are not performing up to the expectations of their
clients.
Another purpose of this study is to determine whether certain
characteristics of ERP consultants contribute to the effectiveness of
ERP consultants. The results of regression analysis of the
characteristics of consultants are not conclusive. In a model using the
respondents' ratings for the phase or subphase in which the
respondent indicated that their consultants were most effective, the
regression equation was significant, but none of the variables were
significant, indicating that none of the characteristics contribute to
consultant effectiveness. The second regression analysis examined the
overall effectiveness of ERP consultants. In this analysis, the
regression equation is significant, and the model includes three
significant variables--TECHNICAL, QUALITY, and MANAGE. This result shows
that the respondents feel that technical skills and knowledge, a
commitment to producing quality work, and ability to manage the ERP
implementation all contribute to the effectiveness of ERP consultants
overall.
This study uses a research instrument to captures respondents'
perceptions on consultant effectiveness and necessity. The limitations
applicable to using a research instrument apply. Examples of these
limitations are self-selection bias, honesty of respondents, and a limit
on the ability to expand on answers, among others.
Another limitation of this study is the low response rate. While
the response rate is low, enough responses were collected to enable
meaningful and sound analyses, and the sample is diverse. Another
limitation is the fact that most respondents were SAP users, which may
have introduced bias into the results. Also, some consultants may have
completed the research instrument, which would likely bias the results
in favor of consultants.
Additionally, the sample of subjects is not completely random since
ERP user group websites were examined to find names of ERP users.
However, the subjects from the two magazines were chosen at random based
on their positions within their organizations. There may also have been
some overlap between the two groups of subjects. That is, some of the
ERP users group subjects may also have been subjects selected by CIO or
Enterprise Systems Journal; however, there is no way of knowing this for
sure.
Finally, the intercepts in the two regression equations are
significant. Such a result generally indicates that independent
variables not included in the analysis contribute to the dependent
variable. Because of the nature of behavioral research and the methods
used in this type of research, it is not uncommon to have potentially
significant independent variables that are not included. This result
could have some impact on the predictive ability and generalizability of
the regression equations. Also, very few independent variables
(characteristics of consultants) are significant. In this study, the
independent variables are taken from prior studies and are not
exhaustive.
In spite of these limitations, the results of this research should
provide many benefits to the companies who have implemented them. The
knowledge of the phases in which ERP consultants are most effective and
necessary and which characteristics contribute to consultants'
effectiveness should provide an advantage to many organizations that
decide to implement ERP systems.
The results of the study provide organizations that plan to
implement ERP systems with knowledge of the phases of ERP
implementations in which consultants are perceived to be most effective
and necessary. By using the consultants only when necessary or only in
certain phases in which the consultants are the most effective,
organizations can utilize internal personnel and resources and reduce
their expenditures for the consultants. The results of this study
provide the knowledge to assist organizations in achieving cost savings
regarding consultants.
Consultants themselves can also benefit from this research because
they can see which areas they should stress more in their practices and
which areas they should improve to give their clients more value. Any
consulting firms that are interested in entering the ERP consulting area
can see the areas on which to concentrate from the beginning. Of course,
the resources and abilities that the vendors and consulting firms
possess may limit their choices in strategy, but this research gives
them insight on how their potential clients feel about the use of
consultants for their ERP implementations.
REFERENCES
Ali B. and M. Jones. 2002. Using Electronic Mail as a Method of
Surveying Medical Students' Opinions and Attitudes. Medical
Education 36 (4): 392.
Arnoudse, D. M., L.P. Ouellette, and J. D. Whalen. 1989. Consulting
Skills for Information Professionals. Homewood, IL: Dow Jones-Irwin.
Bancroft, N., Seip, H., and Sprengel, A. 1998. Implementing SAR
R/3, 2nd Edition. Greenwich, CT: Manning Publications.
Bingi, P., M. K. Sharma, and J. K. Godla. 1999. Critical Issues
Affecting an ERP Implementation. Information Systems Management 16 (3):
7-14.
Chen, I. Planning for ERP Systems: Analysis and Future Trend. 2001.
Business Process Management 7 (5): 374-386.
Davenport, T. H. 1998. Putting the Enterprise into the Enterprise
System. Harvard Business Review 76 (4): 121-132.
Donovan, R. M. 2002. Why the Controversy over ROI from ERP?
Accounting Software 411.com (October 29). Retrieved from http://
www.as411.com/AcctSoftwaie.nsf/00/prCA43313C9BF808FF86256C6200173446.
Fricker, R. D., and M. Schonlau. 2002. Advantages and Disadvantages
of Internet Research Surveys: Evidence from the Literature. Field
Methods 14 (4): 347-368.
Glover, S. M., D. F. Prawitt, and M. B. Romney. 1999. Implementing
ERP. Internal Auditor 61 (1): 40-45.
Gupta, A. 2000. Enterprise Resource Planning: The Emerging
Organizational Value System. Industrial Management & Data Systems
100 (3/4): 114-118.
Jacobs, F. R., and D. C. Whybark. 2000. Why ERP? A Primer on SAP
Implementation. Boston, MA: The McGraw-Hill Companies, Inc.
Karakanian, M. 1999. Choosing an ERP Implementation Strategy. Year
2000 Practitioner 2 (7): 1-6.
Ko, D. G., L. J. Kirsch, and W. R. King. 2005. Antecedents of
Knowledge Transfer from Consultants to Clients in Enterprise System
Implementations. MIS Quarterly 29 (1): 59-84.
Koch, C. 2002. The ABCs of ERP. CIO.com (February 7). Retrieved
from http://www.cio.com/research/erp/edit/erpbasics.html.
Loh, I. C., and S. C. L. Koh. 2004. Critical Elements for a
Successful Enterprise Resource Planning Implementation in Small- and
Medium-Sized Enterprises. International Journal of Production Research
42 (17): 3433-3455.
Markus, M. L., and C. Tanis, 2000. The Enterprise System
Experience--From Adoption to Success. In: Zmud, R. W., ed. Framing the
Domains of IT Management: Projecting the Future Through the Past.
Cincinnatti, OH: Pinnaflex Educational Resources, Inc., 173-207.
Mische, M. A. 2000. Choosing a Systems Integrator. Enterprise
Systems Integration, J. Wyzalek, Editor. Boca Raton, FL: Auerbach,
873-885.
Nah, F. F., J. L. Lau, and J. Kuang. 2001. Critical Factors for
Successful Implementation of Enterprise Systems. Business Process
Management 7 (3): 285-296.
O'Leary, D. E. 2000 Enterprise Resource Planning Systems:
Systems, Life Cycle, Electronic Commerce, and Risk. Cambridge, UK:
Cambridge University Press.
Parr, A., and G. Shanks. 2000. A Model of ERP Project
Implementation. Journal of Information Technology 15 (4): 289-303.
Ross, J. W. 1998. The ERP Revolution: Surviving Versus Thriving.
Cambridge, MA: Centre for Information Systems Research, Sloan School of
Management, Massachusetts Institute of Technology.
Schuldt, B. A., and J. W. Totten. 1994. Electronic Mail vs. Mail
Survey Response Rates. Marketing Research 6 (1): 36-40.
Slater, D. 1999. An ERP Package for You ... and You ... and You ...
and Even You. CIO 12 (9): 31-36.
Stevens, J. 1999. Intermediate Statistics: A Modern Approach, 2nd
Edition. Mahwah, NJ: Lawrence Erlbaum Associates, Inc.
Spacek, M. 2000. Is Your IT Project Stuck in Analysis/Paralysis
Mode? Strategic Finance 81 (7): 24-28.
Thong, J. Y. L., C. S. Yap, and K. S. Raman. 1994. Engagement of
External Expertise in Information Systems Implementation. Journal of
Management Information Systems 11 (2): 209-232.
Willcocks, L. P., and R. Sykes. 2000. The Role of the CIO and IT
Function in ERP. Communications of the ACM 43 (4): 32-38.
Yap, C.S., C.P.P. Soh, and K.S
Eddie Metrejean, Georgia Southern University
Morris H. Stocks, University of Mississippi
Table 1. Activities Included in the Phases of the Current ERP
Implementation Model
Phase/Subphase Activities
Planning Phase Building a business case for ERP system
Phase Assembly of a steering committee
Selection of an ERP system
Selection of project managers
Determination of scope of project
Approval of budget and schedule for
implementation
Project Phase
Reengineering Subphase Detailed examination and evaluation of
current processes
Installation of ERP system
Design Subphase High-level and detailed design
Prototyping to get feedback from users
Configuration and Creation of the detailed configuration
Integration Subphase based on processes
Preliminary testing of the configurations
Building and testing interfaces and reports
Testing Subphase Testing of the system as a whole
Installation and Building necessary networks
Conversion Subphase Installing software on users' PCs
Training of users Conversion of data and archive files
Shakedown Phase Testing the system in operating mode
Debugging and fine-tuning the system
Retraining users Staffing to handle temporary inefficiencies
Operation Phase Continuous improvement and modifications
Debugging
Repair and maintenance
Continuous user Post-implementation benefits analysis
training
Table 2. Mean Ratings on Consultant Effectiveness and Necessity by
Phase
Consultant Consultant
Effectiveness Necessity
Phase or Mean Standard Rank Mean Standard Rank
subphase n=177 Deviation n=177 Deviation
Planning phase 4.83 1.47 3 5.23 1.771 3
Reengineering 4.46 1.505 6 4.74 1.816 5
subphase
Design 4.88 1.462 2 5.35 1.514 2
subphase
Configuration 5.21 1.534 1 5.52 1.612 1
and
integration
subphase
Testing 4.67 1.539 5 4.66 1.713 6
subphase
Installation 4.82 1.5 4 4.87 1.786 4
and
conversion
subphase
Shakedown 4.41 1.434 7 4.44 1.647 7
phase
Operation 3.95 1.625 8 3.6 1.739 8
phase
Overall 4.83 1.284 5.2 1.452
Note: Ratings were on a scale of 1 to 7 with 1 being strongly
disagree that consultant is effective-necessary and 7 being
strongly agree that consultant is effective-necessary. Only
responses of the ERP implementers (n=177) are used in the analysis
of consultant necessity because so few responses were received from
non-ERP implementers. No significant differences were observed
between the two groups.
Table 3. Correlation between Consultant Effectiveness and Necessity
by Phase
Effectiveness
Plan Reeng Design Config
Plan .542 *
Reeng .591 *
Design .580 *
Necessity Config .678 *
Test
Install
Shake
Oper
Effectiveness
Test Install Shake Oper
Plan
Reeng
Design
Necessity Config
Test .506 *
Install .566 *
Shake .584 *
Oper .639 *
* Significant at p<.01
Table 4. Differences in Mean Ratings for Effectiveness in ERP
Implementation Phases and Subphases
Mean Plan Reeng Design Config
Mean 4.83 4.46 4.88 5.21
Plan 4.83 --
Reeng 4.46 0.37 --
Design 4.88 -0.05 -.42 * --
Config 5.21 -0.38 -.75 *** -.33 ** --
Test 4.67 0.16 -0.21 0.21 .54 ***
Install 4.82 0.01 -0.36 0.06 .39 *
Shake 4.41 .42 ** 0.05 .47 ** .80 ***
Oper 3.95 .88 *** .51 ** 93 *** 1.26 ***
Overall 4.83 -- -0.37 0.05 .38 ***
Test Install Shake Oper Overall
Mean 4.67 4.82 4.41 3.95 4.83
Plan
Reeng
Design
Config
Test --
Install -0.15 --
Shake 0.26 .41 *** --
Oper .72 *** .87 *** .46 *** --
Overall -0.16 0.01 -.42 *** -.88 *** --
* Significant at p<.05
** Significant at p<.01
*** Significant at p<.001
Note: Differences are calculated as the mean rating for the phase on
the top of the chart minus the mean rating for the phase on the left
side of the chart. A positive difference indicates that the mean for
the phase on the top is higher than the mean for the phase on the
left side. A negative difference indicates that the mean for the
phase on the top is lower than the mean for the phase on the left
side.
Table 5. Differences in Mean Ratings for Necessity in ERP
Implementation Phases and Subphases
Mean Plan Reeng Design Config Test
Mean 5.23 4.74 5.35 5.52 4.66
Plan 5.23 --
Reeng 4.74 .49 ** --
Design 5.35 -0.12 -.61 *** --
Config 5.52 -0.29 -.77 *** -0.17 --
Test 4.66 .57 ** 0.08 .69 *** .86 *** --
Install 4.87 0.36 -0.13 .48 ** .65 *** -0.21
Shake 4.44 .79 *** 0.3 .91 *** 1.08 *** 0.22
Oper 3.6 1.63 *** 1.14 *** 1.75 *** 1.92 *** 1.06 ***
Overall 5.2 0.03 -.46 ** 0.15 .32 ** -54 ***
Install Shake Oper Overall
Mean 4.87 4.44 3.6 5.2
Plan
Reeng
Design
Config
Test
Install --
Shake .43 ** --
Oper 1.27 *** .84 *** --
Overall -0.33 -.76 *** -1.60 *** --
** Significant at p<.01
*** Significant at p<.001
Note: Differences are calculated as the mean in the phase on the top
of the chart minus the mean in the phase on the left side of the
chart. A positive difference indicates that the mean of the phase on
the top is higher than the mean of the phase on the left side. A
negative difference indicates that the mean of the phase on the top
is lower than the mean of the phase on the left side.
Table 6. Differences in Mean Ratings for Effectiveness and Necessity
by Phase
Phase or Subphase Effectiveness Necessity Difference
in Means
Planning phase 4.83 5.23 -.40 **
Reengineering subphase 4.46 4.74 -0.28
Design subphase 4.88 5.35 -.47 ***
Configuration and 5.21 5.52 -.31 **
integration subphase
Testing subphase 4.67 4.66 0.01
Installation and 4.82 4.87 -0.05
conversion subphase
Shakedown phase 4.41 4.44 -0.03
Operation phase 3.95 3.6 .35 **
Overall 4.83 5.2 -.37 ***
** Significant at p<.01
***Significant at p<.001
Note: Differences are calculated as mean on effectiveness minus mean
on necessity. A positive difference indicates that the consultants
were rated as being more effective than necessary; a negative
difference indicates that consultants were rated as being more
necessary than effective.
Table 7. Means of Respondents' Ratings on Characteristics of ERP
Consultants
Characteristic Mean (n=177) Standard Rank
Deviation
Technical skills and
knowledge 5.59 1.403 1
Human interaction and
communication skills 4.81 1.345 6
Business context
skills 4.77 1.503 7
Consulting skills and 5.06 1.335 3
knowledge Objectivity 4.84 1.449 5
Experience with ERP 5.49 1.434 2
implementations 4.87 1.529 4
Commitment to quality
Ability to manage ERP
implementations 4.73 1.554 8
Note: Ratings were on a scale of 1 to 7 with 1 being low
contribution to effectiveness and 7 being high contribution to
effectiveness. Only the responses of the ERP implementers (n=177)
are used in the analysis of the characteristics because so few
responses were received from non-ERP implementers. No significant
differences were observed between the two groups.
Table 8. Explanation of Variables for Regression Equations
Variable Explanation
TECHNICAL Technical knowledge of the ERP consultant
COMM Human interaction and communication skills of the
ERP consultant
BUSINESS Business context skills of the ERP consultant
CONSULT Consulting skills and abilities of the ERP
consultant
OBJECT Objectivity of the ERP consultant
CONSULTERPEXPER ERP consultant's experience with IS and ERP
implementations
QUALITY ERP consultant's commitment to quality
MANAGE Management skills of the ERP consultant
Table 9. Regression Results for Contributions of Characteristics to
Effectiveness of ERP Consultants by Phase in which Consultants
are Rated Most Effective
Variable Predicted Sign Coefficient Estimate p-value
Intercept ? 2.921 0
TECHNICAL + 0.157 0.075
COMM + 0.111 0.245
BUSINESS + -0.153 0.07
CONSULT + 0.003 0.976
OBJECT + 0.156 0.054
CONSULTEXPER + 0.044 0.669
QUALITY + 0.109 0.244
MANAGE + 0.083 0.385
Adjusted [R.sup.2] = .192
F=6.081, p<.001
Table 10. Regression Results for Contributions of Characteristics to
Overall Effectiveness of ERP Consultants
Variable Predicted Sign Coefficient Estimate p-value
Intercept ? 1.784 0
TECHNICAL + 0.173 0.009
COMM + 0.025 0.721
BUSINESS + -0.071 0.258
CONSULT + 0.062 0.413
OBJECT + 0.033 0.577
CONSULTEXPER + -0.067 0.385
QUALITY + 0.149 0.031
MANAGE + 0.276 0
Adjusted [R.sup.2] = .394
F=15.289, p<.001
