扫一扫关注微信
国家哲学社会科学文献中心版权所有
EAC evaluation methods: do they still work? - Research - Estimate at Completion model for evaluating defense contracts
David ChristensenSeveral methods are described to evaluate the accuracy of the estimated final cost of a defense acquisition contract, termed the "Estimate at Completion" (EAC). The methods are based on the Department of Defense experience that cost variances tend to worsen after the 20 percent completion point, and were validated on contracts completed in the 1970s and 1980s. This study tests the validity of two EAC evaluation methods on a sample of defense acquisition contracts completed in the 1990s. Results show the mean cost performance on contracts completed in the 1990s did not worsen significantly from the 20 percent completion point, thus challenging the basic premise of the evaluation methods. We speculate that acquisition reform initiatives, energized by the 1991 cancellation of the Navy's A-12 program, have improved defense cost performance.
**********
When Secretary of Defense Richard Cheney cancelled the A-12 program in January 1991, he complained in a press conference that no one could tell him the program's final cost (Morrison, 1991). (l) In fact, a Navy investigation led by Chester P. Beach (1990) revealed that there were many estimates of the program's final cost, and some were more reasonable than others.
Citing Department of Defense (DoD) experience with over 400 programs completed since 1977, Beach concluded that the Estimate at Completion (EAC) supported by the Navy's program manager was too low. Beach (1990) suggested that similar problems were likely on other defense programs because of an "abiding cultural problem" found in all the military services; namely, program managers do not always tolerate an accurate EAC, especially when it may jeopardize the funding of the program (Christensen, 1993, 1996; Fox, 1974; Mayer 1991).
The "DoD experience" referenced by Beach pertains to the fact that defense cost variances tend to worsen from the 20 percent completion point. Based on this fact, three "rules of thumb" have been useful for evaluating the accuracy of the EAC (Christensen, 1999) and encouraging cost realism:
(1.) The final cost variance (in dollars or as a percentage) will be worse than the cost variance at the 20 percent completion point.
2. The cumulative cost performance index (CPI) will not change by more than 0.10 from its value at the 20 percent completion point, and in most cases it only worsens.
3. The EAG computed using the cumulative CPI is a reasonable lower bound to the final cost of a defense contract.
Each rule has been validated by empirical analysis of completed defense acquisition contracts (Christensen, 1993, 1996; Christensen & Heise, 1993; Christensen & Payne, 1992). When an EAC for a major defense acquisition contract violates any of these rules, the accuracy of the EAC should be questioned. (2) However, a recent opinion survey of 10 major DoD contractors (Coopers & Lybrand, 1997) indicates that current acquisition reform initiatives are improving the cost performance of defense acquisition contracts. (3) This suggests that the rules of thumb validated on contracts completed in the 1970s and 1980s may no longer be valid. In this study, we tested the validity of the first two evaluation rules on contracts completed in the 1990s. The last rule will be evaluated in another study.
METHODOLOGY
HYPOTHESES
Null and alternative hypotheses for the two evaluation rules are listed in Figure 1. Hypotheses 1 and 2 pertain to the first evaluation rule. Hypotheses 3 and 4 pertain to the second evaluation rule. If the null hypotheses are rejected, the evaluation rules are confirmed.
The first EAC evaluation rule pertains to cost variances, expressed in dollars or percentages. A cost variance (CV) is defined as the difference between the budgeted cost of work performed (earned value) and the actual cost of work performed. (4) A percentage cost variance is the dollar cost variance divided by earned value.
Cost Variance (CV$) = Earned Value - Actual Cost (1)
Cost Variance (CV%) = (CV$ / Earned Value) x 100 (2)
The second evaluation rule pertains to the cumulative Cost Performance Index (CPI). The CPI is earned value divided by actual cost, and can be computed based on a single month, an average of several months, or all months to date (cumulative). When the cumulative CPI is less than one, an adverse cost variance (i.e., a cost overrun) is indicated.
CPI = Earned Value / Actual Cost (3)
We computed the cumulative CV and CPI at the 20 percent completion point (CV 20 and CPI 20) and at the end of the contract (CV final and CPI final) for each contract in our sample. Percent complete was defined as cumulative earned value divided by the total budget for the planned work on the contract, termed the Budget at Completion (BAC).
Percent Complete = Cumulative Earned Value / BAC (4)
The 20 percent completion point was chosen because earlier performance data are often either not available or considered unreliable. For example, it has sometimes taken over one year for a contractor to be found compliant to the earned value management systems (EVMS) criteria, or to establish a performance measurement baseline (Fleming, 1992). Until each is accomplished, performance measurement data are of dubious value.
Hypothesis 3 pertains to the stability of the cumulative CPI. Based on an analysis of 155 defense acquisition contracts, Christensen and Heise (1993) reported that the range of the cumulative CPI from the 20 percent completion point to contract completion was less than 0.20 for every contract. This result is usually interpreted to mean that the cumulative CPI does not change by more than plus or minus 0.10 from its value at the 20 percent completion point, and is used to evaluate the reasonableness of projected cost efficiencies on future work. (For a description of the technique see Christensen, 1999.)
Hypothesis 4 pertains to the remaining portion of the CPI rule: in most cases the cumulative CPI worsens. If the cumulative CPI at the 20 percent completion point is greater than the final CPI, then the cost variance worsened. The cumulative CPI is often used to determine a reasonable lower bound to the EAC. Since the A-12 cancellation, DoD policy requires that if the EAC derived from the cumulative CPI is larger than the EAC supported by the contractor, then the contractor's EAC needs to be explained.
THE DATABASE
We collected contract performance data from the Defense Acquisition Executive Summary (DAES) database, maintained by the Office of the Under Secretary of Defense for Acquisition, Technology, and Logistics (OUSD [AT&L]). The database contains cost and schedule performance data on more than 500 completed and ongoing contracts from as early as 1971. A contractor prepares a monthly cost performance report (CPR) that summarizes cost and schedule performance then sends it to the government program office for analysis. The program office summarizes the CPR data into a DAES report that is sent to OUSD (AT&L) for analysis and storage in the DABS database.
The reliability of the data is controlled by a DoD requirement for contractors to comply with Earned Value Management Systems (EVMS) criteria (DoD, 1997). The criteria are internal controls intended to encourage adequate planning and control. When the contractor's management control systems are compliant to the EVMS criteria, the government assumes that the performance data are reasonably reliable. In general, contractors that submit the CPR are required to be EVMS criteria-compliant.
THE SAMPLE
The sample consisted of 240 contracts identified with the necessary data to test the four hypotheses. The necessary data included values for cumulative earned value, cumulative actual cost, and BAC at the 20 percent completion point and after the 80 percent completion point. For the 20 percent completion point, any contract with pefformance data within 17.5 and 22.5 percent was selected. Because many contractors discontinue CPR reporting after the 80 percent completion point, the final cost is defined here as the cumulative actual cost from the last available CPR for each contract, and included if the percent complete exceeded 80 percent and could be matched with the same contract at the 20 percent completion point.
HYPOTHESIS TESTING
We tested each hypothesis on the entire sample and on various categories within the sample using the paired-t test and the non-parametric Mann-Whitney test. Previous research shows that cost variances are not normally distributed (Christensen, Conley, & Kankey, 2000). Although the paired-t test is generally robust to modest violations of normality, this study includes the non-parametric test for completeness (Conover, 1980). All testing was conducted at an alpha of 0.05.
December 31, 1991 was chosen, about one year after the A-12 cancellation was announced, as a cut-off date for distinguishing pre-A-12 contracts from post-A- 12 contracts. Virtually any cut-off date is arbitrary because the ability of the A-12 cancellation to influence the "abiding cultural problem" described by Beach (1990) could take years. Accordingly, we divided our sample into three time periods (pre-A-12, contracts finished before December 31, 1991; transitional, contracts started before but finished after December 31 1991; and post-A-12, contracts started after December 31, 1991). The hypotheses were tested on each period.
RESULTS
HYPOTHESES 1, 2, AND 4
Tables 1-3 show the mean cost performance (expressed in dollars, percentages, and the cumulative CPI) at the 20 percent and at the 100 percent completion points for pre--A-12, transitional, and post--A-12 contracts, as welt as for various subcategories of these contracts. (5) These tables also show the results of testing hypotheses 1, 2, and 4. (The results of testing hypothesis 3 appear in Table 4). When the null hypothesis was rejected (indicated with an asterisk), the mean cost performance worsened significantly from the 20 percent completion point.
Pre--A-12 Contracts. For the 147 pre--A-12 contracts, the two FAG evaluation rules were confirmed (null hypotheses 1, 2, and 4 were rejected). The mean final cost variance (in dollars and as a percentage) and the mean final CPI were each significantly worse than their mean values at the 20 percent completion point. For example, Table 1 shows that the mean final cost variance (CV final) of the pre--A-12 contracts was -$12.2 million. The mean cost variance at the 20 percent completion point (CV 20) was -$0.8 million. The mean difference (CV final - CV 20) of -$11.4 million was statistically significant (one-tailed p < 0.000). (6)
Tables 2 and 3 are interpreted the same way. For example, Table 3 shows that the mean final cumulative CPI (CPI final) was 0.951, and the mean cumulative CPI at the 20 percent completion point (CPI 20) was 0.996. The mean difference (CPI final - CPI 20) of -0.045 was highly significant (one-tailed p < 0.000). This result was the same for nearly all subcategories of pre-A-12 contracts. The only exception was pre--A-12 Navy contracts, where the mean decrease in the cumulative CPI was not significant (one-tailed p = 0.054).
Post--A-12 Contracts. For the 52 post--A-12 contracts, the two EAC evaluation rules were not confirmed (null Hypotheses 1, 2, and 4 were not rejected). The mean cost performance (measured as a percentage or as the cumulative CPI) of post--A-12 contracts did not worsen significantly from the 20 percent completion point. For example, the mean CPI of the 52 post--A-12 contracts at the 20 percent completion point was 0.977, and the mean final CPI was 0.964. The decrease of 0.013 was not significant (one-tailed p = 0.217). As shown in Tables 2 and 3, the only exceptions to this result were post--A-12 development and Army contracts. The mean cost performance of all other post--A-12 contracts did not worsen significantly. (7)
Based on these results, testing was done on whether the mean final cost performance on post-A-12 contracts was significantly better than the mean final cost performance on pre-A-12 contracts. Results showed that the mean improvement in cost performance of the post-A-12 contracts was not significant, regardless of how cost performance was defined, and regardless of the contract category. For example, the mean final cost variance on the 147 pre-A-12 contracts was -6.5 percent, and the mean final cost variance on the 52 post-A-12 contracts was -4.4%. The mean difference of 2.1 percent was not statistically significant (one-tailed p = 0.214).
HYPOTHESIS 3
Table 4 summarizes the results of testing Hypothesis 3, pertaining to the stability of the cumulative CPI. With only a few exceptions, the stability of the cumulative CPI was confirmed (null Hypothesis 3 was rejected). The mean cumulative CPI did not change by more than 0.10 from its mean value at the 20 percent completion point (one-tailed p < 0.000). This was true, regardless of whether the contracts were pre--A-12 or post--A-12. For pre--A-12 and post--A-12 contracts, the mean absolute deviations in the cumulative CPI were 0.086 and 0.067, respectively, and each was significantly less than 0.10 (one-tailed p < 0.000).
There were some pre--A-12 subcategories (i.e., development, Army), where the mean absolute deviation of the cumulative CPI was not significantly less than 0.10. Although this result may appear to invalidate results reported by Christensen and Heise (1993), we used a more rigorous definition for CPI stability. Christensen and Heise (1993) defined the cumulative CPI to be stable when its range was within 0.20. This study defined the cumulative CPI to be stable when its mean absolute deviation was less than 0.10, an arguably more stringent criterion. If we increased the mean absolute deviation to 0.20, our results would be entirely consistent with the results reported by Christensen and Heise (1993).
We also tested whether the mean absolute deviation of the cumulative final CPI on the post--A-12 contracts was significantly smaller than the mean absolute deviation on pre--A-12 contracts. Results showed that the decrease was not statistically significant. For the 147 pre--A-12 contracts, the mean absolute deviation was 0.085. For the 52 post--A-12 contracts, the deviation was 0.067. The mean difference of 0.018 is not significant (one-tailed p = 0.153). This was true of all subcategories of pre--A-12 and post--A-12 contracts.
CONCLUSION
The A--12 cancellation was a bitter pill to swallow, but it appears to have done some good. The mean cost performance of pre--A-12 (contracts that finished before December 31, 1991) worsened significantly from the 20 percent completion point, regardless of the contract phase, contract type, or the military service managing the contract. For post--A-12 contracts, the mean cost performance did not worsen significantly from the 20 percent completion point. However, the mean final cost performance on post--A-12 contracts was not significantly better than the mean final cost performance on pre--A-12 contracts. Taken together, these results suggest that while cost performance is improving, it would be unwise to stop using the EAC evaluation rules to encourage more realistic estimates. The A-12 cancellation highlighted the use of these rules in arriving at a more realistic EAC (Beach, 1990).
Our results suggest that the A-12 cancellation was a "significant emotional event" that may have helped to correct the abiding cultural problem described by Beach (1990). Research shows that numerous prior initiatives to reform defense acquisition have not reduced defense cost overruns (Drezner, Jarvaise, Hess, Hough, & Norton, 1993). Accordingly, Green, King, and Rappaport (2000) conclude that when viewed through the "lens of past experience" today's reform initiatives are doomed to failure. We interpret our results as counter-evidence to their pessimistic projection. Things are getting better. (8)
Another consideration is the unexpected poor cost performance of the contracts that started before and finished after the A-12 cancellation (transitional contracts). Cost performance for these contracts was much worse than the pre-A-12 and post-A-12 contracts. During this transition period, the DoD and contractors were changing systems (and probably cultures) to take advantage of acquisition reform initiatives. It appears that transitional contracts were not benefited by acquisition reform, and they may have received adverse effects from proposing and contracting under preacquisition reform rules while all or part of actual contract performance occurred in the acquisition reform period. That may be the price of acquisition reform.
Figure 1
Null and Alternative Hypotheses for the Two Evaluation Rules
H1o: CV$ final -- CV$ 20 [blank.sup.3] 0
H1a: CV$ final -- CV$ 20 < 0
H2o: CV% final -- CV% 20 [blank.sup.3] 0
H2a: CV% final -- CV% 20 < 0
H3o: \CPI final -- CPI 20\
[blank.sup.3]0.10
H3a: \CPI final -- CPI 20\ < 0.10
H4o: CPI final -- CPI 20 [blank.sup.3] 0
H4a: CPI final -- CPI 20 < 0
H1o: The cost variance (CVS) did not
worsen
H1a: The CV$ worsened
H2o: The percent cost variance (CV%)
did not worsen
H2a: The CV% worsened
H3o: The cost performance index (CPI)
changed by more than 0.10
H3a: The CPI did not change by more
than 0.10
H4o: The final CPI did not worsen
H4a: The CPI worsened
Table 1
Mean Cost Variances on Defense Acquisition Contracts at 20 Percent
Complete (CV 20) and at Completion (CV Final)
CV 20 CV Final
($ Millions) ($ Millions)
Cagetory N Mean Std Dev Mean Std Dev
Pre-A-12 147 -0.8 4.0 -12.2 45.4
Development 44 -1.4 4.5 -11.3 18.0
Production 49 -0.3 3.0 -9.9 26.3
Air Force 53 -0.3 3.7 -8.1 18.6
Army 45 -0.7 2.5 -12.7 21.8
Navy 49 -1.4 5.2 -16.2 73.7
Cost-reimbursable 53 -1.8 4.8 -22.5 69.2
Fixed-price 90 -0.2 3.3 -6.3 21.9
Trans-A-12 41 -3.4 9.8 -120.8 464.6
Post-A-12 52 -3.0 6.6 -15.8 39.3
Development 24 -0.8 2.2 -11.7 17.3
Production 26 -5.3 8.5 -20.9 53.0
Air Force 12 -0.5 1.6 -7.5 19.2
Army 14 -2.2 4.0 -21.5 39.6
Navy 26 -4.6 8.5 -16.6 46.0
Cost-reimbursable 34 -1.5 3.1 -12.9 28.9
Fixed-price 15 -7.2 10.3 -25.1 59.0
CV Final -- Mann-Whitney
CV 20 Test
Cagetory Mean Std Dev Asyp Z 1-tall p
Pre-A-12 -11.4 43.6 -5.72 0.000 *
Development -9.9 15.2 -4.65 0.000 *
Production -9.6 25.6 -3.23 0.001 *
Air Force -7.8 16.8 -3.39 0.000 *
Army -12.0 21.6 -3.98 0.000 *
Navy -14.8 71.0 -2.46 0.007 *
Cost-reimbursable -20.7 67.1 -4.47 0.000 *
Fixed-price -6.2 20.1 -3.77 0.000 *
Trans-A-12 -117.4 462.2 -4.21 0.000 *
Post-A-12 -12.8 36.3 -2.42 0.008 *
Development -10.9 16.8 -3.36 0.000 *
Production -15.6 49.1 -0.83 0.205
Air Force -7.1 19.0 -0.39 0.348
Army -19.3 39.6 -2.35 0.009 *
Navy -12.0 41.0 -1.21 0.114 *
Cost-reimbursable -11.4 28.8 -2.51 0.006 *
Fixed-price -17.9 52.8 -0.97 0.167
H1o: CV final -- CV [20.sup.3] 0 (The mean CV did not worsen from the
20% completion point)
H1a: CV final -- CV 20 < 0 (The mean CV worsened from the 20% completion
point)
* Reject Ho at alpha = .05
Table 2
Mean Cost Variance Percentages on Defense Acquisition Contracts at 20
Percent Complete (CV% 20) and at Completion (CV% Final)
CV% 20 CV% Final
Category N Mean Std Dev Mean Std Dev
Pre--A-12 147 -1.7 12.0 -6.5 13.2
Development 44 -2.0 13.8 -6.2 10.4
Production 49 -1.9 12.5 -8.1 14.4
Air Force 53 -2.0 14.5 -4.7 11.4
Army 45 -1.3 11.9 -9.8 16.5
Navy 49 -1.7 8.8 -5.5 11.1
Cost-reimbursable 53 -2.3 11.5 -8.7 15.5
Fixed-price 90 -1.2 12.4 -5.5 11.6
Trans--A-12 41 -2.0 10.7 -13.9 17.0
Post--A-12 52 -2.6 6.0 -4.4 9.1
Development 24 -0.9 5.4 -5.8 7.7
Production 26 -4.3 6.3 -3.9 9.8
Air Force 12 0.3 4.5 -0.3 7.1
Army 14 -2.3 5.0 -7.9 10.8
Navy 26 -4.2 6.8 -4.4 8.5
Cost-reimbursable 34 -1.5 4.9 -4.9 9.9
Fixed-price 15 -5.8 7.0 -4.0 8.0
CV% Final - Mann-Whitney
CV% 20 Test
Category Mean Std Dev Asyp Z 1-tail p
Pre--A-12 -4.8 13.5 -4.90 0.000 *
Development -4.2 14.4 -3.16 0.001 *
Production -6.2 12.6 -3.18 0.001 *
Air Force -2.7 12.6 -2.98 0.001 *
Army -8.5 15.9 -3.60 0.000 *
Navy -3.9 11.6 -1.70 0.044 *
Cost-reimbursable -6.4 14.1 -3.47 0.000 *
Fixed-price -4.3 13.2 -3.55 0.000 *
Trans--A-12 -11.9 15.9 -4.01 0.000 *
Post--A-12 -1.7 9.4 -0.57 0.286
Development -4.9 7.7 -3.36 0.000 *
Production 0.3 10.0 -0.83 0.795
Air Force -0.5 7.4 0.00 0.500
Army -5.6 11.1 -1.66 0.048 *
Navy -0.3 8.9 -0.65 0.258
Cost-reimbursable -3.4 9.7 -1.58 0.057
Fixed-price 1.8 8.2 -1.19 0.884
H2o: CV% final -- CV% 20 [blank.sup.3] 0 (The mean CV% did not worsen
from the 20% completion point)
H2a: CV% final -- CV$ 20 < 0 (The mean CV% worsened from the 20%
completion point)
* Reject Ho at alpha = .05
Table 3
Mean Cumulative Cost Performance Indices (CPI) on Defense Acquisition
Contracts at 20 Percent Complete, (CPI 20) and at Completion (CPI Final)
Cumulatie Cumulative
CPI 20 CPI Final
Category N Mean Std Dev Mean Std Dev
Pre-A-12 147 0.996 0.109 0.951 0.102
Development 44 0.996 0.124 0.949 0.085
Production 49 0.995 0.109 0.939 0.109
Air Force 53 0.996 0.116 0.965 0.086
Army 45 1.001 0.123 0.928 0.121
Navy 49 0.991 0.088 0.957 0.096
Cost-reimbursable 53 0.989 0.107 0.935 0.108
Fixed-price 90 1.001 0.112 0.958 0.096
Trans--A-12 41 0.990 0.095 0.896 0.125
Post--A-12 52 0.977 0.056 0.964 0.077
Development 24 0.994 0.053 0.949 0.063
Production 26 0.962 0.056 0.969 0.079
Air Force 12 1.005 0.047 1.002 0.074
Army 14 0.980 0.050 0.934 0.081
Navy 26 0.964 0.060 0.963 0.072
Cost-reimbursable 34 0.988 0.049 0.961 0.083
Fixed-price 15 0.949 0.058 0.967 0.072
CPI Final -- Mann-Whitney
CPI 20 Test
Category Mean Std Dev Asyp Z 1-tail p
Pre-A-12 -0.045 0.112 -4.96 0.000
Development -0.078 0.113 -3.31 0.000
Production -0.055 0.111 -3.24 0.001
Air Force -0.032 0.103 -3.05 0.001
Army -0.071 0.124 -3.66 0.000
Navy -0.034 0.107 -1.61 0.054
Cost-reimbursable -0.054 0.106 -3.55 0.000
Fixed-price -0.066 0.115 -3.62 0.000
Trans--A-12 -0.094 0.232 -3.99 0.000
Post--A-12 -0.013 0.081 -0.61 0.271
Development -0.044 0.067 -2.57 0.005
Production 0.007 0.082 -1.23 0.891
Air Force -0.002 0.078 0.00 0.500
Army -0.045 0.087 -1.66 0.048
Navy 0.000 0.079 -0.60 0.725
Cost-reimbursable -0.27 0.082 -1.53 0.063
Fixed-price 0.018 0.074 -1.14 0.872
Mann-Whitn
ey
Test
Category
Pre-A-12 *
Development *
Production *
Air Force *
Army *
Navy
*
Cost-reimbursable
*
Fixed-price
*
Trans--A-12
Post--A-12
Development *
Production
Air Force
Army *
Navy
Cost-reimbursable
Fixed-price
H40: CPI final -- CPI 20 (3) 0 (The mean cumulative CPI did not worsen
from the 20% completion point)
H4a: CPI final -- CPI 20 < 0 (The mean cumulative CPI worsened from the
20% completion point)
* Reject Ho at alpha = .05
Table 4
Cumulative Cost Performance Index Stability on Defense Acquisition
Contracts
|CPI FINAL Mann-Whitney
- CPI 20| Test
Category N Mean Std Dev Asymp Z 1-tailed p
Pre--A-12 147 0.085 0.085 -3.405 0.000
Development 44 0.085 0.087 -1.610 0.054
Production 49 0.086 0.089 -2.024 0.021
Air Force 53 0.075 0.077 -2.793 0.003
Army 45 0.105 0.096 -0.762 0.223
Navy 49 0.078 0.081 -2.333 0.010
Cost-reimbursable 53 0.084 0.084 -2.421 0.008
Fixed-price 90 0.087 0.086 -2.428 0.008
Trans--A-12 41 0.117 0.101 -0.032 0.487
Post--A-12 52 0.067 0.053 -4.207 0.000
Development 24 0.060 0.052 -3.000 0.001
Production 26 0.062 0.053 -3.111 0.001
Airforce 12 0.059 0.048 -2.118 0.017
Army 14 0.072 0.064 -1.789 0.037
Navy 26 0.059 0.050 -3.111 0.001
Cost-reimbursable 34 0.064 0.058 -3.171 0.001
Fixed-price 15 0.060 0.046 -2.329 0.010
Category
Pre--A-12 *
Development
Production *
Air Force *
Army
Navy *
Cost-reimbursable *
Fixed-price *
Trans--A-12
Post--A-12 *
Development *
Production *
Airforce *
Army *
Navy *
Cost-reimbursable *
Fixed-price *
H3o: | CPI final - CPI 20 | [[blank].sup.3] 0.10 (The mean absolute cum
CPI changed by more than .10)
H3a: | CPI final - CPI 20 | < 0.10 (The mean absolute cum CPI did not
change by more than .10)
* Reject Ho at alpha = .05
ENDNOTES
(1.) The A-12 program was a fixed-price incentive contract in the full-scale development phase when it was cancelled.
(2.) These rules of thumb are statistical statements about the mean cost performance of defense contracts. While the cost performance of an individual contract may differ from these rules, it would be a statistical outlier in the sense that its cost performance would be more than two standard deviations away from the mean.
(3.) See Coopers & Lybrand (1997) report for a detailed listing of the acquisition reform initiatives that are intended to improve cost performance of defense acquisition contracts.
(4.) The budgeted cost of work performed is also known as BCWP. The actual cost of work performed is also known as ACWP.
(5.) Some of the subcategories do not add to the total because some contracts in the DAES database are not completely described.
(6.) All p values shown in the Tables are based on the Mann-Whitney test. The results of the paired-test were consistent with the results of the MannWhitney test.
(7.) We do not know why the mean cost performance of these subcategories of post--A- 12 contracts was different than the rest; however, the small sample sizes of these categories may be a contributing factor. Accordingly, we place more confidence in the overall result, than the analysis of the smaller subcategories. In addition, the analysis of cost variances in dollars (Table 1) shows slightly different results on post--A-12 contracts. We suspect the differences are due to averaging dollars (Table 1) instead of ratios of dollars (Tables 2 and 3). Cost variances in dollars (CV$) could be biased by inflation and differences in contract sizes.
(8.) We recognize that improved cost performance is not the only indicator of success in defense acquisition reform.
REFERENCES
Beach, C. P. (1990). A-12 administrative inquiry. Washington, DC: Department of the Navy. Available at http://www.suu.edu/faculty/christensend/ev-bib.html.
Christensen, D. S. (1993, March). Cost overrun optimism: Fact or fiction? Acquisition Review Quarterly 1, 25-38.
Christensen, D. 5. (1996, Spring). Project advocacy and the estimate at completion problem. The Journal of Cost Analysis and Management, 35-60.
Christensen, D. S. (1999, Summer). Using the earned value cost management report to evaluate the contractor's estimate at completion. Acquisition Review Quarterly 19, 283-296.
Christensen, D. S. & Payne, K. (1992, April). Cost performance index stability: Fact or fiction? Journal of Parametrics 10, 27-40.
Christensen, D. S. & Heise, S. R. (1993). Cost performance index stability. National Contract Management Journal 25, 7-15.
Christensen, D. S., Conley, R. J., IV, & Kankey, R. D. (2000, Winter). Some empirical evidence on the non-normality of cost variances on defense contracts. The Journal of Cost Analysis and Management, 3-15.
Conover, W. J. (1980). Practical non-parametric statistics (2nd ed.). New York: John Wiley & Sons, Inc.
Coopers & Lybrand L.L.P. (1997, October). Acquisition reform implementation -- An industry survey. Available at http://www.acq.osd.mil/ar/clreport.htm.
Department of Defense. (1997, October). Earned value management implementation guide. Washington, DC: Author.
Drezner, J., Jarvaise, J., Hess, R., Hough, P., & Norton, D. (1993). An analysis of weapon system cost growth. Santa Monica, CA: RAND.
Fleming, Q. W. (1992). Cost/schedule control systems criteria (Rev. ed.). Chicago: Probus Publishing Company.
Fox, J. R. (1974). Arming America: How the U.S. buys weapons. Boston: Harvard University Press.
Green, S. G., King, D. R., & Rappaport, N. J. (2000, Winter). Bringing acquisition reform into focus. The Journal of Cost Analysis and Management, 69-82.
Mayer, K. R. (1991). The political economy of defense contracting. New Haven, CT: Yale University Press.
Morrison, D. C. (1991, March). Deepsixing the A-12. Government Executive, 30-35.
David Christensen is a professor of accounting at Southern Utah University at Cedar City, Utah. After earning his Ph.D. at the University of Nebraska in 1987, he joined the faculty of the Air Force Institute of Technology, where he taught undergraduate and graduate courses in earned-value for over 10 years. Christensen is a CMA, CGFM, and CCEA and is active in the Society of Cost Estimating and Analysis, the American Accounting Association, and the Institute of Management Accounting.
(E-mail address: [email protected])
CarlTemplin is the dean of the College of Business and Technology and professor of management at Southern Utah University. After earning his Ph.D. at Arizona State University in 1988, he joined the faculty of the Air Force Institute of Technology, where he taught graduate courses in contracting management for seven years and served as associate dean and acting dean of the School of Logistics and Acquisition Management. Templin is Fellow of the National Contracting Management Association and is a member of the National Association of Purchasing Management.
(E-mail address: [email protected])
COPYRIGHT 2002 Defense Acquisition University Press
COPYRIGHT 2004 Gale Group
扫一扫关注微信