Analysis of Contract-type and Acquisition Performance

Analysis of Contract-type and Acquisition Performance

1 Analysis of Contract-type and Acquisition Performance Have fixed-price contracts resulted in superior acquisition outcomes? 2 Testing conventional wisdom Some believe that more use of fixed-price contracting will reduce cost and schedule growth rates on MDAP contracts If this is true, we would expect fixed-price contracts in the past to have experienced systematically lower cost growth and schedule growth rates, other things being equal (ceteris paribus)

The hypothesized predictor is: an indicator variable (i.e. dummy variable) for fixed-price contracts But, we must control for effects of other significant predictors as well 3 The data for development contracts There were 433 contracts in the sample Break-down by contract-type: 74 CPIF 108 CPAF 100 CPFF 78 fixed-price 73 hybrid (mixture of CLIN contract types) DoD performance data for large MDAP contracts from January 1970 through December 2011

All data not adjusted for inflation (then-year) 4 The data for early production contracts There were 440 contracts in the sample Break-down by contract-type: 23 CPIF 16 CPAF 48 CPFF 293 fixed-price 60 hybrid (mixture of CLIN contract types) DoD performance data for large MDAP contracts from January 1970 through December 2011 All data not adjusted for inflation (then-year) 5

Data methodology Collected performance data by contract as recorded in DAMIR segregated by Development (433) Early production (440) Converted data into comparable metrics Percentage cost growth as difference between final reported PMs EAC and initial baseline (initial CBB) Percentage schedule growth as difference between final schedule and initial baseline schedule Percentage scope growth as difference between final reported CBB and initial baseline (initial CBB) Appropriate indicator (dummy) variables 6

Sample development contracts characteristics Sample cost growth characteristics Average cost growth: 75% Median cost growth: 33% Minimum cost growth: -61% Maximum cost growth: 1221% Only 6.7% of contracts had negative cost growth Sample schedule characteristics Average schedule growth: 32% Median schedule growth: 14% Minimum schedule growth: -49% Maximum schedule growth: 582% Only 5.8% of contracts had negative schedule growth Duration of contract Average: 6.7 years

Median: 6.2 years 7 Sample early production contracts characteristics Sample cost growth characteristics Average cost growth: 46% Median cost growth: 11% Minimum cost growth: -46% Maximum cost growth: 677% Only 21% of contracts had negative cost growth Sample schedule characteristics Average schedule growth: 30% Median schedule growth: 11% Minimum schedule growth: -34% Maximum schedule growth: 691%

Only 7% of contracts had negative schedule growth Duration of contract Average: 5.2 years Median: 4.6 years 8 Specific analysis methodology Regressed outputs on inputs or likely predictors of selected outputs Did regression diagnostics Did sample analysis using standard robust non-parametric sample tests Ran bootstrap simulations (1000) on all regressions to obtain Unbiased coefficient estimates Correct standard errors

Correct confidence intervals 9 Development regression Modeled total cost growth as a function of Scope growth Aircraft indicator UCA indicator Indicator for fixed-price contract-type IF more fixed-price contracting would have improved performance, expect the coefficient estimate to be negative and significant A constant 10

Early production regression Modeled total cost growth as a function of Scope growth Schedule growth Indicator for an Army contract Indicator for fixed-price contract-type IF more fixed-price contracting would have improved performance, expect the coefficient estimate to be negative and significant A constant 11 Results of analysis of cost growth in development phase Variable Percentage growth in work

content of contract Indicator for aircraft contract Indicator for UCA Indicator for fixed price contract-type Constant Coefficient (Standard error) Significant at 5% Level of Signicance? 1.025 (0.0200) 0.230 (0.0612)

0.0791 (0.0348) 0.0698 (0.0447) 0.0999 (0.0245) Yes Yes Yes No Yes Model p-value 0.0000*** Coefficient of determination

0.939 Adjusted coefficient of determination 0.939 12 Results of analysis of cost growth in early production phase Variable Percentage growth in work content of contract Schedule growth rate Indicator for Army Indicator for fixed price

contract-type Constant Coefficient (Standard error) Significant at 5% Level of Signicance? 1.07 (0.0304) 0.0949 (0.0327) 0.124 (0.0458) 0.0340 (0.0313)

0.0270 (0.0256) Yes Yes Yes No No Model p-value 0.0000*** Coefficient of determination 0.922 Adjusted coefficient of

determination 0.922 13 Results of non-parametric tests of development contract sample No significant difference between fixedprice and cost-reimbursable contracts with respect to total cost growth No significant difference between fixedprice and cost-reimbursable contracts with respect to schedule growth Level of significance used in analysis: 5% 14 Results of non-parametric tests of early production contract sample

No significant difference between fixedprice and cost-reimbursable contracts with respect to total cost growth No significant difference between fixedprice and cost-reimbursable contracts with respect to schedule growth Level of significance used in analysis: 5% 15 Results of analysis of development regression Regression explains 94% of variation in the data Other things being equal A 1 percentage point increase in scope growth predicts a 1.025 percentage point increase in total cost growth Aircraft contracts generally experience 23% higher cost growth than other commodity contracts Contracts with a UCA generally experience 8% higher cost growth than contracts not experiencing a UCA

There was no significant difference noted between the cost performance of fixed-price contracts and costreimbursable contracts All hypotheses were tested at the 5% level of significance 16 Results of analysis of early production regression Regression explains 92% of variation in the data Other things being equal A 1 percentage point increase in scope growth predicts a 1.07 percentage point increase in total cost growth A 1 percentage point increase in schedule growth predicts a 0.095 percentage point increase in total cost growth Army contracts generally experience 12% higher cost growth than the other two services There was no significant difference noted between the cost performance of fixed-price contracts and cost-reimbursable

contracts All hypotheses were tested at the 5% level of significance 17 Conclusions In general, managers have correctly selected contract-type for large development contracts for MDAP programs In general, managers have correctly selected contract-type for large early production contracts for MDAP programs 18 Backups

19 Results of diagnostics run on OLS regression of development contracts sample Test of normality of residuals Smirnov- Kolmogorov test: Residuals are nonnormal Shapiro- Wilk test: Residuals are non-normal Cook- Weisberg test for heteroskedasticity: Heteroskedasticity is a problem Variance inflation factor (VIF) test: Multicolinearity is not a problem Ramsey RESET omitted variable test: Model has no omitted variables Linktest: model is correctly specified 20

Results of diagnostics run on OLS regression of early production contracts sample Test of normality of residuals Smirnov- Kolmogorov test: Residuals are nonnormal Shapiro- Wilk test: Residuals are non-normal Cook- Weisberg test for heteroskedasticity: Heteroskedasticity is a problem Variance inflation factor (VIF) test: Multicolinearity is not a problem Ramsey RESET omitted variable test: Model has no omitted variables Linktest: model is correctly specified 21

Non-parametric tests used on samples Wilcoxon rank-sum test Kolmogorov- Smirnov test Median test

Recently Viewed Presentations

  • Conventions of Psychological Reporting

    Conventions of Psychological Reporting

    experiment (e.g. Kim's Game, ruler drop test) and correlation (e.g. height and shoe size). When referencing a website or a journal article, the principles are the same as for a book, you just have to put in some other information..
  • Enfoque: jueves, el 3 de diciembre, 2008 ("dos mil ocho")

    Enfoque: jueves, el 3 de diciembre, 2008 ("dos mil ocho")

    More than 2 wrong = 1) Practice Homework sheets, Quia and @Hometutor 2) Extra Help 7th period Mon & Tues and after school on Thursday. Grammar Quiz (-ar verbs) on Friday! La Tarea Finish Worksheet: "Did you get it?" #...
  • Welcome to English IH

    Welcome to English IH

    SB, TKAM, grammarbook, IR book. Reading Quiz! Ch9. IR 10 min. Grammar pg 50 51. Review introduction chart/annotate EWP for key phrases. In groups of 2-3, complete the Explanatory Writing Prompt on pg 236. Work in groups of 2-3 write...
  • Diapositiva 1 - Es de libro

    Diapositiva 1 - Es de libro

    Tras esto, prosiguieron investigaciones en varios miles de voluntarios adultos, así como en niños de 5 años, niños de 2 a 16 años, bebés de menos de 11 meses, etc, tanto en varios países sudamericanos como africanos, con resultados mínimos...
  • 8TH GRADE SCIENCE LEAF PROJECT - Auburn Middle School

    8TH GRADE SCIENCE LEAF PROJECT - Auburn Middle School

    LEAF PRESENTATION HOLLY SASSAFRAS REDBUD BOXELDER Chestnut, Horse SWEETGUM Ash Birch CATALPA WILLOW, WEEPING BUCKEYE, OHIO GINKGO ELM, AMERICAN Elm, Slippery Hawthorn, Green Maple, Red MAPLE, SUGAR BALDCYPRESS HACKBERRY MAPLE, SILVER MIMOSA HICKORY, SHAGBARK MULBERRY, RED COTTONWOOD LOCUST, HONEY PINE,...
  • Fossil Fuels - Western Illinois University

    Fossil Fuels - Western Illinois University

    Fossil Fuels Exponential growth is a very POOR model for the consumption of non-renewable resources. M.K. Hubbert in 1957 made a startling prediction about the production of oil in the US. "Oil Production in the US will peak in 1967."...
  • Generating Energy Solutions - OAPT Physics

    Generating Energy Solutions - OAPT Physics

    "In science, 'fact' can only mean 'confirmed to such a degree that it would be perverse to withhold provisional assent.' I suppose that apples might start to rise tomorrow, but the possibility does not merit equal time in physics classrooms."
  • Regno delle piante - WordPress.com

    Regno delle piante - WordPress.com

    Richiedono acqua per la riproduzione (i gameti maschili nuotano sino alle cellule uovo) Hanno generazione gametofitica dominante Ciclo delle briofite Principali gruppi Le piante senza semi, comunemente conosciute come pteridofite, sono suddivise nelle 4 divisioni: Psilophyta (psilofite) Lycopodiophyta (licopodi) Equisetophyta...