A suggestion for the improvement of performance risk assessment

Performance risk assessment has been more commonly known as past performance assessment. Only recently has the Air Force, at least, been referring to it as performance risk assessment.

The definitions in the left column of the sidebar on the next page are found in the Past Performance Evaluation Guide, U.S. Air Force, March 2003, Version 1.1. They are used in the evaluation of past performance information and in exact or near-exact form, have been found on the majority of proposal requests that I have seen used in Air Force programs. (The Office of the Secretary of Defense Guide to Collection and Use of Past Performance Information, Version 3, May 2003, contains very similar definitions.)

Language Issues

Only fairly recently have I become convinced of the potential benefits that past performance evaluations can give us in terms of choosing the right contractors for our work. Earlier, the Federal Acquisition Regulation requirement that past performance be at least as important as the most important non-cost/price factor didn't make sense to me. And while I am still not necessarily convinced that this stricture works in every situation, under certain conditions it makes great sense.

[ILLUSTRATION OMITTED]

In my view, the best use of a source selection is to find a contractor who is truly looking for a win-win scenario, a contractor with the best interests of the government and its customers in mind, who intends to give the government fair value while making a reasonable profit. This doesn't necessarily mean choosing the contractor with the lowest cost or price, or even the contractor with the best cost or price in relation to having a technically superior proposal--though these things are certainly factors to consider.

Language can be a very imprecise tool. The meaning of words, even the wording of supposedly iron-clad contracts, is often (some might say always) open to interpretation. My understanding of tort law is that it often follows what is known as "the reasonable man interpretation": If a reasonable person could interpret something in a particular way, then that is a valid interpretation. So if we have a contractor who isn't driven by what we (and by extension, the warfighters) want and/or need, but is driven instead by the desire to give us what he or she wants to give us, then the specific wording of the contract may not matter, if it could reasonably be interpreted to mean what the contractor understands it to mean, not what we understand.

Conversely, if we have a contractor who's driven by a desire to work with us to meet our requirements, the specific wording of the contract may not be what's important. We will get what we need anyway. And that's a good thing!

So choosing a contractor with a good attitude is very important. While we cannot evaluate attitude, we can evaluate actions that might indicate attitude, and one of the most effective ways is past performance.

One way of looking at a contractor response to a request for proposal (RFP) is to think of the proposal as a promise of what the contractor intends to do if awarded the contract. Our technical (or mission-capability, to be more correct) evaluation of the proposal is to determine whether this promise meets our requirements. Past performance evaluations answer the question, "Does this contractor (or contractor team) have a history of living up to its promises?"

Given these premises, the obvious question is this: What is wrong with the rating definitions as they stand? Don't they answer the promise-keeping question? Well, yes they do--but not necessarily in a complete manner.

The Need for Alternatives

Let us look at just one definition--that of "exceptional": "The contractor's performance meets contractual requirements and exceeds many (requirements) to the Government's benefit." That's certainly exceptional, and I have no problem with it as such. However, to it is added: "The contractual performance was accomplished with few minor problems for which corrective actions taken by the contractor were highly effective." I mostly like the second half of the sentence but not when coupled with the first half. And it is the whole gestalt of the first and second sentences together that defines exceptional.

So what do I feel is wrong with "The contractual performance was accomplished with few minor problems ..."? The lack of problems is not what defines, for me, an exceptional contractor. Some of the work we do really pushes the envelope in terms of performance, and to be brutally honest, we rarely budget the right amount of money or time to execute a lot of these programs correctly. If you're to encounter only a few minor problems, you have to be extremely lucky and have everything go your way, or you have a contractor who isn't pushing the envelope. I would rather have a contractor who runs up against problems (minor or not) and takes exceptional action to work through them, taking "highly effective corrective actions."

[ILLUSTRATION OMITTED]

But this situation would not be true for those types of efforts where we are not pushing the envelope or where we are asking for "standard" commercially available goods and services, particularly for things like service contracts. Here, we refer to items such as purchasing cars for a motor pool, or janitorial services for our buildings.

One Size Doesn't Fit All

I can foresee the need for an alternative standard, where the past performance rating definitions are appropriate to the needs of the effort. One set would be the existing definitions shown on the preceding page. These would serve situations where we are seeking commercially available goods and services. I would rewrite the definitions for the second set as shown on the right in the sidebar. These would be applied when we are attempting to expand the state of the art, particularly when there is a lot of research and development to be accomplished.

Some RFPs at Brooks City-Base have attempted to address this issue. Over the past two years, common language in Air Force Center for Environmental Excellence RFPs has included the following paragraphs in the Section M and in the instructions to the past performance response:

RFP Sec. M -- Where relevant performance record indicates performance problems, the Government will consider the number and severity of the problems and the appropriateness and effectiveness of any corrective actions taken (not just planned or promised). The Government may review more recent contracts or performance evaluations to ensure corrective actions have been implemented and to evaluate their effectiveness.

PPI Form -- Please provide a brief description of service provided under this contract. Include details that will indicate specific efforts of key personnel identified in Question 14 below. Clearly demonstrate management actions employed in overcoming problems and the effects of those actions, in terms of improvements achieved or problems rectified. This may include a discussion of efforts accomplished by the Offeror, or applicable Teaming Partner, to resolve problems encountered on prior contracts as well as past efforts to identify and manage program risk. For example, submit quality performance indicators or other management indicators that clearly support that an Offeror, or applicable Teaming Partner, has overcome past problems.

An Outrageous Suggestion?

To take it one step further, I might suggest asking offerors to address the verbiage below in their proposals. It is very similar in concept to the paragraphs presented above, but it is (perhaps) just a bit more direct. The answer would provide excellent potential for determining an aspiring contractor's commitment to real performance.

Please present a situation during a program where a significant problem or problems developed. Explain the origins and causes of the problem(s) and how the problem(s) was/were detected. If applicable, explain how the problems were solved including (as applicable) how the customer was informed or brought into the process of fixing the problem. Alternatively, if no such situation exists, present a situation where a significant problem was avoided.

My new ratings definitions and the suggestion for a proposal evaluation question aren't going to fit all acquisitions. When we are looking for a system or service in well-defined, well-practiced areas, the current guides serve well. But there are applications where we're pushing the envelope of performance, where there are a lot of potential unknowns, or where we feel that our picture of what we might really need could evolve during the course of an acquisition--and those situations are where my suggestions would have value.

The author welcomes comments and questions. Contact him at [email protected].

Slate currently works as a business integration specialist for 311 HSW/XP at Brooks City-Base. Texas.

Past Performance Rating Definitions

Existing Definitions                Alternative Definitions

BLUE/EXCEPTIONAL
The contractor's performance meets  The contractor's performance meets
contractual requirements and        contractual requirements and exceeds
exceeds many (requirements) to the  many (requirements) to the
Government's benefit. The           Government's benefit. In addition,
contractual performance was         if confronted with problems, the
accomplished with few minor         contractor took corrective actions
problems for which corrective       that were highly effective and
actions taken by the contractor     showed significant effort directed
were highly effective.              to working with the government. Such
                                    corrective actions were often taken
                                    proactively.

PURPLE/VERY GOOD
The contractor's performance meets  The contractor's performance meets
contractual requirements and        contractual requirements and exceeds
exceeds some (requirements) to the  some (requirements) to the
Government's benefit.               Government's benefit. In addition,
contractual performance was         the contractual performance was
accomplished with some minor        accomplished with some minor
problems for which corrective       problems, and when confronted with
actions taken by the contractor     problems, minor or otherwise, the
were effective.                     contractor took corrective actions
                                    that were effective.

GREEN/SATISFACTORY
The contractor's performance meets  The contractor's performance meets
contractual requirements. The       contractual requirements. The
contractual performance contained   contractual performance contained
some minor problems for which       some problems for which corrective
actions taken by the contractor     actions taken by the contractor were
contractor appear or were           satisfactory or for which
satisfactory.                       exceptional efforts were taken but
                                    still proved not to be completely
                                    effective for reasons typically
                                    beyond the contractor's control.

YELLOW/MARGINAL
Performance does not meet some      Performance does not meet some
contractual requirements. The       contractual requirements. The
contractual performance reflects a  contractual performance reflects
serious problem for which the       minor problem(s) for which the
contractor has not yet identified   contractor did not identify
corrective actions or the           corrective actions, or the
contractor's proposed actions       contractor's proposed actions for
appear only marginally effective    problems (serious or minor) appeared
or were not fully implemented.      only marginally effective, or (when
                                    under contractor control) were not
                                    fully implemented.

RED/UNSATISFACTORY
Performance does not meet most      Performance does not meet most
contractual requirements and        contractual requirements, and
recovery is not likely in a timely  recovery is not likely in a timely
manner. The contractual             manner. The contractual performance
performance contains serious        contains serious problem(s) for
problem(s) for which the            which the contractor's corrective
contractor's corrective actions     actions were ineffective or
appear or were ineffective.         reflected serious problems for which
                                    the contractor did not identify
                                    corrective actions.

NOT APPLICABLE
Unable to provide a score

COPYRIGHT 2005 Defense Acquisition University Press
COPYRIGHT 2005 Gale Group