A new approach to measuring health system output and productivity.

Castelli, Adriana ; Dawson, Diane ; Gravelle, Hugh 等

This paper considers methods to measure output and productivity in the delivery of health services, with an application to the NHS hospital sector. It first develops a theoretical framework for measuring quality adjusted outputs and then considers how this might be implemented given available data. Measures of input use are discussed and productivity growth estimates are presented for the period 1998/9-2003/4. The paper concludes that available data are unlikely fully to capture quality improvements.

Keywords: Health output; productivity; quality adjustments

JEL classifications: D24; 111; L8

1. Introduction and context

Previous papers in this issue have set out the methodology and results from applying the standard growth accounting approach to measuring productivity. The EU KLEMS database includes estimates of output and productivity growth for the non-market services including public administration, education and health and social services, employing data from the National Accounts in each country. However, output in these services is known to be very poorly measured with little yet by way of international consistency. This is now changing following the guidelines set out in the Eurostat (2001) Handbook on price and volume measures in National Accounts and the publication of the Atkinson Review (Atkinson 2005)--see discussion below.

Measurement of output and productivity in non-market services is not just of interest to those working within the national accounts tradition but is also important for policymakers charged with providing, funding and/or regulating these services. In March 2004 the UK Department of Health (DH) commissioned a research team from the Centre for Health Economics at the University of York and the National Institute of Economic and Social Research to develop new approaches to measuring NHS outputs and productivity. This paper summarises the main findings from that work. The main focus of this paper is to set out the methodology put forward by the research team to measure outputs and inputs and presents a few sample results for the UK NHS. Readers interested in details of the methods, data and calculations should consult the final report to DH, Dawson et al. (2005), and the method applied to selected health treatments in Castelli et al. (2007).

The Eurostat Handbook made important recommendations on measurement of non-market output. For purposes of national accounting the preferred method is to measure outputs (e.g. treatment received by a patient) rather than activities (number of operations or prescriptions), and outputs should be quality adjusted. Eurostat deemed the previously widely employed method of measuring outputs by inputs as unacceptable and as a result EU countries are moving away from this method in large areas of non-market services in the direction of the preferred method. (1) In major respects Atkinson (2005) recommended a methodology for measuring NHS output growth advocated in the preliminary reports of the York/NIESR team as applied to health services, as detailed below.

In this paper we formulate an index of health service productivity that incorporates quality change and calculate the annual value of this index for the English NHS from 1997/8 to 2003/4. The next section describes the approach taken to output measurement and we then describe the data used to populate the output index (section 3) and the results (section 4). The input index, its construction, data and sources are briefly described in section 5, which also includes estimates of NHS hospital sector productivity growth. Concluding comments are offered in section 6.

2. Measuring output

An aggregate indicator of the movement of the output of an industry which produces a range of goods is constructed by measuring the percentage volume change in each output and weighting the resulting percentage changes by the share of the value of each product in the value of total output. Thus, in Laspeyres form, where activities are valued in the base period, the index takes the form:

[I.sup.1] [J.summation over j=1][x.sub.jt+1][v.sub.jt] / [J.summation over j=1][x.sub.jt][v.sub.jt] (1)

where [x.sub.jt] is the volume of output j in period t, and [v.sub.jt] is the marginal social value of output J. For many goods and services, prices are taken to reflect the consumer's marginal willingness to pay for them. So, in the National Accounts, prices are used to value private sector output. But consumers do not face full price for goods and services that are financed by the public sector so values must be imputed in other ways. Eurostat recommends the use of unit costs for this purpose. In this case the output index becomes:

[I.sup.2] [J.summation over j=1][x.sub.jt+1][c.sub.jt] / [J.summation over j=1][x.sub.jt][c.sub.jt] (2)

where [c.sub.jt] is the cost of output j at time t. There are two major deficiencies with this approach. First, it is questionable whether costs are proportional to marginal social values. Proportionality requires the use of marginal costs and that public sector resources are allocated in line with social preferences (i.e. that the system is allocatively efficient). Second, the index takes no account of changes in quality. For example, when the health system adopts more cost effective ways of treating patients, a cost weighted output index may record a fall in output, depending on the relative magnitudes of unit cost reductions and increased treatments.

It is helpful to distinguish between activities (operative procedures, diagnostic tests, outpatient visits, consultations), outputs (courses of treatment which may require a bundle of activities), and outcomes (the characteristics of output which affect utility). The distinction between outputs and outcomes is identical to that between goods and characteristics in consumption technology models (Deaton and Muellbauer, 1980, Ch. 10; Lancaster, 1971) where consumers value goods because of the bundle of characteristics that yield utility.

Measurement in the private sector focuses on outputs rather than the characteristics they produce because of the assumption that the market price of the output measures the consumers' marginal valuation of the bundle of characteristics from consuming the output. In the private sector we do not need to concern ourselves with measuring characteristics or counting activities because they are embodied in the outputs which are produced and the prices at which they are sold. In the public sector there are no prices to reveal patients' marginal valuations of services so we have to find other means of estimating their value and we define the quality of the output as a function of the vector of outcomes it produces. This means that values can be found by combining information on outcomes with that about outputs or activities; see the discussion for the health sector in Triplett (2001) or non-market services in general in O'Mahony and Stevens (2006).

How do we combine information on outcomes with outputs? We can do so in two equivalent ways: we can measure the outputs and attempt to estimate the marginal valuations attached to them, or we can measure the outcomes produced by each unit of output and attempt to estimate marginal valuations of the outcomes. The bundle of outcomes produced by a unit of output is likely to change over time in the health sector because of, among other things, changes in technology or treatment thresholds. In a private market the price of output would change to reflect this. But in the absence of market prices for health sector outputs it is likely to be easier to calculate the change in the marginal value of output by focusing on the change in the vector of outcomes. We show below how the changing mix of outcomes (quality change) may be allowed for in principle. We discuss in section 4 how quality adjustments based on the currently available data can be incorporated into an output index and show the results of applying these methods to calculate experimental quality adjusted indices. We have made suggestions as to how the quality adjustment can be improved by the collection of additional data in the concluding section.

Health services have many characteristics which patients regard as important. These include the impact of treatment on health outcomes, the length of time waited for treatment, the degree of uncertainty attached to the waiting time, distance and travel time to services, the interpersonal skills of GPs, the range of choice and quality of hospital food, the politeness of the practice receptionist, the degree to which patients feel involved in decisions about their treatment, etc. In this paper we consider how to incorporate two important dimensions of quality in a cost-weighted index: the health outcomes arising from treatment and how long patients have to wait before receiving treatment.

The most important characteristic of health care is the contribution it makes to health status. The measure of health outcome should indicate the 'value-added' to health as a result of contact with the health system. Given the objectives of the health system, it is important to measure the health gain accruing to patients as a consequence of treatment. A unit of measurement used in a number of countries is the Quality Adjusted Life Year (QALY). This measures the change in the quality of life and the duration of the benefit and, therefore, allows both for treatments that improve the quality of life without affecting life expectancy and treatments that improve life expectancy. Health gains accruing over time are discounted and appear as the present value of the benefit of treatment. In theory the health outcome consequent on treatment is the difference between QALYs with treatment and without treatment. In figure 1, the area under the 'with treatment' curve less the area below the 'without treatment' curve gives the health benefit of the treatment. In practice, even with routinely collected data on health outcomes, it will not be possible to measure this true treatment effect. First, for ethical reasons, few patients are left without treatment. All we will observe is the difference between health state before treatment and after treatment. Second, even routinely collected outcomes data would not provide a continuous monitoring of post treatment health state. We would only have snapshot estimates at particular points in time.

[FIGURE 1 OMITTED]

Lack of data on without treatment health states may not be a serious deficiency for construction of an output index designed to measure the rate of growth of quality adjusted output. If, over time, improved treatment results in after treatment health state [h.sup.*] in year t+1 being higher than in year t, an index using observational data will record an increase in quality adjusted output. Where the aim is to measure the rate of growth of output, we are interested in whether the rate of growth of [DELTA]h = [h.sup.*] - [h.sup.o] is a reasonable approximation to the rate of growth of the effect of treatment on the discounted sum of QALYs. The important issue is how well the rate of change in measures based on the snapshots [h.sup.*], [h.sup.o] approximates the rate of change in the areas under the two time profiles of health streams with treatment [h.sup.*](s) and without treatment [h.sup.o](s).

Hence, despite the imperfections of the difference between snapshots of before and after treatment health status for calculating the level of productivity, we suggest that rates of change of measures based on snapshot measures of [h.sup.*], [h.sup.o] will improve estimates of health sector output growth compared to estimates where such information is not used.

In practice, including health outcomes in the output index has proved difficult, mainly because of a lack of data. One important outcome is whether patients survive their treatment. Survival rates, though, are only part of the story. In England, fewer than 3 per cent of NHS patients die within 30 days of their hospital treatment (Dawson et al., 2005). Routine data on health outcomes are not collected for the majority of patients treated in most health care systems so there is no information with which to quality-adjust the vast majority of health care activities.

In the absence of data on health effects of treatment we have no alternative to using unit costs to weight outputs. Hence we suggest indices that make use of currently available data to quality adjust cost weighted outputs. Setting health status when dead to zero, the expected increase in discounted QALYs from treatment j at time t is:

[q.sub.jt] = (1 - [m.sub.jt])[q.sup.*.sub.jt] - [q.sup.o.sub.jt] (3)

where [m.sub.jt] is the probability of death within a short period of treatment for condition j, [q.sup.*] are discounted QALYs if treated and [q.sup.o] are discounted QALYs if patients were left untreated. Letting a denote the survival rate (= 1 - m), the ratio of expected discounted QALYs across time period t + 1 and t, on the assumption that [q.sup.*.sub.jt] - [q.sup.*.sub.jt+1] does not change over time is given by:

[q.sub.jt+1] / [q.sub.jt] = [a.sub.jt+1][q.sup.*.sub.jt] - [q.sup.o.sub.jt] / [a.sub.jt][q.sup.*.sub.jt] - [q.sup.o.sub.jt] = [a.sub.jt+1] - [k.sub.jt] / [a.sub.jt] - [k.sub.jt] (4)

where [k.sub.jt] = [q.sup.o.sub.jt] / [q.sup.*.sub.jt]. The only reason why [q.sub.jt] changes over time is that the post-operative survival rate changes. Clearly increases in [a.sub.jt+1], other things equal, lead to a higher quality adjustment factor.

Two variants of equation (4) were used to estimate a quality adjusted output index. The first assumed that the no treatment outcome would have been death so [q.sup.0] and hence k equals 0. In this case, assuming resources are optimally allocated so unit costs are a valid measure of the value of treatment, we can calculate the survival adjusted cost weighted output index:

[I.sub.1] = [[summation].sub.j][x.sub.jt+1]([a.sub.jt+1] / [a.sub.jt])[c.sub.jt] / [summation.sub.j] [x.sub.jt][c.sub.jt] (5)

where x are activities, c are unit costs and a are the survival rates from treatment. In practice there may be conditions whereby the no treatment outcome, [q.sup.0], would have been survival but at a reduced level of health. Suppose the ratio [k.sub.jt] = [q.sup.o.sub.jt] / [q.sup.*.sub.jt]) was constant through time. Then the quality adjusted cost weighted activity index is given by:

[I.sub.2] = [[summation].sub.j][x.sub.jt+1]([a.sub.jt+1] - [k.sub.j] / [a.sub.jt] - [k.sub.j])[c.sub.jt] / [summation.sub.j] [x.sub.jt][c.sub.jt] (6)

which we call the survival adjusted cost weighted output index with health effect.

Dawson et al. (2005) show that the index (5) will underestimate the more general index (6). Recall that these indexes are merely taking account of increases in survival. Suppose we compare two patients, one whose condition is such that he has zero chance of survival if not treated (k = 0) and another who would continue to live without treatment, although at a reduced health level. Suppose there is an increase in the mortality rate from treatment. The loss to the patient who would have died anyway is small relative to the loss experienced by the patient who chooses treatment rather than enjoying her remaining QALYs had she foregone treatment. Conversely, if a new treatment increases the survival rate for both, the risk from undertaking the operation is reduced more for the patient who had some chance of survival. Sample calculations of both the simple survival adjustment and that involving health effects are presented in section 4 below.

An important observation is that unless we are willing to assume that k = 0, we cannot divorce a survival adjustment from the health effect. In addition the assumptions underlying these calculations are unlikely to hold in practice, especially given that post-treatment outcomes do not change over time. Without data on impacts on health status before and after treatment, quality adjustments based on health outcomes are likely to be minimal. The assumption underlying equations (5) and (6) that resources are allocated efficiently is also unlikely to hold in most health systems. If unit costs are not proportional to health gains, then the index is likely to be biased in directions difficult to fathom. For example an increase in survival will have a smaller effect on the index the smaller is the cost weight [c.sub.jt]. As long as unit costs are proportional to health gains this is reasonable. The increase in the health effect from an increase in survival is proportional to [q.sup.*.sub.jt] and the smaller is [c.sub.jt] the smaller is [q.sub.jt] = [a.sub.jt][a.sup.*.sub.jt] - [q.sup.o.sub.jt] and the more likely is [q.sub.jt] to be small. But if this condition does not hold, the fact that survival gains in low cost activities will have smaller effects on the index than survival gains in high cost activities is less appealing if the health gains from those low cost activities are significant.

Instead, if patients' marginal valuations of the gains from treatment were available, one could use a value weighted activity index--see examples of the application of this alternative in Castelli et al. (2007).

The index in equation (6) assumes that there is no change in life expectancy over time. We can argue that, for some treatments, changes in life expectancy after treatment and without treatment are primarily due to factors outside the control of the health system and so should not affect the calculation of the growth rate of health output from one period to the next. However, the age structure of patients treated may change over time and, if younger patients are treated, they will have longer to enjoy the increased health status post treatment. Therefore we can modify the basic index to take account of the age of treated patients--the modification to the basic equation to account for this is given in the Appendix table.

In conclusion, on health effects, in future it may be possible to estimate [q.sup.*.sub.jt], [q.sup.o.sub.jt], using new data on longer-term survival and on health status from surveys of patients before and after treatment and from the results of evaluations of different types of treatment. Until then the quality adjustments for this characteristic are likely to be a serious underestimate of the true change in output of the health sector.

The second characteristic which the research project focused most attention on was waiting times. Waits for diagnostic tests and treatment may affect individuals in two ways. First, they may dislike waiting per se irrespective of the effect of treatment on the discounted sum of their quality adjusted life year ([q.sub.jt]). Thus waiting time is regarded as a separate characteristic of health care, distinct from its effect on health. Second, longer waits delay and hence reduce the health gain from treatment. Delay may be associated with deterioration in the patient's condition and the pain and distress while waiting for treatment results in a loss of quality adjusted life years for the patients affected. Here, the waiting adjustment is akin to a scaling factor multiplying the health effect.

Two main ways of modelling waiting as a scaling factor were developed. The first is to value treatment at the time the patient is placed on the waiting list; health effects are therefore discounted to this date. The second is to value treatment at the date it is actually received; health effects are discounted to the date of treatment. The first has the property that waiting has a cost, in units of health, which increases with the length of the wait but at a decreasing rate; an extra day after a long wait costs less than an extra delay after a short wait. The second adjustment is more consistent with the timing implicit in equations (5) and (6) above--we measure activity when it takes place, which suggests that we should measure the benefit from treatment at the time it takes place. In this case discounting acts like an interest charge--the longer the wait the greater the charge, so that reducing long waits has a greater impact than reductions in short ones.

Finally we also considered incorporating waiting times as a separate characteristic where results from contingency valuation studies were used as weighting factors. These studies ask patients how much money they would be prepared to forego for both an increase in health level and a reduction in waiting times. Equations incorporating waiting times adjustments are given in the Appendix table.

3. Data for the output index

Activity data

We calculate output growth in the NHS from 1998 to 2004 (Dawson et al., 2005). It has been possible to make the NHS output index more comprehensive over this period, due to better data collection. The number of different activities for which data were available expanded from 1,321 categories in 1998/9 to 2,060 by 2003/4. In table 1, these activities are grouped by sector under broad headings for the hospital and primary & community care sectors.

Given space constraints, this paper concentrates solely on hospital activity--readers are referred to Dawson et al. (2005) for details for other activities. The best data in the UK relate to hospital inpatient activity and detailed electronic information is available for every patient treated in the NHS hospitals (the Hospital Episode Statistics, HES). We define a unit of hospital output as 'continuous inpatient spells of NHS care' (CIPS). These are defined as continuous periods of patient care anywhere within the NHS. An NHS spell might comprise multiple episodes, including the transfer from one hospital to another as well as the move from one consultant to another within a given hospital (Lakhani et al., 2005). CIPS capture the 'completed treatment' to the maximum extent that is currently feasible in the NHS.

Activity is described using 565 Healthcare Resource Groups (HRG), which are the UK equivalent to Diagnostic Resource Groups (DRG). Activity in each HRG is reported separately for elective and non-elective (emergency) treatment. Figure 2 graphs the number of episodes for each year from 1998/9 to 2003/4. It shows little change in electives up to 2001/2 with some growth thereafter. Non-electives show more significant growth, especially in the final year.

[FIGURE 2 OMITTED]

Health outcomes

The health outcomes associated with each treatment are not observed directly, so have to be estimated by piecing together the probability of short-term survival from treatment, post-treatment health status and healthy life expectancy. Thirty day survival rates are derived from HES and show a continuous improvement from 1998/9 to 2003/4 (table 2).

Healthy life expectancy is derived from life tables and the 1996 Health Survey for England (Prescott-Clarke et al., 1998) and estimated according to the average age of patients in each HRG.

Waiting times

A survey of the literature on the cost of waiting (Hurst and Siciliani, 2003) found some evidence on deterioration of health and premature death associated with waiting for cardiology treatment but little for other procedures. The main variant of our suggested scaling waiting time adjustment has a charge for waiting which represents the welfare loss as a result of not having been treated immediately. This is an offset against the benefit of the treatment which applies for the residual life span. Interest is charged on the cost of waiting and the marginal disutility of waiting increases with the length of the wait. We refer to this as 'discounting to date of treatment with charge for waiting'.

We calculate indices using a measure of inpatient waiting time termed the 'certainty equivalent wait'. This reflects the possibility that the disutility of being on a waiting list depends not just on the average wait but also on the risk of a longer than average wait. We measure the certainty equivalent wait as the waiting time for patients at the 80th percentile of the waiting time distribution for each elective HRG. Reductions in these relatively high waiting times deliver benefits to all patients on the waiting list by reducing the uncertainty of a long wait.

Waiting times are measured as the time between being placed on the inpatient waiting list and being admitted. Year-on-year changes in waiting times were volatile over the period, whether measured as the mean days spent waiting or at the 80 per cent percentile of the distribution (table 2).

Cost weights

Since 1998, every NHS provider has been required to provide information on its cost and volume of activity by HRG to the Department of Health. This information is published annually as the National Schedule of Reference Costs. These costs are the basis for setting tariffs for hospital care under 'Payment by Results' (Department of Health, 2006a). The procedure for assigning costs to CIPS is detailed in Dawson et al. (2005). The coverage of the NSRC has increased over time, to include costs of outpatient care, critical care services, accident & emergency, pathology, radiology, chemotherapy, and rehabilitation (Department of Health, 2006b).

4. Results on output growth

We first present calculations of output growth to illustrate the sensitivity of estimates to various quality adjustments to take account of survival, for inpatient activity (table 3). The first quality adjustment includes the survival adjustment alone. The next two include the health effect; assuming a value of k = 0.8 across all treatments except where mortality (m) is high (column 3) and the previous calculation but including the life expectancy adjustment (column 4). The value of k = 0.8 is derived from the sample of procedures where data on before and after treatment outcomes are available (Castelli et al., 2007). Looking at the average across the five years of the study, including a simple survival adjustment increases growth in output by a small amount. Including a health effect increases the adjustment for the reasons set out above. Finally the inclusion of the life expectancy term reduces the quality adjustment reflecting the increasing average age of patients treated in the NHS during this time (Dawson et al., 2005).

We next consider the impact of allowing for changes in waiting times. The result was that none of the waiting time adjustments had any marked effect on growth rates, which is unsurprising given the relatively small changes observed in table 2. We report summary results in table 4 in the form of average annual growth rates over the period 1998/9 to 2003/4. All are based on the same survival adjustment with k = [q.sup.0]/[q.sup.*]=0.8 and the mortality cut off set to m = 0.10. Other uniform survival adjustments made little difference to the effects of the waiting time adjustments.

The first and third rows show that there is little difference in the effect of the forms of the waiting time adjustment. Comparison of the first and second row shows that the choice of waiting time measure has no impact. Since the waiting times and life expectancy factors are non-linear, and there is a variation in waiting times and in ages within an HRG in a given year, it is possible that our use of a single waiting time and life expectancy estimate for each HRG may lead to misleading results. To explore this, we used individual-level data to compute the equivalent of the waiting time adjustment with discounting to date of treatment with a charge for waiting. The results are in the fourth row and again differ little from the same index form using the 80th percentile wait in the first row.

The results show little or no impact from adjusting for changes in waiting times, regardless of the formulae or measures of waiting time employed. The small effects of waiting time adjustments are largely driven by the lack of change in waiting times rather than the methods used. To see this, suppose waiting times for the 80th percentile were reduced by 10 per cent for all HRGs comparing 2003/4 with 2002/3. Then the discount to date of treatment with charge for wait and low discount rates equal to 1.5 per cent would add 0.16 percentage points. With the same discount rates, reducing waits at the 80th percentile by 50 per cent would add 1.12 percentage points to the growth rate in that year. While further significant reductions in waiting time will increase the growth of NHS output, it is important to note that the index measures changes in both health gain through treatment and reduced waiting time. Treatment may generate improved quality of life for ten years while a reduction in waiting from six months to three months will add only a fraction to the overall gain in output.

In addition, the impact of changes in waiting times is dependent on the cost shares used to weight activities. If reductions in waiting time are mainly concentrated in low cost procedures, they will have a lower impact on the aggregate cost weighted output index than if reductions were more uniformly distributed. This appears to be true for the data employed in this study, as illustrated in figure 3 for the final growth period. In this year 63 per cent of the elective HRGs showed a decrease in waiting times, with a smaller 52 per cent showing reduction in 80th percentile waits. Very large reductions in either mean waits or 80th percentile waits appear to be concentrated in low cost procedures.

[FIGURE 3 OMITTED]

In summary, the results above suggest that the impact of incorporating survival adjustments into a cost weighted output index depends on assumptions about the health effect but, even so, the adjustments are quite small. These results are hardly surprising since the majority of health procedures have little impact on survival. The data necessary to incorporate changes over time in health status from treatment are not yet available. Waiting time reductions do not appear to have a significant impact in the time period considered, regardless of which model or measure of waiting times is used. This is due to the small reductions in waiting times over the period and to the limited impact of these reductions on health status.

5. The input index and productivity

To calculate total factor productivity we calculate an index of total input use over the period:

[Z.sub.t] = [summation over i][[bar.[omega]].sub.it][z.sub.it] (7)

where [z.sub.it] is the amount of the i'th input used in producing the activities in period t and [[bar.omega].sub.it] is the social cost per unit of the i'th input.

As in previous papers in this issue, we divide inputs into three broad categories: labour, capital and purchased inputs. These papers emphasised that changes in the quality of factor inputs are important in explaining productivity change. Labour is by far the most important input used in producing health services, accounting for about 75 per cent of total hospital expenditures. Current practice by DH and ONS is to calculate labour input by deflating payments to labour by a wage index. This is an indirect measure of the volume of labour input. It is more usual to estimate labour inputs based on direct measures, such as the number of persons engaged or hours worked. In particular, the OECD productivity manual recommends the use of annual actual hours worked, to allow for time paid including overtime, holidays and sick leave.

In addition, labour input should be adjusted to take account of variations in types of workers employed, in particular the changing use of skilled workers. The growth accounting method employed to adjust for the greater productivity of highly skilled workers mirrors that in previous papers in this issue. This divides labour hours by skill type and then weights the growth in hours of each type by their wage bill shares. This captures the fact that more highly skilled workers get paid more than the unskilled, and, under competitive market conditions, the wage paid reflects the marginal productivity of workers of different types.

Two principal sources of data about NHS labour are available--the NHS Workforce Census, an annual census carried out by the Department of Health, and the Labour Force Survey, a quarterly sample survey carried out by ONS. The two sources show similar but not identical trends through time (Dawson et al., 2005). Data from the NHS Workforce Census was used to obtain a headcount measure of labour input, while data from the LFS was used to incorporate hours worked, to incorporate quality adjustments, and to adjust for agency staff.

To construct a quality adjusted measure of labour input, data on the proportion of workers in each skill group and their wage rates from the LFS were combined with the NHS Census data, which acted as control totals. The data show evidence of upskilling across the NHS workforce, not only among doctors, nurses and other health professionals, but also at the lower end of the skill distribution. There has also been a marked decline in the share of the workforce with no skills.

However, changes in the skill use pattern will only impact if there are also significant differences in wage rates across skill groups. This appears to be the case in the NHS - on average those with higher degrees were earning about four times the average unskilled wages.

Additional refinements to the quality adjusted index were also considered. First, a further disaggregation of doctors was carried out using data from the NHS staff earnings survey--this resulted in growth in quality adjusted doctors about 30 per cent above numbers employed, suggesting the qualification data alone (which resulted in an increase of about 4 per cent) are not sufficient to capture all quality change. However, with doctors representing less than 1.5 per cent of the NHS Trusts wage bill, this leads to only a very small adjustment for the hospital sector.

Secondly, an adjustment for training received by an individual in addition to their certified qualifications was also made. In 2003/4, for example, LFS data showed that just over 50 per cent of staff in the NHS had received some form of job related training. Clearly, the impact of training on workers' productivity will vary considerably depending on the type of training in question. The results of adjusting for job related training were to raise the quality adjusted labour input growth rate on average from 1999/2000 by about 5 per cent, a small but not insignificant impact. Therefore our estimates of quality adjusted labour were scaled up to reflect the impact of on-the-job training.

Intermediate and capital inputs

Intermediate input expenditure data for the hospital sector comes from Trust Financial Returns (TFR) and is deflated by a modified version of the DH Health Services Cost Index (HSCI) to derive a volume measure. Intermediate input was defined as all current non pay expenditure items in the TFR, and hence excluded all purchases of capital equipment and capital maintenance expenditures as these items cannot be allocated to a particular year's output. The share of hospital drugs in intermediate expenditure has been rising rapidly, from about 24 per cent in 1998/9 to 34 per cent in 2003/4. The share of external purchase of health care from non-NHS bodies has also increased through time but remains small at about 6 per cent of total intermediate expenditure in 2003/4.

These numbers for intermediate input were deflated by an aggregate price index, derived as a chain linked index of corresponding HSCI items. This resulted in a very small upward adjustment in the intermediate input deflator compared with one using all items in the HSCI, as the prices of capital items have been growing more slowly than current items and in the case of computers have been falling. Capital input for the hospital sector is measured by depreciation reported in the Trust Financial Returns, deflated by the ONS capital consumption deflator, plus an allowance for depreciation of capital purchases in the current year, deflated by a chain linked deflator for capital items in the HSCI.

It should be noted that the prices and therefore also the quantities of drugs used are measured in a way which almost certainly does not take account of improvements in quality over time. Thus, compared with a quality adjusted index of inputs, we probably understate the growth in inputs used by the NHS and overstate the rise in productivity.

Table 5 shows average period input shares and average growth in the three inputs where labour input is the quality adjusted variant. Labour represents the highest share followed by intermediate inputs. The growth rates of each of the three input categories are very close in this time period.

Productivity

Combining the input shares with growth in real inputs allows the calculation of total input growth. Subtracting this from output growth yields total factor productivity growth rates. In order to calculate hospital level productivity growth rates, it is necessary to incorporate activities other than inpatients into the analysis, to maintain consistency between output and input measures. Thus we added outpatients, accident and emergency and other procedures carried out in hospital to our cost weighted output index; the only quality adjustment for these additional activities was the inclusion of waiting times for outpatients. In addition before-and-after measures of health status were available for a handful of HRGs (Castelli et al., 2007). In order to apply equation (6) to all inpatient activities we have to assume values for k. This is included mainly for illustrative purposes. For elective procedures we assume that the ratio of before-and-after health status was 0.8, as suggested by the mean for the elective procedures for which estimates were available. For non-elective HRGs we assume [h.sup.o.sub.j]/[h.sup.*.sub.j] = 0.4 on the grounds that non-elective patients may have worse health than elective patients if not treated.

The fact that we do not have comprehensive data on before-and-after treatment outcomes means that our uniform adjustment understates health outcomes for some HRGs and overstates health outcomes for others. Most especially, some patients are admitted to hospital when the expectation of death is very high. For HRGs where the mortality rate is high, we make no health effect adjustment and use only the change in survival. Finally we included adjustments for waiting times using 80th percentile waits and by assuming discounting to date of treatment with charge for wait, with discount rates on waits and health equal to 1.5 per cent.

Table 6 shows the TFP growth rates for both the unadjusted and quality adjusted variants of our output measures. In most years this is negative, with a slight tendency to improve over time. The finding that TFP growth is negative is not unusual in the private sector, as can he gleaned from the EU KLEMS database. Negative TFP growth is most likely to occur in service sectors where output is poorly measured and quality adjustment is minimal. When inputs are measured correctly, with adjustments for quality change, then the TFP residual is close to a measure of pure technical change so long as output is also measured correctly. But as emphasised in many parts of this paper, we are only capturing part of the improvement in quality of care via our proposed adjustments for survival, health effects and waiting times. Because of this incomplete adjustment for quality change we expect to underestimate TFP growth. There are also reasons why in the short term at least we might expect negative growth rates. The literature on the impact of information technology on productivity in the private sector points to an important role of organisational changes in facilitating benefits from new technology, with the suggestion that we could observe declining TFP in the short run due to disruption of production processes. There is no doubt that the NHS is undergoing significant organisational change.

Of more consequence for the health sector is the notion that there are diminishing returns as increased activity allows treatment of more complex and hence most costly cases. Some evidence in support of this is provided by the increased average age of patients treated in hospitals, from 48.6 years in 1999/00 to 50 years in 2003/4. In addition there has been some increase in the expenditure shares of HRG categories with the title 'complex elderly' from 3.4 per cent of expenditures to 4.2 per cent over the same period. Changes in the case mix are likely to be larger within than across HRGs but we lack the necessary data to examine this. Data that identified the characteristics of patients would also be useful in identifying the extent to which changes in health sector productivity are affected by diminishing returns.

6. ONS work

The Office for National Statistics has built on the work described here (UKCEMGA, 2006) and has added a number of extra dimensions to the approach. The study rightly explores a number of issues not covered in this paper, such as benefits of improved primary health care; progress obviously needs to be made with such issues. Nevertheless they draw public attention to some issues which we feel deserve comment.

The ONS study focuses on the benefits of statins in reducing coronary heart disease. This brings to the fore a question we allude to only in passing; the benefits from statins obviously lead to an increase in the gross output of the health service. But do they reflect an increase in total factor productivity? In other words, are they an input supplied by the pharmaceutical industry or are the benefits resulting from them entirely a result of the skills of health service staff and nothing to do with the pharmaceutical industry? However this issue is to be resolved, it seems to us unlikely that the answer is that adopted in the ONS study--that the benefits are entirely due to medical staff and nothing to do with the industry which supplies the drugs. Thus we fear that the ONS study overstates the productivity gains arising from this source. To address this issue the same sort of effort needs to be put into measuring the output of the pharmaceutical industry, which has been deployed in measuring production of computers.

Secondly the study includes an element reflecting changes to 'patient experience' based on data collected from patients about how happy they have been with a range of aspects of their treatment, such as politeness of doctors and quality of hospital food. Quantifying and aggregating such data in order to produce a single index raises all sorts of problems. But even if these have been resolved there is a more important one which the ONS study does not address. How much does patient experience matter, relative to what we assume to be the core function of the health service, treating patients successfully? The ONS study assumes that a 1 per cent improvement in patient experience is of equal importance in an overall output index to a 1 per cent point increase in survival rates. It seems to us very unlikely that most patients, when presented with this sort of equivalence, would say that it reflected their views. A weighting of even one to ten comparing 'patient experience' to survival rates might seem to be on the high side for the former. In any case some thought needs to be given to the issue before it is possible to produce a satisfactory composite index.

Finally, the ONS study presents figures taking account of the Atkinson adjustment to allow for the fact that the value of health increases with time and probably broadly in line with real earnings. While we do not have any difficulty with the proposition that the value of health increases with time, we have yet to be convinced of the case for the Atkinson adjustment, noting that it is difficult to relate to an analysis of the link between national accounts, the concept of income and economic welfare (Sefton and Weale, 2006).

7. Conclusion

We have demonstrated how quality change can be accounted for in calculating indices of output and input for the health sector and explored the influence of different assumptions about the form of quality adjustment. These indices have been estimated for the hospital sector of the English NHS using the best available data. As more data become available it will be possible to obtain a clearer impression of the progress that the health sector makes in advancing social welfare.

Dawson et al. (2005) recommend routine collection of data on outcomes before and after treatment. They suggest that the following points need to be addressed in such a data collection exercise. First, since the scale of NHS activity is so broad and the potential volume of patients is so large, sampling seems a more sensible strategy than attempting to measure health effects for all NHS patients in a sector. Second, the timing of before and after health status measurement may depend on the type of activity; clinical advice could be used to determine the appropriate period for follow-up surveys. Third, given the pace of technological change in medicine, a continuous sampling of the NHS patient population will be required to capture trends in the impact of NHS services on patients.

Finally, the research team considered the feasibility of a programme of routine collection of outcomes data. They suggest this could start with a few high volume elective and medical conditions that would permit sampling rather than complete coverage. The data would also be immensely useful for other purposes, including monitoring of Trust performance and improved cost-effectiveness analysis of particular treatments.

ACKNOWLEDGEMENTS

This research was funded by the Department of Health, to whom we owe thanks. The views expressed here are not necessarily those of the Department of Health. This report has benefited greatly from discussion with numerous experts, including the members of the Steering Group Committee, Jack Triplett (consultant to the project team), Sir Tony Atkinson, Barbara Fraumeni, Andrew Jackson, Azim Lakhani, Phillip Lee, Alan Maynard, Alistair McGuire and Alan Williams. A number of individuals and organisations contributed to assemble the data used in this report. We are grateful to Kate Byram, Mike Fleming, Geoff Hardman, James Hemingway, Sue Hennessy, Sue Macran, Paula Monteith, Casey Quinn, Sarah Scobie, Bryn Shorney, Craig Spence, Karen Wagner, Chris Watson, BUPA, the Cardiff Research Consortium, Health Outcomes Group and York Hospitals NHS Trust. Any errors and omissions remain the sole responsibility of the authors.

REFERENCES

Atkinson, T. (2005), 'Measurement of government output and productivity for the national accounts', Atkinson Review: Final Report, HMSO, 31 January.

Castelli, A., Dawson, D., Gravelle, H., and Street, A. (2007), 'Improving the measurement of health system output growth', Health Economics (forthcoming).

Dawson, D., Gravelle, H., O'Mahony, M., Street, A. and Weale, M., Castelli, A., Jacobs, R., Kind, P., Loveridge, P., Martin, S., Stevens, P. and Stokes, L. (2005), 'Developing new approaches to measuring NHS outputs and productivity: final report', York: Centre for Health Economics, Research Paper 6: 2005.

Deaton, A. and Muellbauer, J. (1980), Economics and Consumer Behaviour, Cambridge, Cambridge University Press.

Department of Health (2006a), Implementing Payment by Results: Technical Guidance 2006/07: Executive Summary, London, Department of Health.

--(2006b), Reference Costs 2006 Collection: Costing and Activity Guidance and Requirements, London, Department of Health.

Eurostat (2001), Handbook on Price and Volume Measures in National Accounts, Luxembourg, Office for Official Publications of the European Communities.

Hurst, J. and Siciliani, L. (2003), 'Tackling excessive waiting times for elective surgery: a comparison of policies in twelve OECD countries', OECD Health Working Paper 6, Paris, OECD, available at www.oecd.org/health.

Lakhani, A., Coles, J., Eayres, D., Spence, C. and Rachet, B. (2005), 'Creative use of existing clinical and health outcomes data to assess NHS performance in England: Part I--performance indicators closely linked to clinical care', British Medical Journal, 330, pp.1426-31.

Lancaster, K. (1971), Consumer Demand: A New Approach, New York, Columbia University Press.

O'Mahony, M. and Stevens, P. (2006), 'International comparisons of output and productivity in public services provision: a review', in Boyne, G.A., Meier, K.A., O'Toole, L.J. and Walker, R.M. (eds), Public Service Performance: Perspectives on Measurement and Management, Cambridge, Cambridge University Press.

Prescott-Clarke, P. and Primatesta, P. (1998), Health Survey for England 1996, London, The Stationery Office.

Sefton, J. and Weale, M.R. (2006), 'The concept of income in a general equilibrium', Review of Economic Studies, 73, pp. 219-49.

Triplett, J. E. (2001), 'What's different about health? Human repair and car repair in national accounts and national health accounts', in Cutler, D. and Berndt, E. (eds), Medical Care Output and Productivity, Chicago, Chicago University Press.

UKCEMGA (2006), 'Public Service Productivity: Health', http:// www.statistics.gov.uk/cci/article.asp?ID=1429.

NOTE

(1) The use of quantity indicators is seen as feasible for services provided to individuals, such as education, health and social services, but less likely to be implementable for collective services such as defence--inputs are likely to remain used for measuring output in these services.

Table 1. Components of the NHS output index

 Activity categories Cost share
 2003/4 2001/2

Hospitai sector
Inpatients 1136 33.48
Outpatients 299 10.99
Accident & Emergency 9 2.17
Critical care services 12 3.97

Primary & Community care sector
GP & practice nurse consultations 5 12.44
Dentists 1 4.69
Prescriptions 184 16.48
Mental Health 19 9.56
Other * 99 6.20
Total 2060 100.00

Note: * includes audiological services, renal dialysis, pathology,
radiotherapy, chemotherapy, spinal injuries, rehabilitation, ambulance
journeys, ophthalmic services etc.

Table 2. Change in mortality rates and waiting times

 Waiting time (days)
 30 day
 survival rate (%) mean 80th percentile

1998/99 96.92 88.7 132.2
1999/00 96.94 80.8 117.7
2000/01 97.07 82.3 119.0
2001/02 97.01 85.2 124.4
2002/03 97.14 88.5 128.9
2003/04 97.24 85.9 126.8

Table 3

Laspeyres CWOI index, CIPS, adjusted for
survival, 30 day mortality rates. Average annual growth
rates 1998/9 to 2003/4 (% p.a.)

 Health Health effect
 effect and life
 expectancy

 Unadjusted Simple [q.sup.0]/[q.sup.*]=0.8
 survival if m<0.10,
 [q.sup.0]/q =0 otherwise

1998/9-1999/00 1.87 1.27 0.78 1.12
1999/00-2000/1 0.91 1.16 1.58 1.37
2000/1-2001/2 0.95 0.89 0.91 0.76
2001/2-2002/3 4.44 5.37 6.59 6.31
2002/3-2003/4 5.81 6.37 7.15 7.13

Average all years 2.78 2.99 3.36 3.30

Table 4. Laspeyres CWOI index, CIPS, adjustments for
changes in waiting times. Average annual growth rates
1998/9 to 2003/4 (% p.a.)

 [r.sub.W] =
Form of waiting Measure of [r.sub.L] = [r.sub.W] = 10%,
time adjustment waiting time 1.5% [r.sub.L] = 1.5%

Discount to date
 treated, with
 charge for waiting 80th percentile 3.33 3.34
Discount to date
 treated, with
 charge for waiting mean wait 3.32 3.32
Discount to date
 on list 80th percentile 3.24
Discount to date
 treated, with
 charge for waiting individual data 3.48 3.49
Survival adjustment
 only 3.30

Note: [r.sub.W] = discount rate on waits, [r.sub.L] = discount rate on
life expectancy.

Table 5. Average period input shares and average growth
in inputs, 1998/9-2003/4, hospital sector.

Total NHS Shares Input growth

Labour 0.72 4.34
Intermediate 0.20 4.93
Capital 0.08 4.76

Table 6. Total factor productivity growth, hospitals, per
cent per annum

 Unadjusted Quality adjusted

1998/9-1999/00 -2.82 -3.53
1999/00-2000/1 0.30 0.56
2000/1-2001/2 0.17 0.01
2001/2-2002/3 -2.01 -0.95
2002/3-2003/4 -1.13 -0.39

Average -1.11 -0.87