The effect of pathways to work on labour market outcomes.

Dorsett, Richard

This article summarises the recent evaluation of Pathways to Work--a package of reforms aimed at encouraging employment among people claiming incapacity benefits (Bewley et al., 2007). The results show an increase of over 7 percentage points in the probability of working about a year and a half after the start of claim. The probability of not claiming incapacity benefits was also increased initially but this effect later reduced to just 1 1/2 percentage points. The findings suggest Pathways to Work may contribute to the Government's aim of increasing the employment rate. They provide less encouragement with regard to the aim of reducing by 1 million the number claiming incapacity benefits.

Keywords: Pathways to Work; evaluation; difference-in-differences; employment; sickness and disability

JEL Classifications: H43; J14; J64; J68

I. Introduction

The Pathways to Work package of reforms is aimed at encouraging employment among people claiming incapacity benefits. (1) Based on proposals outlined in the 2002 Department for Work and Pensions (DWP) Green Paper 'Pathways to Work: helping people into employment', these reforms were introduced on a pilot basis in three areas of Britain in October 2003. Four further areas became part of the pilot in April 2004. Since then, it has been extended so that, by December 2006, 40 per cent of the country was covered. By April 2008, all new or repeat incapacity benefits claimants in Britain will be eligible.

Pathways to Work represents an important development in labour market policy. In common with many developed countries, Britain has seen a large and sustained increase in the number of individuals claiming incapacity benefits. At the time of the 2002 Green Paper, there were roughly 2.7 million claimants; more than the combined total number of people claiming benefits on the grounds of unemployment or lone parent status. The overwhelming majority of people starting an incapacity benefits claim expect to work again (Woodward et al., 2003) and in fact many do--in 2004, almost 60 per cent left benefit within a year. However, for those who remain on benefit beyond this point, the chances of leaving declines markedly--29 per cent will still be claiming after another eight years (see the 2002 Green Paper for further details). A key aim of Pathways to Work is to intervene early, to reduce the incidence of prolonged benefit dependency.

In attempting to raise employment rates among those claiming incapacity benefits, Pathways to Work increases the conditionality of benefit receipt. For some time, all individuals have been required to attend a work-focused interview (WFI) shortly after commencing a claim for incapacity benefits. Under Pathways to Work, most people remaining on incapacity benefits beyond this point are obliged to attend five further WFIs. Failure to comply with this requirement can result in benefits sanctions. There are two groups of people for whom this is voluntary rather than mandatory. First, there are those identified through the Personal Capability Assessment (PCA) as having particularly severe medical conditions. Second, there are those assessed on the basis of a screening questionnaire administered during the first WFI as being likely to return to work within a year without any additional support. The screening process was designed such that the 30 per cent of individuals with the highest predicted probability of a successful unassisted return to employment were not required to attend any WFIs beyond the first.

Under Pathways to Work, the first WFI takes place eight weeks after the start of the incapacity benefits claim, rather than as soon as possible, in order to allow time for any claim processing issues and problems to be identified, possibly for health conditions to stabilise, and for short-term claims quickly to flow off. This first WFI is designed to have a different content from subsequent WFIs in that it involves collecting information about the individual (including the information needed for the screening decision outlined above), providing reassurance that the individual will not be forced to enter employment and introducing the range of help available under Pathways to Work. Subsequent WFIs are intended to reflect the particular circumstances and requirements of individual claimants. In the course of these WFIs, new information or additional detail on potentially helpful activities may be provided and progress against action plans may be discussed.

A key aim of these WFIs is to encourage individuals to participate in the universally voluntary elements of Pathways to Work. Under the 'Choices' package, a range of new and existing programmes are offered. The most important in terms of the number of participants are the New Deal for Disabled People (NDDP) and the Condition Management Programme (CMP). The NDDP has operated nationally since 2001 through a network of Job Brokers drawn from the public, private and voluntary sector who aim to support individuals make the transition into sustained employment. The CMP is a new programme run in collaboration with local health providers and has the aim of helping individuals to manage their disability or health condition. Another innovation of Pathways to Work is the Return to Work Credit which offers customers who find work of at least 16 hours a week a payment of 40 [pounds sterling] a week for a year, so long as their gross annual earnings are below 15,000 [pounds sterling].

Precise detail on how Pathways to Work operates in practice (for example, the average number of WFIs individuals have, the proportion participating in Choices, receiving RTWC etc.) will be available shortly (Bailey et al., forthcoming). In the meantime, administrative data available from the DWP (2) gives some indication of the scale of the intervention. By September 2006, more than 315,000 individuals had made initial contact with Jobcentre Plus with a view to claiming Incapacity Benefit and so qualified as a Pathways to Work 'start'. An identifiable benefit claim resulted in 200,000 cases (some initial enquiries were not pursued or resulted in a benefit of too short a duration to be captured in the data) and of these a number ended before their scheduled WFI. In all, by September 2006, there had been 103,000 initial WFIs and 83,000 repeat WFIs. Roughly 15,500 had entered NDDP and 10,500 had participated in CMP. Of the 20,000 recorded as starting work, 19,500 were awarded RTWC.

This article summarises the evaluation of Pathways to Work presented in Bewley et al. (2007). It is the overall impact of Pathways to Work in the pilot areas that is considered rather than the individual effect of a specific component such as NDDP or CMP; such estimates will be the focus of subsequent research. It should be noted that the overall impact analysis is one element of the broader programme of evaluation being carried out by a consortium of research organisations led by the Policy Studies Institute. (3) The evaluation involves qualitative analyses, large-scale quantitative surveys, further impact analyses, cost-benefit analyses and a literature review of relevant programmes in the USA.

2. Methodological approach

The effect of Pathways to Work was estimated using a difference-in-differences approach. This is a standard evaluation technique (see, for example, Heckman, Lalonde and Smith, 1999) so its properties are not discussed in detail here. However, the broad approach is to compare the labour market outcomes for individuals starting new claims in the pilot areas before Pathways to Work with outcomes for individuals starting new claims in non-pilot areas at the same time and to see how this relationship changes after Pathways to Work was implemented. The intuition behind this is that, in the absence of Pathways to Work, one would expect the relationship between pilot and non-pilot areas to remain constant. Any change is attributed to the effect of Pathways to Work.

This approach requires that, in the absence of Pathways to Work, the relationship between outcomes in the pilot and non-pilot areas would remain stable over time. We can investigate the extent to which this stability has existed prior to the introduction of Pathways to Work by carrying out pre-programme tests (Heckman and Hotz, 1989). These amount to difference-in-differences estimates based on two periods of time before Pathways to Work was introduced. Should such estimates be statistically significant, this casts doubt on the relationship between the pilot and non-pilot areas being stable. Bewley et al. (2007) present the results of such tests. These suggest that difference-in-differences estimates of the effect of Pathways to Work for the first set of pilot areas (those that commenced in October 2003) may be biased when estimated using the time periods for which the survey samples are available. For the second set of pilot areas (those that commenced in April 2004), the test results were more encouraging and did not suggest any problems when considering the effect on longer-term outcomes, although some of the shorter-term outcomes may be more questionable.

In line with the guidance provided by these tests, Bewley et al. (2007) focused on the April 2004 pilot areas--Essex; Gateshead and South Tyneside; East Lancashire; and Somerset--and this approach is also followed here.

The analysis was performed using regression so that it was possible to control for the effects of observed characteristics on outcomes. It also made use of both survey and administrative data. The advantage of working with survey data is that, since they were collected for the express purpose of understanding the impact of Pathways to Work, they provide information on a rich set of outcomes and a comprehensive range of factors likely to influence the effectiveness of the programme. With administrative data, fewer background and outcome variables are available. However, administrative data offer the important advantage of maximising the number of cases available for analysis. Furthermore, using administrative data avoids the possibility of nonresponse and recall biases that may affect survey data. (4)

The survey sample was drawn from the Business Information Service (BIS) database; a management information reporting system that can be used to identify those people getting in touch with Jobcentre Plus Contact Centres in order to claim incapacity benefits. The 'before-Pathways' sample was made up of individuals enquiring about claiming incapacity benefits in the period January-March 2004. The 'after-Pathways' sample was made up of individuals enquiring about claiming incapacity benefits in the period August-November 2004. Sample members were drawn from both the pilot areas and a set of non-pilot areas chosen to resemble the pilot areas on the basis of the 2001 area classification for local authorities carried out by the Office for National Statistics. (5) All individuals were surveyed near the time that they made the enquiry and their outcomes were observed through a further interview about a year and a half later. In the course of the outcome interviews, respondents provided retrospective information on all changes to their employment status since the time of their initial interview.

The number of sampled individuals for whom outcome interviews were achieved is shown in table 1. While a reasonable number of interviews was achieved in the April 2004 pilot areas, the number achieved in their comparison areas (that is, the non-pilot areas that were selected to provide a comparison group in the difference-in-differences analysis) was considerably smaller. A practical implication of this is that the subsequent estimates are less precise than would be the case had more interviews been achieved with those in the non-pilot areas, particularly in the 'before-Pathways' sample. In other words, it becomes more difficult to detect statistically significant effects. In recognition of this, results in the analysis that follows are regarded as significant if they have a p-value of 10 per cent or less, rather than the more standard p-value of 5 per cent, when based on the survey data.

The administrative data were taken from the National Benefits Database (NBD). This is constructed from a series of snapshots of live social security benefits data and so captures the full population of claimants at the time of those snapshots, back as far as June 1999. (6) It can be seen from table 1 that the administrative data offer much larger sample sizes than the survey data. (7) It should be noted that there is a qualitative difference between individuals in the survey sample and those in the NBD. Whereas the survey sample is made up of individuals making an initial enquiry about claiming incapacity benefits, some of whom may not go on to claim, the NBD includes only those who did make a claim.

3. Results

This section contains estimates of the effect of Pathways to Work on a range of outcomes using both survey and administrative data. Before presenting the results themselves, it is worth clarifying the nature of the estimated effects and how these differ according to whether they are based on the survey or the administrative data. As already mentioned, the survey data captured individuals making an enquiry about claiming incapacity benefits. Accordingly, estimates based on the survey data do not distinguish between those individuals who went on to make a successful claim and those who did not. In this regard, the results allow for the possibility that one effect of Pathways to Work may be to change the probability of individuals proceeding to the stage of making a full claim. Estimates based on the administrative data, on the other hand, relate to the population of successful claimants and so may not be expected to correspond with estimates based on the survey data. No distinction is drawn between those expected to participate fully in the mandatory components of Pathways to Work and those for whom participation was voluntary due to being PCA-exempt or screened-out at the first WFI.

The effect on employment

Questions included in the survey allowed the employment status of respondents to be observed from the point of first interview onwards. The estimated effect on employment over this period is shown in figure 1. (8) This has two panels. The upper panel presents the impact estimates--the estimated percentage point increase in employment due to Pathways to Work--together with its confidence intervals (shown by dotted lines). The lower panel plots actual employment over time (shown by the solid line) together with an estimate of the counterfactual employment (labelled 'Base'). This is shown by the dashed line and provides an estimate of what the month-on-month level of employment would have been had Pathways to Work not been introduced.

The results suggest that the effect of Pathways to Work evolved gradually and did not approach statistical significance for the first year after the initial enquiry. However, the effect appears to have grown such that eighteen months after the initial enquiry a positive effect of about 7 percentage points was evident. By the time of the outcome interview, the effect of Pathways to Work was estimated to be 7.4 percentage points with a p-value of 9 per cent. Moreover, this effect appears to be relatively stable over the few months leading up to month eighteen. Without Pathways to Work, it is estimated that employment over time would have increased from 18 per cent in month five to 28 per cent in month eighteen. This allows for the possibility that individuals may be employed in one month but not in a later month. The effect of Pathways to Work appears quite substantial in this context, raising the probability of employment in month eighteen from roughly 28 to 35 per cent. However, other results not presented here showed that this employment effect was not accompanied by an increase in earnings from employment. (9)

[FIGURE 1 OMITTED]

Incapacity benefits receipt

The estimated effect of Pathways to Work on incapacity benefits receipt at the time of the survey interview--roughly a year and a half after the initial claim enquiry--is given in table 2. This shows a reduction of 1.7 percentage points by this time but this is not statistically significantly different from zero. Without Pathways to Work, it was estimated that the level of claiming among the survey population in the April 2004 pilot areas would have been 51 per cent.

It is possible that a statistically significant effect was evident at an earlier point. This cannot be examined using the survey data since benefit history from the time of first interview onwards was not collected. However, administrative data can be used for this purpose. Figure 2 shows the impact of Pathways to Work on the probability of not claiming incapacity benefits in each of the eighteen months following the start of the incapacity benefits claim, based on administrative data. The results suggest Pathways to Work increased the chances of being a non-claimant of incapacity benefits but that this effect was greatest quite soon after the start of the claim and declined in size thereafter. After peaking at 6.3 percentage points five months after the start of the claim, the effect reduced to a fairly stable level of about 1 1/2 to 2 percentage points after month ten. The size of this eventual effect and the estimate of what the level of incapacity benefits receipt would have been had Pathways to Work not been introduced (52 per cent) matches very closely the estimate based on survey data (table 2). This effect was statistically significant for the period between two and fourteen months following the start of the claim and narrowly failed to achieve statistical significance at conventional levels from month fifteen onwards. The fact that the effect estimated using survey data was not statistically significant is likely to be due simply to the smaller number of observations available in the survey data reducing the statistical power of the analysis.

[FIGURE 2 OMITTED]

The effect on employment and benefits combined

These results present something of a puzzle in that Pathways to Work appears to have a long-term positive effect on employment but was not found, using survey data, to reduce significantly receipt of incapacity benefits a year and a half after the initial enquiry. (10) To explore this further, table 3 considers four possible combinations of employment status and incapacity benefits receipt at the time of the survey (11) interview:

* in work, not receiving incapacity benefits

* in work, receiving incapacity benefits

* not in work, not receiving incapacity benefits

* not in work, receiving incapacity benefits.

Since these four categories capture all possible combinations of employment and incapacity benefits status, the four estimated effects must sum to zero. In other words, should Pathways to Work increase the share of the population accounted for by one combination, this must be offset by a similar-sized reduction in the share accounted for by the other three categories. With this in mind, we can see from the results in the first row that Pathways to Work significantly increased the probability of working and not receiving incapacity benefits at the time of the outcome interview by 8.7 percentage points from a base of nearly 25 per cent. The main decrease offsetting this was the probability of being out of work and not receiving incapacity benefits (third row). This fell by a statistically significant 6.9 percentage points, from a base of 24 per cent.

So despite there being little evidence of an effect on the probability of receiving incapacity benefits, Pathways to Work does appear to increase the likelihood of employment among those not receiving incapacity benefits at the time of the outcome interview. In other words, those not claiming incapacity benefits are more likely to be employed as a result of Pathways than would otherwise be the case. Note that this result relates to a period about a year and a half after the start of the claim and is not informative about effects in the intervening months. The year and a half result may have come about through a variety of routes, for example:

* Pathways to Work could have increased the probability of moving from incapacity benefits directly to employment among those who would have left incapacity benefits anyway.

* Pathways to Work could have increased the probability of job retention among those who would have left incapacity benefits and found work anyway.

* Pathways to Work could have increased the probability of entering employment at a later date among those leaving incapacity benefits for a reason other than employment.

Health

Since part of the motivation for Pathways to Work is that work can have beneficial health effects, it is of direct interest to examine the evidence that it has had an effect on health. It does not immediately follow that there should be such an effect. For example, Pathways to Work may make individuals feel under increased pressure to move into work and accept unsuitable employment with consequent negative effects on health.

Two health outcomes are considered. The first is whether individuals report having a health condition or disability which affects their everyday activities. The second is whether individuals report having a health condition or disability which affects their everyday activities "a great deal". (12) It is important to understand what these outcomes actually capture. The first depends on three factors: the respondents' health, the nature of their everyday activities and how they report this to the survey interviewer. The second depends on these same factors but also the degree of limitation on everyday activities and, again, how this is reported to the survey interviewer.

A complication with dealing with self-reported health outcomes is that individuals' responses may be influenced by their employment status. While it is wholly possible that employment has consequences for health, it is also possible that being employed changes how individuals report their health for reasons unconnected with any real change in underlying health. One possibility is that individuals not in work report poor health as a justification for not being employed, perhaps for reasons of self-esteem. If this were the case then an increase in employment might be accompanied by an improvement in self-reported health regardless of whether there was a change in actual health. Another possibility is that a move into work involves a change in an individual's everyday activities. This might increase the likelihood that the individual becomes aware of their health problem limiting their activities. In this case, it could be the change in activities that affects the individual's assessment rather than a change in underlying health, which is the outcome of interest.

With these caveats in mind, table 4 presents effects for the two health outcomes mentioned above. In the first row, the effect of Pathways to Work on the probability of individuals reporting that, at the time of their survey interview, they had a health condition or disability which affects their everyday activities is presented. From this, it appears that Pathways to Work did not have a statistically significant effect on whether individuals reported such a health problem or disability. In the second row, the degree of limitation arising from the health problem or disability is considered. This shows that the estimated effect of Pathways to Work is to reduce significantly the probability that their health condition or disability limited their ability to carry out day-to-day activities "a great deal" by nearly 11 percentage points. In the absence of Pathways to Work, nearly half the sample would report a health condition or disability that limits their ability to carry out day-today activities "a great deal". So it seems that Pathways to Work does not reduce the incidence of self-reported health conditions or disabilities but does reduce the probability of individuals reporting that their health condition or disability substantially limits their ability to go about their everyday lives.

Variation across subgroups

Table 5 shows how the estimated effect on employment at the time of the outcome interview varies across subgroups within the population. The subgroups considered are those defined on the basis of sex, age, the nature of the health problem and the presence of dependent children in the household. The estimates presented for a particular subgroup are based on only those individuals within that subgroup. This has two consequences that are relevant for the interpretation of the results. First, it means that the subgroup estimates are based on a smaller number of observations than when considering the April 2004 areas as a whole. Consequently, effects are less precisely estimated than when considering the full population so that it becomes more difficult to detect a statistically significant effect, should one exist. In view of this, the lack of statistically significant effects should not necessarily be taken to mean that Pathways to Work has no effect for a particular subgroup; rather, it should be viewed as indicating that Pathways to Work does not have a sufficiently large effect for it to be captured statistically. (13) In view of this, the approach followed is to focus mainly on the question of which groups of individuals appear to be affected more (or less) by Pathways to Work, rather than interpreting the actual size of the effects too closely.

The second consequence of estimating within-subgroup effects is that comparisons of the two resulting estimates do not control for differences in composition between the subgroups. For example, a comparison of the estimated effect for men with the estimated effect for women takes no account of the possibility of differences between men and women in other characteristics that might influence effects. The implication of this is that, while the results can show that Pathways to Work had a greater effect on some subgroups than others, this difference is not directly attributable to the characteristic that identifies the subgroup. In terms of the example above, should the effect of Pathways to Work be greater for women than for men, we cannot infer from this that being female in itself increases the likely effect of Pathways to Work; rather, it is the case that the combined characteristics of women pre-dispose them to being affected more by Pathways to Work than men, with their set of characteristics.

With these caveats in mind, table 5 presents the estimated effects of Pathways to Work on employment by subgroup. (14) By the time of the final interview, marked differences between men and women were apparent. No statistically significant effect was found for men but, for women, Pathways to Work had a significant positive effect on the probability of being in work. The positive employment impact also appears to be largely concentrated among the younger age group. In particular, Pathways to Work significantly increased the probability of being employed among those under the age of 50 but no significant effect was found for those aged 50 or more. The extent to which the estimated impact varied according to the nature of the individual's health problem was also explored. The survey questionnaire asked respondents to identify the type of health problems or disabilities associated with their main condition. For those individuals who reported a health problem at the time of the first survey interview, we can distinguish between those whose main condition involved a mental illness and those whose main condition was not of this type. Dividing the population along these lines, it seems that Pathways to Work had no significant effect on employment at the time of the final interview for those respondents reporting a mental illness. Significant positive effects were detected for those whose main condition did not involve a mental illness. Lastly, table 5 presents estimated impacts for those with and without dependent children. A significant employment effect was evident for those with dependent children while no significant effect was evident for those without dependent children.

Summary and conclusion

The evaluation results suggest that Pathways to Work significantly increased the probability of being employed at the time of the survey interview--about a year and a half after the original incapacity benefits enquiry--by 7.4 percentage points. Without Pathways to Work, it is estimated that 29.7 per cent of individuals would have been in work at this time. This employment effect was quite stable over the final six or so months observable. While no effect on earnings was detected, this is perhaps unsurprising given the small sample size on which the estimates are based and the fact that earnings can only be observed for the minority of individuals who are in work.

The effect on incapacity benefits receipt about a year and a half after the initial enquiry about claiming incapacity benefits was small and not statistically significant. This finding (which was based on survey data) was confirmed using administrative data on those who went on to make a claim for incapacity benefits. These data allow more precise estimates due to the larger number of observations available for analysis and suggested a marginally significant reduction of 1 1/2 percentage points in the probability of claiming incapacity benefits a year and a half after the start of claim, from a base of 52 per cent. This is very similar in size to the estimate based on survey data--a reduction of 1.7 percentage points from a base of 51 per cent. Using the administrative data allows the effect on incapacity benefits receipt to be estimated for each month following the start of the incapacity benefits claim. This revealed a sizeable effect on incapacity benefits receipt--reducing it by a maximum of 6.3 percentage points five months after the start of the claim. However, this effect eroded with time. The seemingly stable long-term effect of 1 1/2 to 2 percentage points was reached in month ten.

The Government has set itself the target of an 80 per cent employment rate. Furthermore, the Welfare Reform Green Paper that was published at the start of 2006 (15) sets out the aim of reducing by 1 million the number on incapacity benefits. The results discussed in this paper have a clear bearing on these targets. With regard to employment, the results are encouraging. They suggest a positive employment effect that is sustained in the medium term, particularly for women, those aged under 50, those whose main health condition was not a mental illness and those with dependent children. It should be noted that the type of employment encouraged by Pathways to Work will not always be full-time. Those claiming incapacity benefits may face particular constraints on the amount of work they are able to do. Some health conditions or disabilities may limit individuals' ability to work more than a small number of hours per week. Other individuals will have caring responsibilities which similarly prevent them from working more than a small number of hours per week. Another factor to consider is the Permitted Work rules. These allow incapacity benefits claimants to be employed in a job paying up to 20 [pounds sterling] a week. They can also work for less than 16 hours a week (on average) for up to a year so long as earnings do not exceed 86 [pounds sterling] a week. (16) This employment can last beyond one year without affecting benefit eligibility so long as an individual is receiving support in employment from a recognised provider or is assessed as having a condition or disability that meets the criteria for exemption from the personal capability assessment. The rules aim to help claimants to try working while continuing to receive benefits, with a view to their eventual movement into more substantial employment. Under Pathways to Work, advisers promoted permitted work for incapacity benefits claimants. This may offer a partial explanation for the finding that Pathways to Work increased employment but not incapacity benefits receipt at the time of the final interview.

With regard to the target of reducing the number of incapacity benefits claimants by 1 million, the results are less encouraging. Pathways to Work reduced the probability of claiming incapacity benefits in the first six months following the start of the claim but this effect slowly declined thereafter to a sustained level of about 1-1 1/2 percentage points. There are two scenarios that can explain this decline in the effect. The first is that those individuals who exited incapacity benefits because of Pathways to Work subsequently returned. This might be expected to be the case if, for example, Pathways to Work increased the willingness of individuals to try working in the knowledge that they could return to claiming incapacity benefits if it turned out not to be suitable. The second scenario is one where the declining effect of Pathways to Work is caused by the counterfactual 'catching up'. It may be that the long-term position among those eligible for Pathways to Work would have been reached in any case but that Pathways to Work accelerated the movement away from incapacity benefits. In other words, Pathways to Work may have caused people to leave incapacity benefits earlier than they otherwise would have done but, over time, the counterfactual level of incapacity benefits exits grew to close the gap.

It is not possible to distinguish between these two alternative explanations and it may indeed be the case that both play a role in explaining the pattern observed. However, the fact that Pathways to Work has an effect on the probability of employment that persists beyond the point at which the effect on benefits has largely disappeared provides support for the belief that Pathways to Work does not merely serve to bring about a situation that would have arisen regardless but that it alters the nature of exits from benefit, and subsequent changes in labour market status, in a meaningful way such that a higher proportion of those no longer receiving incapacity benefits are in work.

It is also perhaps consistent with the structure of Pathways to Work that exits from incapacity benefits should be concentrated in the first six months or so of the claim starting. One factor that may contribute to this is the accelerated PCA process under Pathways to Work. Since the results of the PCA should be available much sooner than previously, those judged by the PCA not to qualify for incapacity benefits will leave incapacity benefits more swiftly than was previously the case. Another potential explanation may be that it is in the first six months that most work-focused interviews take place and that perhaps it is the intense, face-to-face nature of early Pathways to Work engagement that is most important in influencing labour market outcomes. Since only about a fifth of those having an initial WFI go on to participate in Choices, for the majority of claimants it is the WFIs that constitute the main element of the Pathways to Work process. This being the case, one might expect most impacts to occur while individuals are still attending WFIs.

The final outcome considered was the self-reported health of the survey respondents. Pathways to Work was not found to have a statistically significant effect on the probability of individuals stating that they had a health condition or disability that limited their ability to carry out their everyday activities. However, it did significantly reduce the probability of respondents reporting that they had a health condition or disability that limited their ability to carry out their everyday activities "a great deal" by 10.8 percentage points from a base of 49.8 per cent. One should treat these findings on self-reported health with some caution since it is possible that this outcome is partly influenced by changes in how individuals report their health due to a change in their activities on moving into work rather than changes in the nature of their medical condition. However, the reduction in the probability of reporting a substantial health-related limitation on everyday activities, apparently due to the effect of Pathways to Work, is larger and more significant than the employment effect. This suggests that the results may be capturing a genuine effect of Pathways on the extent to which everyday activities are limited. Such an effect is an important finding and entirely consistent with the design and aim of Pathways to Work. Specifically, part of the rationale for Pathways to Work was that work can have positive medical benefits. In terms of provision, the Condition Management Programme aims to help individuals better understand and manage their health condition.

Finally, it should be noted that while this paper summarises the findings for the overall effect of Pathways to Work on new claimants, other analyses underway as part of the broader evaluation programme will add to these results in important ways. These include investigations of the effect of particular components of the Pathways to Work package; consideration of how the effects may generalise to areas where Pathways to Work does not yet operate; an assessment of the extent to which Pathways to Work may have indirect or 'spillover' effects on other people; and, an evaluation of the effect of the extension of Pathways to Work to those individuals who were already claiming incapacity benefits at the time Pathways to Work was introduced for new claimants. The net benefit of the programme is being rigorously examined through a detailed cost-benefit analysis. In addition, a separate evaluation of the expansion of Pathways to Work to new areas is also underway.

REFERENCES

Bailey, R., Hales, J., Hayllar, O. and Wood, M. (forthcoming), Incapacity Benefit Reforms Pilot: Findings from Wave I of the Survey of Customers, DWP Research Report.

Bewley, H., Dorsett, R. and Haile, G. (2007), The Impact of Pathways to Work, DWP Research Report 435.

Heckman, J. J. and Hotz, V. J. (1989), 'Choosing among alternative nonexperimental methods for estimating the impact of social programs: the case of manpower training', Journal of the American Statistical Association, 84, 408, pp. 862-74.

Heckman, J., LaLonde, R. and Smith, J. (1999), 'The economics and econometrics of active labor market programs', in Ashenfelter, O. and Card, D. (eds), Handbook of Labor Economics, Vol. IV, pp. 1865-2073.

Woodward, A., Kazimirski, A., Shaw, A. and Pires, C. (2003), New Deal for Disabled People. Evaluation. Eligible population survey. Wave one. Interim report, DWP Research Report W170.

NOTES

(1) That is, people claiming Incapacity Benefit, or income Support on the grounds of disability.

(2) See http://www.dwp.gov.uk/asd/workingage/pathways2work/pathways_perf_0507.pdf (web link valid at time of publication). Note that the figures for new claimants are across all pilot areas rather than the April 2004 areas on which this evaluation primarily focuses.

(3) The other members of the consortium are the Institute for Fiscal Studies, Mathematica Policy Research, the National Centre for Social Research, the Social Policy Research Unit and David Greenberg of the University of Maryland.

(4) The control variables included in the survey data-based estimates include sex; age; whether there were dependent children in the household; ethnicity; age at which they left formal schooling; partner status; partner's employment status; type of health problem and duration of health problem. The control variables included in the administrative data-based estimates include sex; age; type of health problem and benefits status in each of the eight quarters before the start of incapacity benefits claim.

(5) http://www.statistics.gov.uk/about/methodology_by_theme/ area_classification/la/methodology.asp

(6) A small proportion of IB spells that end within six weeks may fall entirely between consecutive snapshots and so not be captured in the administrative data.

(7) Note that the definition of pilot and comparison areas is consistent across survey and administrative data-based analyses. The timing of the 'post-Pathways' sample is also equivalent in both cases. However, the 'pre-Pathways' sample used in the administrative data-based analysis is a year before the 'pre-Pathways' sample used with the survey data-based analysis. The choice of this earlier 'pre-Pathways' sample with the administrative data-based analysis is guided by the results of pre-programme tests which suggested that using the later 'pre-Pathways' sample may result in bias (Bewley et al., 2007). For the survey data, it is not possible to carry out pre-programme tests so there is no similar guidance available.

(8) Delays between the initial enquiry and the first survey interview mean that there were too few observations available for reliable analysis prior to month four. Between month five and month eighteen the number of observations on which the estimates were based remained broadly stable.

(9) This is perhaps unsurprising given the small sample size on which the estimates are based and the fact that earnings can only be observed for the minority of individuals who are in work.

(10) As shown in the previous section, the small reduction seen in the survey data was of a similar size to the reduction estimated using administrative data which was marginally significant.

(11) In principle, we could have also combined administrative benefit records with administrative employment records to investigate the same issue. However, the evaluation did not include administrative employment records, whose use in evaluations within the UK is still not well-established.

(12) Survey respondents were asked whether their health condition or disability limited their ability to carry out their day-to-day activities "a great deal", "to some extent", "a little" or "not at all".

(13) This does not preclude the possibility that Pathways may have had no effect for a particular subgroup, of course.

(14) Subgroup analysis for other outcomes (benefit receipt and self-reported health) are reported in Bewley et al. (2007). Briefly, these suggest Pathways to Work caused a more sustained increase in the probability of not claiming incapacity benefits for men than for women and for those aged under 50 than for those aged 50 and over. With regard to the extent to which respondents' health condition affected their everyday activities, significant improvements were found for men, for those aged under 50, for those without a mental illness and for those without dependent children.

(15) http://www.dwp.gov.uk/welfarereform/docs/ A new deal for welfare-Empowering_people_to_workFull_Document.pdf

(16) An important point though is that, for those on Incapacity Benefit, such earnings have no effect on the amount of benefit paid, while for those claiming Income Support, any earnings over 20 [pounds sterling] are counted against benefit entitlement.

Richard Dorsett, Policy Studies institute. e-mail: [email protected]. This paper mostly summarises analysis reported in Bewley et al. (2007). This earlier research was funded by the Department of Work and Pensions. I thank Helen Bewley for helpful comments.

Table 1. Estimation sample sizes

 Number of individuals in each sample

 Survey data

 'before Pathways' 'after-Pathways'
 sample sample

Pilot areas 1,260 1,217
Non-pilot
(comparison) areas 157 659

 Number of individuals in each sample

 Administrative data

 'before-Pathways' 'after-Pathways'
 sample sample

Pilot areas 17,581 13,465
Non-pilot
(comparison) areas 13,460 10,331

Table 2. Estimates of the effects of Pathways to Work on
incapacity benefits receipt at time of outcome interview,
April 2004 pilot areas

 Impact P-value Base Sample
 estimate size

Receiving incapacity
benefits -1.7 72 51.1 3,212

Source: survey data.

Note: ** denotes statistical significance at the 5 per cent level;
* at the 10 per cent level.

Table 3. Estimates of the effects of Pathways to Work on
combined employment/incapacity benefits status at time
of outcome interview, April 2004 pilot areas

 Impact P-value Base Sample
 estimate size

In work, not receiving
 incapacity benefits 8.7 * 5 24.8 3,210
In work, receiving
 incapacity benefits -1.6 18 4.6 3,210
Not in work, not
 receiving incapacity
 benefits -6.9 * 8 24.0 3,210
Not in work, receiving
 incapacity benefits -0.2 96 46.6 3,210

Source: survey data.

Note: ** denotes statistical significance at the 5 per cent level;
* at the 10 per cent level.

Table 4. Estimates of the effects of Pathways to Work on
self-reported health at time of outcome interview, April
2004 pilot areas

 Impact P-value Base Sample
 estimate size

Health problem affects
 day-to-day activity -4.0 27 86.1 3,177
Health problem affects
 day-to-day activity
 "a great deal" -10.8 ** 2 49.8 3,124

Source: survey data.

Note: ** denotes statistical significance at the 5 per cent level;
* at the 10 per cent level.

Table 5. Variations across subgroups in the effect of
Pathways to Work on employment at the time of outcome
interview, April 2004 pilot areas

 Impact P-value Base Sample
 estimate size

By sex
Men 3.0 62 35.8 1,786
Women 13.0 ** 5 22.2 1,505

By age
Under 50 10.6 * 6 31.3 2,101
50 and over 2.3 75 26.7 1,190

By nature of health
 problem
Not mental health
 problem 10.7 * 6 23.1 1,985
Mental health problem -1.1 90 29.4 700

By dependent children
No dependent children 3.5 49 31.2 2,416
Dependent children 17.6 ** 4 27.1 875

Source: survey data.

Note: ** denotes statistical significance at the 5 per cent level;
* at the 10 per cent level.