High-stakes literacy tests and local effects in a rural school.

Cormack, Phillip ; Comber, Barbara

Introduction

I would welcome more skilling of myself to be able to unpack the mandated tests. I'm happy to administer them because I think they can give some direction to programming, but whether it's good direction or token, or whatever, is dependent on how well we unpack it and drill down into it. I guess I'm crying out for some help in that area. (Principal, small rural school in high poverty region)

This statement, from an experienced primary school principal in a poor rural community, touches on the complexity of the work and emotions associated with mandated literacy testing. Like most principals, he wants to do the right thing and willingly gets on with 'administering' the tests. He hopes that the results might 'give some direction', however he admits to being unsure about how he should interpret the results in order to decide on a 'good direction'. There's a lot going on here and we can hear his anxiety in the request for help. The principal was one of a number of school leaders from Victoria and South Australia who participated, along with their teachers, in our research project designed to investigate the reorganisation of educators' work in the wake of the implementation of National Assessment Program: Literacy and Numeracy (NAPLAN). As did other principals in our study, he grapples daily with what it means to be working in a site which has been assessed a 'failing school' on the basis of a high-stakes test.

The project's design and analysis was informed by the Institutional Ethnography of Dorothy Smith (1987; 2005; 2006). The lived experiences of school-based educators are typically absent from policy discussions and broader political, media and educational debates concerning standardised testing. The work--emotional, everyday and pragmatic--is often invisible (Blackmore, 2004; Nichols & Griffith, 2009; Smith, 2005). Smith (2005), however, has developed an approach to researching how the practices of everyday life are organised and how those practices are coordinated beyond the local level, often by texts which come to rule how people go about their work. Yet it is people working and living in particular places who activate these texts in specific ways, eliciting chains of related actions with particular consequences for particular people. Following Smith, we argue that it is important to hear from front-line workers--educators in schools, especially those who are working in challenging circumstances --about what mandated testing actually does to everyday practices. Hence our research endeavours to trace in the talk of educators the activation of a range of practices associated with high-stakes testing. In the case of the rural school considered here, we discuss how contemporary policy is enacted in situ though staff's reported experiences of a school review, subsequent to the students' poor performance on literacy tests, and their efforts to improve student test performance through the ways they purchase resources, group their students, spend their time, prioritise aspects of literacy and so on.

In terms of our ongoing program of research concerning the relationships between literacy education and social justice, we believe it is increasingly urgent to consider the material effects of high-stakes tests in particular places. It is important in doing this to build on the work of others who have explored the compounded educational disadvantage often experienced by young people growing up in rural poverty (Green & Letts, 2007) and the challenges faced by their teachers (Somerville, Plunkett & Dyson, 2010). At the same time, we recognise that the principal's positioning as a school leader has altered with devolution to school-based management, performative cultures, concern with risk management, intensification of work, the emergence of new accountabilities, and the proliferation of educational policy and associated legal and financial reporting demands, and there has been useful work on these effects in policy and practice (Ball, 2001; Blackmore, 2004; Lingard, 2010; McWilliam & Singh, 2004; Thomson, 2009). In what follows, we begin by examining the claims and assumptions upon which high-stakes testing is built, before moving on to examine the impact of these policies on one rural school.

NAPLAN, My School and ICSEA: High stakes for principals and teachers

For several decades in Australia, literacy has been a focus of government policy and now with the advent of the My School website's (myschool.edu.au) annual reporting of school performance data, the standardised measurement of literacy achievement has become high stakes. The centrepiece of the My School website and associated public discussions is the data generated through the standardised tests known as NAPLAN conducted annually on Year 3, 5, 7 and 9 students in all Australian schools. The reforms with which NAPLAN is associated operate within what can be labelled a 'discourse of data' (Poster et al., 1996) where numerical, standardised data have come to be the only information that counts when discussing the value of schools (Comber, 2012; Comber & Cormack, 2011; Kostogriz & Doecke, 2011). Within the proliferation of information and data about schools, cohorts of students, national and international performance, and so on, the narrow underlying logics of this discourse can be lost. These logics can be summarised as:

* standardised test results provide an accurate and reliable insight into students' literacy and numeracy practice and performance

* literacy performance or standards as measured by these tests are the key indicator of the 'value' that schools provide to students and, by proxy, to the students' future performance as workers and citizens--a logic Comber and Hill (2000) identified as 'literacisation', especially for schools serving low SES students

* if these data are provided in ways that allow managers and parents (as consumers who 'choose' schools) to compare the 'value' being added to their students by schools, underperforming schools can be identified and 'incentivised' to lift their performance, learning from those schools which are performing best, thus solving the problem of unequal outcomes

For these logics to work, as Lingard points out, schools need to be recast as part of a competitive educational 'market'.

A basic assumption is that competition between schools and parental pressures will push up standards and strengthen accountabilities. There is also at times a parental market choice discourse underpinning the policy (Lingard, 2010, p. 132).

The assumption that schools operate on a level playing field within a market can only be sustained if much of the actual work of teachers and principals, and of school context, is bracketed out. The emphasis in this discourse is on the teacher as the most important factor in student achievement (Berliner, 2013 forthcoming), eliding such issues as location, socio-economic status, race and linguistic background. What counts in determining the value of schooling becomes standardised data and the only solution for change within schools is teaching. Labaree (2011, p. 628) notes that statistics has come to dominate research about school effectiveness and determine what will count as trustworthy information such that significant issues that don't fit their frame receive virtually no attention--as he puts it, '[w]hen you are holding a hammer, everything looks like a nail'.

Significantly for the work of teachers and school leaders, the focus on data underpinned by competitive logics is that the market will provide its own corrective solutions to school underperformance. Consider the way that this logic works as it is expressed on the My School and NAPLAN websites. In a page titled 'Why NAP?' a 'key benefit' of NAPLAN is described as 'driving improvements'.

 All Australian schools benefit from the outcomes of
 national testing, with aggregated results made available
 though comprehensive reports at the national
 and school level, accessible on-line (see Test Results).
 Schools can gain detailed information about how they
 are performing, and they can identify strengths and
 weaknesses which may warrant further attention.

The focus here is on the identification of 'strengths and weaknesses' through comparison with other schools. As explained on the My School website, key in this process of comparison is the concept of 'like schools' where, through the application of the Index of Community Socio-Economic Advantage (ICSEA), schools are grouped into those serving 'statistically similar populations'.

 ICSEA enables student results on national tests to be
 understood and compared in a fair and meaningful
 way, and enables schools seeking to improve their
 performance to learn from other schools with statistically
 similar populations.

Here we see an assumption that comparison (underpinned by competition) will foster a bootstrapping process to occur whereby 'underperforming' schools learn from their better-performing peers. A virtuous cycle of continuous improvement is thus assumed, centred on what teachers do in their classrooms and principals in their schools, with difference in location, community resources, and so on, taken care of statistically. One of our key questions in working with teachers serving students in poverty and in different locations such as rural towns or poorly-resourced low socio-economic suburbs was how such logics worked, or didn't work, for them in making a difference for their students.

League tables are dangerous (Alexander, 2010) and can result in the stigmatisation of school communities especially in the face of a dominant common-sense rationality that if poor people try harder/ work harder they should prosper. High-stakes testing, combined with educational policies based on meritocracy (Teese, 2000; Teese & Polesel, 2003) or bootstrap mentalities (Berliner, 2013 forthcoming), make poor performance the problem of the poor and their teachers. If only the teachers taught the basics, if only the parents read them a bed-time story, if only the children worked harder, they would improve. These, 'if only' rationales ignore how poverty is actually produced. Poverty and the associated material realities of everyday life become statistical 'noise' that distracts from the (moral) imperative to develop better pedagogies, better programs, better interventions. By this account the tests identify the problem within the 'student body' and teachers are positioned as the solution, through attending to the insights in the data and applying the pedagogic remedy. Berliner (2013 forthcoming, p. 1), in the US context, shows these logics have failed to deliver on their promise and asks:

 What does it take to get politicians and the general
 public to abandon misleading ideas, such as 'Anyone
 who tries can pull themselves up by their bootstraps', or
 that 'Teachers are the most important factor in determining
 the achievement of our youth'?

Those who do manage to pull themselves up by their bootstraps and teachers who manage to turn around the learning lives of their students are the extraordinary cases and not the norm, hence policy should not be based upon such exceptional instances. Bluntly Berliner argues: 'Most children born into the lower social classes will not make it out of that class, even when exposed to heroic educators' (Berliner, 2013 forthcoming, p. 2). He concludes that educational reform can only achieve so much when there is great inequality of wealth in a society and that at least part of the solution to better learning outcomes involves different economic policies and practices--including fair wages, higher taxes to better resource schools (e.g. librarians, counsellors, nurses) and providing better infrastructure in poor areas (e.g. transportation, health care, policing). In short, there is a need in wealthy nations with high inequality to provide substantive and ongoing investments in education and services allied with basic human rights. Schools cannot solve the effects of poverty.

This argument about the need to interrogate taken-for-granted assumptions about school reform and literacy education in low socio-economic communities underlies our discussion of the local effects of global trends toward standardisation. We wish to complicate the logics described above by considering the reported experiences of educators in one rural school in a poor regional community. We begin by describing the school's location and population. We then turn to the principal's and teachers' reported experience of high-stakes testing in literacy. Finally we describe how students come to be understood in this context and the subsequent solutions designed for these 'problems' and how such practices are contingent upon the resources the school community can assemble.

School context

Wheatville (a pseudonym) is a small rural primary school with an enrolment of around 140 to 150 students, with some variation due to transience associated with mobile populations. The local community suffers a high level of poverty and the principal and teachers see the context as disadvantaged and challenging. According to the My School website Wheatville's ICSEA score was between the low 950s and high 940s from 2008 to 2010 (with 1,000 being the mean for Australian schools) (1). The My School website glossary claims that ICSEA is 'a scale that enables meaningful comparisons to be made across schools' because it identifies 'schools serving similar student populations'. It goes onto explain that:

The variables used in calculating a value on the ICSEA scale include student-level data on the occupation and education level of parents/carers, and/or socioeconomic characteristics of the areas where students live, whether a school is in a metropolitan, regional or remote area, proportion of students from a language background other than English, as well as the proportion of Indigenous students enrolled at the school. (http://www.acara.edu.au/myschool/myschool_glossary:.html)

A detailed fact sheet has been produced to elaborate on how the scale has been calculated. However since its inception, school principals have contested the extent to which the ICSEA scale really does identify similar populations. In the case of Wheatville the ICSEA score indicates that it is below the mean but not dramatically so. However, the 2006 ABS data (http://www.censusdata.abs.gov.au) indicated that for the town in which the school was located, the median weekly income was significantly lower than national and state means (Table 1).

At the time of the study, according to the state education department's Index of Disadvantage, Wheatville Primary was a Category 2 school, where there are 7 categories with 1 being the lowest. The department's Index of Disadvantage is calculated on measures of:

* parental economic resources

* parental education and occupation

* Aboriginality

* student mobility

As the principal and teachers reported to us, Wheatville experienced a great deal of mobility of population associated with casualised and seasonal labour, cheap housing and so forth. By the state education department's standards, Wheatville experienced a high level of disadvantage. Since the period in which the study was undertaken (2008-2010) some areas of the town have become gentrified with the arrival of 'sea-changers' which would account for the slightly higher incomes recorded in the 2011 census. However the points which we need to make here are concerned with the relationships between the school's literacy results on NAPLAN (and other standardised measures), the actual experience of poverty and its effects, the path to being declared a failing school and how this impacts on educators' work in a small rural community. The short story here is that Wheatville students and their families experienced a high level of relative poverty. The teachers recognised this, as we will discuss below. Two other features of the school need to be stressed at this point. First, there was a very small enrolment, typically there were only about 20 students (maximum) at any one level to be tested (i.e. in Years, 3, 5 & 7). Second, the school population was highly mobile, meaning it was quite likely that some students were recently arrived at the time of testing and rare for students to remain in the school for years 3, 5, 7. Small numbers and high mobility introduce statistical uncertainty, so whether the ICSEA can represent the school population in terms of comparability is questionable, as is the assumption that schools will be able to analyse their results in terms of 'value-added'. These are complex questions and more than we can address in this paper, however the point we wish to make is that school context is very important in terms of how educators experience their work and not sufficiently accounted for by ICSEA. Indeed, as Smith (2005, pp. 165-182) has shown, it is the work of abstract texts such as the ICSEA to strip out the local and everyday, in order to make practices comparable and transportable. However, by ignoring the local and the diversity of populations, they represent schools in ways that simply don't make sense to the participants who experience the daily/nightly effects of such things as student mobility, distance from services and regional poverty.

The principal's and teachers' experience of NAPLAN and being a failing school

To consider the way in which the language and practices of NAPLAN positioned the principal and teachers in Wheatville and the effects on their experience, we conducted an extended interview with the principal, Barry (a pseudonym), and a focus group discussion with the teachers who taught the grades where NAPLAN was used. Significant for all the participants was that the school was assessed as failing through a diagnostic review process initiated by the state education department as a result of poor performance in the first round of NAPLAN. Interestingly, the whole notion of a diagnostic review has been imported into schools from the world of business, promoting analysis by outside experts as an unproblematic process. While this review had impacts on everyone, these effects were not necessarily the same for the principal and the teachers.

The principal

Barry, as principal, saw that the diagnostic review and follow up ultimately provided him with the opportunity to learn 'how to lead literacy in the school'.

 [W]e had the opportunity to be part of the literacy
 and numeracy diagnostic review process that involves
 Central Office and District Office review, and peer
 principals and staff, going into classrooms, looking at
 the way teachers teach, looking at the documentation
 we have relating to literacy and numeracy, and looking
 at our school development plans.

He explained that, following the initial Central Office and District Office review, he was then invited to be part of a national project for principals.

 [O]ne of the things that process did was open a door
 for me to be part of the 'Principals as Literacy Leaders'
 project (emphasis added).

He expressed his strong enthusiasm for that project, describing it as 'nation-wide' and as 'something that I have really valued'. In the same vein, he mentioned the opportunities that he believes teachers need 'to take them out of their comfort zone'. We were struck in reading the transcript by the ways in which spatial metaphors featured in his talk, as is highlighted by terms italicised in the quotations from the transcript.

 My keenness was to try and get some outside influence,
 particularly with the Year 3-7 group, so it was just
 another opportunity for them to consider how they're
 going about things (emphasis added).

It is important to note that Barry was working as principal in a small rural school. Given his institutional locale, the spatial metaphors are telling. He did not have an assistant principal with whom he could consult or to whom he could delegate various tasks. Due to the difficulty of securing relieving teachers in his area, he was restricted in how much time he or his teachers could undertake professional learning away from the school. So his keenness for 'outside influence' for his middle and upper primary teachers and for himself as a principal from a 'nation-wide' project is understandable. The fact that his school was judged as failing before he was offered this opportunity is not stressed; rather, he ran with the affordances it provided and positioned himself as a principal who welcomed opportunities to meet educators from interstate. It is clear he also welcomed the chance to learn alongside his peers. He 'received' information he believed he should have had in teachers' college and subsequently developed confidence to question his teachers. Here we see the principal making the most of such a program, when he is repositioned as a literacy leader and given some assistance in learning practices for working with his teachers, such as observation and questioning.

However, his 'opening doors' and 'opportunities' rhetoric did not transfer into how he approached change to literacy pedagogy in the school and classroom. Here his language altered. Keywords included: 'focusing', 'zeroing', 'digging', and 'drilling'. The frame of reference was narrower, deeper, and closed rather than wider and open. This is not surprising. As Lingard (2010, p. 131) points out, 'a narrowed focus on literacy and numeracy', is what the then Minister for Education, and current Prime Minister, argued was required. The language of 'narrowing' and 'focusing' appears ubiquitous and not contestable.

 As a school we've zeroed in on reading this term,
 because we felt ... while some of our kids are below
 national benchmark in reading and writing, that
 reading, that's where we've got most of our resources,
 that's where we've got some key people on staff who are
 good resource people, and to go broadly literacy, it can
 sometimes water stuff down, so we just wanted to go
 narrower and a bit deeper with that (emphasis added).

Barry explained the need to use further testing such as the Neale Analysis for Years 4-7 students, so that teachers 'can dig a bit deeper and find out more' about students who may appear to read fluently but understand little of what they read. He wanted to get beyond what he described as 'throw away lines' from teachers. This relationship with data and the need to do something with it was of considerable interest to all the principals we have interviewed. 'Data' was a keyword; as it is across the field of education as part of an emerging 'discourse of data'. There was a sense in which principals needed to learn the mysteries of data interpretation and the implications for practice. As Smith (2006) and colleagues have pointed out, data allows for experts to 'know' about people's experiences in ways that bracket out their lived experience; their everyday knowing.

There is a range of contradictions and tensions in Barry's account of the impact of mandated literacy assessment upon his work as a school principal. In the interview Barry described surveillance and diagnosis of the school (from the 'office') and then in turn by him of classrooms. In Smith's terms the key texts--namely the diagnostic review and the high stakes literacy tests unleash a chain of action and further text-mediated practices, whereby people reorganise their work around the centrality of the data. While he welcomed opportunities, he saw a need to narrow and focus, to drill down. He was wary of teacher knowledge and judgment and, indeed, doubted his own professional knowledge. The discourses of data and improvement are not immediately meaningful yet they trigger a range of responses associated with the need to know differently.

A repeated theme of wanting not to be bound by the limits of the local and what that might entail, including being too comfortable and having low expectations, is reiterated. There was also a belief that solutions might be found elsewhere by taking opportunities and opening up in-school practices to review by outsiders. The resultant action however, and the resources required, must be found within the school, and the responsibility for change rests with the principal.

Barry: I mean I guess, just on an aside, my concern for that, as principal in a busy school, is it's all very well to identify, but if it's then the school needing to drive some sort of program or project, what support is there for me as leader for our staff to do more than we're already doing without resources, so it's going to be.

Researcher: So is there a potential criticism in there that we're doing more and more to identify but not necessarily more and more to respond?

Barry: I think there is, yeah, I think so. I think there's a feeling at times that the school's asked to do more and more of a family's responsibility, or a health agency responsibility, or a counselling role, whatever, and we don't have the resources.

Barry almost apologised for raising his concern about the lack of extra resources 'to close that gap significantly' and emphasised that his school 'did the best we could with the resources we had'. In an interview which is peppered with the principal's statements about meeting departmental expectations for improvement, this is one of the few criticisms he made about the impact of mandated literacy assessment on his work. He reported that he redirected the resources he had to meet the new priorities as he saw them. He was a willing subject appropriating the discourses which assessed his school as failing, taking on the responsibility that he needed to learn how to lead literacy in his school, assiduously meeting departmental expectations, and actively researching programs to 'close the gap'. Still, he portrayed himself as someone who needed 'skilling' in terms of interpreting what the mandated tests show and the implications for practice they imply. He concluded with a plea for help in that regard to the interviewer. The loss of trust in the teaching profession and the undermining of the reliability of their judgments which has been noted by critical policy analysts were evident here (see also Ball, 2001, 2003; Codd, 2005). The principal not only questioned his teachers' capacities to make sense of, and work productively with, data, but also his own. As Blackmore (2010, p. 642) observes, 'Leadership is about fear and desire the desire to make a difference, the desire to prove and improve oneself.'

This insight applies to Barry and, in terms of Smith's (2005) approach to understanding translocal discourses and practices on people's everyday doings, it is clear that from the time when Barry's school was designated as failing in terms of the NAPLAN results and the subsequent external review, to the time when the interview was conducted approximately a year later, he had significantly reorganised his approach to being a principal. He literally subjected himself to what he could and should learn about how to 'lead literacy in the school'. The new imperative to lift measurable standards in his school resulted in considerable attention and resources being redirected to assessment of literacy. His account indicated that he allocated a significant amount of his time and that of his teachers on activities designed to 'bump' up their results on NAPLAN.

The teachers

There were interesting patterns of similarity and difference with the principals' account when considering what the teachers had to say about their experience of NAPLAN and the broader discourse of data. For the teachers as well as the principal, the review process which followed the early NAPLAN results which represented Wheatville as a failing school was a key event. However, while the principal was able to represent this as an opportunity for opening out his own knowledge and experience, for some of the teachers it was the stigmatisation of failure that stood out for its implied judgement of their skill as teachers.

Our whole school got low figures over the last few years, and that meant that last year we had like a review. There were some people that came to our school, remember that, a team came to our school, and interviewed every one of us, and it really made me feel under pressure and like it was questioning my performance as a teacher, and my abilities, and made me feel quite low and flat ... They sent a team out to ... the aim was to review why we were getting low figures, and maybe develop some strategies, but it didn't come across that way at all I didn't feel. I felt they were very, quite intrusive and really not assisting us at all. They were more critical and judging and not sympathetic to the scenarios that we were [working] with (Year 3 teacher).

This extract illustrates the heightened anxiety that was evident when the teachers discussed the judgements arising from the school's NAPLAN results. It also illustrates Smith's (2005) insight that knowing from the standpoint of a practitioner, in this case, the teachers, is very different from the knowing that comes from abstraction, figures, and categorisation which ignores people's experience, in this case to the point where the reviewers were not interested at all in the actual 'scenarios' which teachers saw as key to their work. The same labels, practices and texts can come to have very different impacts on people's work, depending on their location or institutional position; and an educational consultant can 'read' a failing school without qualms. Another teacher emphasised that it was the potential for public naming and shaming that was a key concern.

Are they going the English and American models of putting them in publications and then, you know, people will be going, 'Oh, look at that school compared to that school', and parents are going to be going, 'Oh well, I'm going to take my child out of that school'. (Year 4/5 teacher)

The central message, according to the teachers, was that only the standardised data count, and that the NAPLAN results would come to be the sole measure of a school's reputation and of the work of the teachers. As the year 6/7 teacher said, 'There's so many brilliant things that take place, yet they'll just look at the results'. At the same time, the teachers had taken up Barry's message that standardised data were important and that improving results on the tests was the key response to be made, but there was little sense of support coming from beyond the school, possibly as a result of the negative experience of the school review. Thus teachers felt thrown back onto their own resources and histories in deciding how to move forward. For some, too, the language and logic of the discourse of data was unfamiliar.

I tell you what, it's very brain daunting when they do all of that in facts and figures and everything. I mean you're trying to relate that to how that works with you and what you're doing, and you've really got to have a mathematical mind to understand some of the stuff that they're putting out. (Year 3 teacher)

These frames of reference obliterate teachers' usual interpretive resources. In contrast to the principal, the teachers' talk was less about NAPLAN as an opportunity, and more about it as a problem to be encountered, wrestled with and overcome where possible. The teachers were not without resources or ideas from their own experience to deal with NAPLAN, but these were in discourses quite different from, or even cynical about, the promise of standardised testing. One of the teachers noted that NAPLAN was the latest in a long line of curriculum and assessment reforms stretching back more than two decades and resisted seeing the latest as the last word in what to do--after all, there would be others to come, and each had its own new lexicon to master.

Unlike the principal, the teachers were highly critical of the test, describing it as: limited to testing only a few components of literacy; emphasising simple answers; too much about pencil and paper; featuring content irrelevant to their students' lives; and, tellingly, not providing information that was particularly useful or timely for helping their students improve. On this last point, they noted that the timing of the results left little time to act on the information, especially as it was on only some students in their multi-grade classrooms. However, they were supportive of Barry on the need to gather more information about their students, so the perceived shortcomings of NAPLAN were compensated for by bringing in other standardised tests that were administered in a more timely way. Some teachers noted that the information from these tests was more user friendly than earlier standards-referenced models of assessment being promoted by the system, as they provided an age or grade level and for every student in the class. They also pointed out how many of the assumptions behind NAPLAN and school improvement were problematic in a small rural school. Teaching in multi-grade classrooms, information was provided about only some students and, due to the mobility of their student population, even those in the grades doing the test might have been elsewhere when the test was given, or moved on since.

The teachers clearly recognised the importance of the test data for the reputation of the school and their own performance as teachers, even as they disputed its validity for those who worked with students in poverty. They knew they had no choice but to engage in the discourse of data, and try to do the best they could to improve results. What this meant in practice, as they pointed out, was a narrowing of the curriculum and a loss of 'variety'. The teachers described how the response to the tests led to a focus on literacy, then within that field, to spelling and reading. The reasoning was that these were the areas that were high profile, and by concentrating on these, at least improvement might be possible. This seemed the only sensible response in practice given that the students were already behind their peers when they arrive at the school gate. As one teacher put it, 'your kids are going to be behind the 8-ball from the start, you know, to keep up with the standard, so yeah, it's a bit of a tough ask'. Led by Barry, the school had used its limited resources to buy in a US-based reading scheme Lexiles (http://lexile. com), which promised to, 'match readers with texts essential for growth and monitor their progress toward standards'. Thus the tests, even as they failed to provide teachers with useful guidance for their teaching, came to regulate their pedagogy to some degree and to determine the (newly narrowed and globalised) curriculum they offered. In spite of their belief in the importance of responding to their students in sensitive ways, the NAPLAN review process overwhelmed local aspirations through its emphasis on standard forms of (evidence-based) practice (see also Nichols & Griffith, 2009 for an example from Canada). As is shown in the next section, the ways that students could be talked about was also strongly affected by the discourse of data.

NAPLAN and the discursive construction of the poor rural student

A number of our research projects have investigated how normative discourses work to constitute the ideal (and indeed the problem) student with respect to English literacy (Comber, 1997; Cormack, 2011). We have always been concerned to understand the fallout for students and, indeed, teachers that emanates from the unproblematic insistence on the norm at the expense of diversity (Luke, 2012). Given that Wheatville was considered a failing school with the advent of high-stakes testing, we are interested in the effects on teachers' discursive and pedagogical practices as well as the interpretive resources they employed at this time and place to constitute their students.

In small rural primary schools, staffing formulas mean that it is inevitable that some classes are multiage, such as 3/4 or 5/6/7 for example. However, students are tested in year level cohorts and performances on NAPLAN are reported in terms of year levels for individual schools, allowing Year 3 cohorts around Australia to be compared. It is not surprising in this policy context that students came to be spoken about in ways that differentiated them from the norm. In the teacher focus group, students were variously described as members of a 'group' (year level, ability or age) and/or as 'disadvantaged', as we illustrate below.

The repeated insistence on the need for ability groupings for teaching component skills of literacy--spelling, reading, writing--was significant and reflected the school's goals to improve their measurable performance on standardised tests of literacy.

Once again it's split into four different groups at the moment in the class depending on their level of spelling. (Year 5/6 teacher)

I do reading groups and I do writing groups and I do spelling groups ... on the spelling side of things at the lower end we're also doing basic sight words, frequency words. (Year 3 teacher)

Here we can see how teachers organised their time and their students in order to accommodate different standards of performance as determined by a range of tests. The logic of NAPLAN transfers into the classroom as teachers grapple with different year level expectations a good example of what Smith (2005) refers to as the 'translocal' organisation of activity through coordinating texts. The acceptance of standards-grouping is not surprising, nor it being considered part of a possible solution for raising test scores. However, elsewhere in the discussion it was clear that this ethos was inconsistent with teachers' preferred ways of operating and their understandings about equity. The very pragmatics of grouping may have unanticipated effects.

4/5 teacher: This year for my NAPLAN somebody else administered the test. I had only about five students that had to sit the test, so those students worked with a .

3 teacher: So we have multi year level classes for starters, so there's one problem, so your class had to be split.

4/5 teacher: Yep. Those students that were going to do that test had to work with a Student Support Officer and go out with them for a number of hours over a week or two weeks I think it was, and go through last year's test and look at questions, talk about concepts, all of that sort of .

6/7 teacher: Which puts them at a disadvantage as opposed to those that are in a classroom where the teacher is, or has, that year level, and where your kids are having to go out for this, and like Teacher 4 had, it's mostly year 5, so a bigger group of year 5s, so what she's teaching she can sort of explicitly teach what they need to be doing, whereas these other kids only have a school support person in order to develop these kids.

Here the teachers explain how the logic of testing year-level cohorts impacts on what they do and how students are prepared for the tests and by whom, with one teacher raising questions about equity. One group of Year 5 students was actually prepared to take the test, not by their classroom teacher, but by an aide. Here we can see how students' educational disadvantage may be unwittingly escalated when the school's actions, intended to give more assistance in small groups, actually result in less teaching by a qualified educator. The Year 6/7 teacher had his students re-take the test so that he could use it to identify problem areas.

That's the most important thing that I use it for, because I've got immediate feedback as to what needs to happen so I can modify my program or modify what we're doing, just to look at what they've missed out on, where the group is lacking. (Year 6/7 teacher)

The mentality associated with the testing becomes one of looking for deficits, which is also implied in the principal's stated goal: 'what I'd like us to do better is to look at individual students and identify their gaps, and then try and address those gaps'. We see that if the student is framed within a discourse of testing and diagnosis, then the pedagogical problem is also framed as the need to identify and apply a cure. This is an example of the way that the discourse of data incorporates tiers of decisions and actions which reach into daily classroom life (Griffith & Smith, 2005) and come to affect how students are actually labelled and treated. As Nichols and Griffith (2009, p. 247) point out, 'people learn to describe their work in these terms'. We do not mean to suggest that there was no contestation amongst these teachers about this logic, nor that they subscribed totally to deficit discourses. The situation was considerably more complex with teachers employing a range of (sometimes contradictory) discourses to talk about their students' life experiences as unconnected to the content of the tests, or to the knowledge that counts. Hence, while the teachers reorganised their literacy lessons in terms of groups based on test standards, they by no means saw such tests as unproblematic. While they acted pragmatically on the findings in a somewhat compliant fashion, they noted issues related to reliability and also with the fairness of the texts of the tests themselves and reiterated the importance of considering context in terms of their work and their students. In the focus group discussions, teachers grappled with the consequences of NAPLAN and the wider discursive construction of the poor rural student.

4/5 teacher: Another way of thinking about is they have, I think it's category 1 to category 7 levels of disadvantage. Well we're a category 2, so we're pretty ... Aboriginal schools are 1s so.

Researcher: And so category 7 is like ...

3 teacher: One is the hardest, 2 is the second hardest.

4/5 teacher: Seven might be a Suburb A or a Suburb B [wealthy suburbs in the state's capital city].

6/7 teacher: The leafy suburbs apparently.

4/5 teacher: Well we're a 2, so that's, you know, and so .

3 teacher: When you come in here you're working hard.

4/5 teacher: It's those low category schools, you know, and it's all based on income and social disadvantage. I mean those low category schools are the ones that are going get most of the fallout from the NAPLAN.

The teachers draw on the state education department's Index of Disadvantage to situate their school on a continuum from most disadvantaged to the most affluent, concluding that working in their school was officially 'hard'. This discussion had been initiated by one of the teachers recalling the history of Wheatville in terms of class, but stressed that politicians and policymakers ('they') only take into account the test results and ignore 'the brilliant things' being achieved by the students and teachers in the school. In the dialogue above, the teachers invoke the school's authorised category of disadvantage to explain the test results and point out that it will be schools such as theirs that 'get most of the fall-out from NAPLAN' (see also Comber, 2012). As they continue the conversation we can see how these students and their families become a problem once again for their lack of literacy. The practices of parents are questioned in terms of their provision of adequate learning opportunities.

3 teacher: When you think of that in terms of literacy I mean: What's literate about their parents? What's in their home that supports literacy? How are they spending their money?

4/5 teacher: How is it valued?

3 teacher: Do they get a newspaper? Can they afford that? What do they do, do they tune into television? Do they read a book at night time?

6/7 teacher: How to get their attention and keep it and [inaudible--overtalk].

3 teacher: How to get them involved in what's going on. Do they actually understand what their children are doing? Are their children even better off than they are?

6/7 teacher: Are they at school?

4/5 teacher: So with those low categories .

3 teacher: Access to computers, you know, money problems and things.

4/5 teacher: So for those low category schools, I mean like we said before, it's no big surprise when we get the NAPLAN stuff back.

These propositions are all too familiar in that here we see a return to the ubiquitous circularity of educational blame and the near impossibility of 'getting out of deficit' (Comber & Kamler, 2004) or shifting 'default positions' (Johnston & Hayes, 2007), whereby the poor are blamed for their lack of effort, inappropriate spending, inadequate practices and so on. So, while they begin by recognising that Wheatville is a disadvantaged school with high levels of transience, low income and social disadvantage, they then buy into to the common sense argument that the parents are responsible for the problems they experience as teachers. The students may not be giving their 'attention', or even be 'at school'; at home they may not be accessing computers, books and parents who should help them become literate. Lack is a key theme here. The teachers then reiterate that the policy-makers and politicians ('they') still expect the teachers to do better.

3 teacher: But they want us to be better than we are.

4/5 teacher: Oh yeah, I know that, and we aspire to be better than we are, but for those low category schools, when those results come back, and if they start using it in a much wider community-based publications or whatever .

3 teacher: 'This area is a low achiever, don't send them there'.

4/5 teacher: ... [T]hose schools that are high category, they're going to go 'Oh yeah, well we're doing alright, we could lift our game but we're doing OK'. It's all the 1s, 2s, 3s, they're all the ones, and what have we learned from that, really, in the big picture things, what have we learnt? Yes, these kids are disadvantaged. Yes, their literacy and numeracy skills are lower. Gee, that's a surprise!

While the teachers accept their responsibility to aspire to do better, buying into the bootstrapping rationality and continuous improvement discourses, they fear the backlash from the publicity associated with failure and what it can do to their community. They predict that their enrolments may diminish as they rehearse what parents might say to each other, even imagining a description of Wheatville portrayed as a 'low achieving area'. It's not so surprising when they end rather cynically pointing out they learn little through these processes of testing and public accountability that helps them to do their work any better. Meanwhile, the leafy green schools are 'doing alright'.

Conclusions

With limited resources at their disposal, teachers and school leaders have little choice but to respond with respect to the logic of the discourse of data and work

to limit reputational damage to the school and themselves. As we have seen, however, the resources for that response may be very limited in a rural school serving a poor community. There seemed little opportunity for the bootstrapping processes assumed in the logic of standards-reform, or for teachers to learn from those places where it is apparently done better. The reality was not of a lively market for reform, but the imposition of standards-based solutions and subsequent take-up of global, generic curriculum from a large multinational company in the form of a reading scheme. Both the principal and the teachers were left to struggle with the implications of meaning of the data they collect which, ironically, only served to confirm what they already knew--that their students were 'behind the 8-ball'--and simultaneously to discount what was actually being accomplished.

Students, in turn, are labelled, grouped and taught in relation to the deficits the tests reveal. They experience a curriculum which focuses on what they can't do, which also squeezes out other, more hopeful and energising, topics and activities. Notwithstanding the limits and the pressures, some teachers still aim for ethical pedagogic practices even in the face of demands to focus on minimal standards. These teachers can call on long experience in finding ways of working with students that don't come from a prescribed program manual, and which are tailored to their students. For example, one teacher reported having his students re-take the test in order to enable him help them with areas they did not understand. Rather than waiting for the results to come several months later, he took pedagogical responsibility for immediately teaching them the specific strategies and approaches to the tasks indicated as a problem by the tests. Despite teachers' best intentions, however, standardised mandated assessments do have differential effects. Rather than 'closing the gap' or ameliorating educational disadvantage, policies may, indeed, result in a limited and reduced education for children growing up in rural poverty. Results from the mandated test led to a further of investment of money and time in testing and the purchase of levelled reading schemes as numerical comparative data is seen as all the matters. The ways in which the work of teachers is coordinated beyond the district or even the state become clear along with the significant risks of educators coming to doubt their own judgments.

One key problem with the discourse of data and its various technologies, is that the materiality of poverty and place is backgrounded while reconstituting teachers' work as sets of generic techniques. In this regime, schools are responsible for failure and left with the problems that are identified through standardised tests. Educational researchers have long stressed the importance of contesting the deficit discourses concerning the learning capabilities of students growing up in poverty. Yet we have found it difficult to help teachers to assemble other discursive resources for understanding the material realities of poverty without assigning blame. Not surprisingly teachers can resort to deficit explanations which, in turn, pass on the blame for failure to parents and the community. This is not the only story to be told of the school, however. Both the principal and the teachers tried to work around the shortcomings of the situation in which they found themselves and had past experience of success--limited and transitory as it may have been--to build on 'the brilliant things' that have worked for students in difficult circumstances. Unfortunately, within the discourse of data, such experience doesn't 'count'.

To return to Dorothy Smith's (2005) desire for a 'sociology for people', whereby people come to understand how their work is being coordinated, our hope is that research such as this which shows how policy is enacted in particular situations will create the possibilities for change and for educators to imagine 'richer and more intelligent' accountabilities (Lingard, 2010, p. 132). The dominance of comparative numerical data about student performance in limited aspects of literacy, as the only data that counts for student and school success, is simply not good enough.

Acknowledgement

This paper is based on work from the research project, Mandated Literacy Assessment and the Reorganisation of Teachers' Work is an Australian Research Council (ARC) Discovery Project (DP0986449) between the University of South Australia, Queensland University of Technology and Deakin University in Australia and York and Victoria Universities in Canada. The chief investigators are Barbara Comber, Phillip Cormack, Helen Nixon, Alex Kostogriz and Brenton Doecke. Partner investigators in Canada are Dorothy Smith and Alison Griffith. The views expressed in this paper are those of the authors only.

References

Alexander, R. (2010, 10 March). The Perils of Policy: Success, Amnesia and Collateral Damage in Systemic Educational Reform. Paper presented at the Miegunyah Distinguished Visiting Fellowship Program 2010 public lecture.

Ball, S. (2001). Performativities and fabrications in the education economy. In D. Gleeson & C. Husbands (Eds.). The performing school: Managing and learning in the performance culture (210-226). London & New York: RoutledgeFalmer.

Ball, S.J. (2003). The teacher's soul and the terrors of performativity. Journal of Education Policy, 18(2), 215-228.

Berliner, David C. (2013 forthcoming) 'Sorting out the effects of inequality and poverty, teachers and schooling, on

America's youth' in S.L. Nichols (Ed.) Educational Policy and the Socialization of Youth for the 21st Century. New York: Teachers College Press.

Blackmore, J. (2004). Restructuring educational leadership in changing contexts: A local / global account of restructuring in Australia. Journal of Educational Change, 5, 267-288.

Codd, J. (2005). Teachers as 'managed professionals' in the global education industry: The New Zealand experience. Educational Review 57(2), 193-206.

Comber, B. (2012). Mandated literacy assessment and the reorganisation of teachers' work: Federal policy and local effects. Critical Studies in Education. 53(2), 119-136.

Comber, B. (1997). Managerial Discourses: Tracking the local effects on teachers' and students' work in literacy lessons.

Discourse: Studies in the Cultural Politics of Education, 18 (3), 389-407.

Comber, B. & Cormack, P. (2011). Education policy mediation: Principals' work with mandated literacy assessment. English in Australia, 46(2), 77-86.

Comber, B & Hill, S. (2000). Socio-economic Disadvantage, Literacy and Social Justice; Learning from Longitudinal Case Study Research, The Australian Educational Researcher, 27(9), 79-98.

Comber, B. & Kamler, B. (2004). Getting out of deficit: Pedagogies of reconnection. Teaching Education, 15(3), 293-310.

Cormack, P. (2011). Reading pedagogy, 'evidence' and education policy: Learning from history? Australian Educational Researcher, 38(2), 133-148.

Green, B. & Letts, W. (2007). Space, equity and rural education: a trialectic account. In K. Gulson & C. Symes (Eds.), Spatial Theories of Education: Policy and Geography Matters (57-76). London: Routledge.

Griffith, A. & Smith, D. (2005). Mothering for schooling. New York: RoutledgeFalmer.

Johnston. K & Hayes, K. (2007). Supporting student success at school through teacher professional learning: the pedagogy of disrupting the default modes of schooling. International Journal of Inclusive Education, 11(3), 371-381.

Kostogriz, A. & Doecke, B. (2011). Standards-based accountability: reification, responsibility and the ethical subject. Teaching Education, 22(4), 397-412.

Labaree, D.F. (2011). The lure of statistics for educational researchers. Educational Theory, 61(6), 621-632.

Lingard, B. (2010). Policy borrowing, policy learning: testing times in Australian schooling. Critical Studies in Education, 51(2), 129-147.

Luke, A. (2012). After the testing: Talking and reading and writing the world. Journal of Adolescent and Adult Literacy 56(1), 8-13.

McWilliam, E. & Singh, P. (2004), Safety in numbers? Teacher collegiality in the risk-conscious school. Journal of Educational Inquiry 5(1), 22-33.

Nichols, N. & Griffith, A. (2009). Talk, texts, and educational action: an institutional ethnography of policy in practice. Cambridge Journal of Education, 39(2), 241-255.

Poster, M., Lash, S. & Morris, P. (1996). Databases as discourse, or electronic interpellations. In P. Heelas (Ed.), Detraditionalization: Critical Reflections on Authority and Identity (277-293). Cambridge, MA: Blackwell Publishers.

Smith, D. (Ed.) (2006). Institutional ethnography as practice. Lanham: Rowman & Littlefield Publishers.

Smith, D. (2005). Institutional Ethnography: A sociology for people. Lanham: AltaMira Press.

Smith, D. (1987). The everyday world as problematic: a feminist sociology. Boston: Northeastern University Press.

Somerville, M., Plunkett, M. & Dyson, M. (2010). New teachers learning in rural and regional Australia. Asia-Pacific Journal of Teacher Education, 38(1), 39-55.

Teese, R. (2000). Academic Success and Social Power: Examinations and Inequality. Carlton South: Melbourne University Press.

Teese, R. & Polesel, J. (2003). Undemocratic Schooling: Equity and Quality in Mass Secondary Education in Australia. Carlton, Vic.: Melbourne University Press.

Thomson, P. (2009). School leadership: Heads on the block. London & New York: Routledge.

Phillip Cormack

University of South Australia

Barbara Comber

Queensland University of Technology

(1) All statistics on Wheatville have been rounded or provided in a range to strengthen anonymity.

Phillip Cormack is Adjunct Associate Research Professor of Education in the School of Education at the University of South Australia, Australia. His research interests include the history of adolescence, and contemporary and historical perspectives on literacy policy, curriculum and pedagogy. He currently works on projects on the history of reading education in Australia and the impact of contemporary standardised testing reforms on teachers' work.

Barbara Comber is a Research Professor in the Faculty of Education at Queensland University of Technology. She is particularly interested in literacy education and social justice. She has conducted a range of research with teachers working in high poverty and culturally diverse communities. She has co-edited a number of books including the forthcoming International Handbook of Research in Children's Literacy, Learning and Culture (Hall et al. in press), Literacies in Place: Teaching environmental communications (Comber et al., 2007) Turn-around pedagogies: Literacy interventions for at-risk students (Comber & Kamler, 2005) Look again: Longitudinal studies of children's literacy learning (Comber & Barnett, 2003).

Table 1
 2006 2011

Median weekly ... Australia- Wheatville Australia- Wheatville
 wide wide

 individual income $466 $300 $577 $390
 household income $1,027 $570 $1,234 $720
 family income $1,171 $660 $1,481 $850