Since its introduction in 2007, there has been much debate over the effectiveness of the Northern Territory Emergency Response (NTER) in improving the quality of life in remote indigenous communities. Public discussion on using good evidence for policymaking has so far not encouraged Indigenous Affairs Minister Jenny Macklin to improve the quality of Families, Housing, Community Services and Indigenous Affairs (FaHCSIA) data. Given the failure rate of policymaking in this area, these data deficits are problematic.
Questions on the value of many of the measures in the NTER have been raised in official reports as well as in independent studies. Jumbunna, the Indigenous House of Learning based at UTS, researched evidence on the effectiveness of income management. We also compiled a report on the data that the minister was using to justify maintaining and extending the program in The Journal of Indigenous Policy. We concluded that existing studies showed no reliable evidence of benefits to individuals or communities.
Income management should be one of the easier programs to measure as there are records of financial transactions. This data could be used to assess changes in savings and purchasing patterns, but none of these have been included in official studies. Statistics collected on school attendance, education, crime, health and child welfare should be able to offer evidence of changes in wellbeing and safety. However, this data — collected since 2007 — has shown scant improvement in wellbeing.
The best evidence offered by government media releases is in the form of anecdotal reports. Official surveys in Western Australia and the Northern Territory included many positive accounts from individuals, but lacked any independent confirmation. Moreover, there are more questions about the validity of the minister’s claims of evidence, which are presumably intended to support legislation in the Senate for extending NTER measures. There are two more research reports and a report that has been designed to support the legitimacy of the Stronger Futures Consultation Report. All show serious flaws in design, and fail to produce clear evidence for benefits that would support legislation in the Senate.
The Committee on Community Affairs is calling for submissions for the three Bills and holding hearings until the beginning of February. If approved, this legislation will continue and extend programs without any serious evidence that the current programs have improved the situation of NT communities covered by the intervention.
Official documents, produced in late 2011, try to claim claim that research proves the benefits of the intervention. The reports included an online survey of service providers and a commissioned study of people living in some of the targeted communities. The material below looks at these studies in more detail.
The survey covered the opinions of nearly 700 service providers, most of which were government-run or funded. Less than 9% were Indigenous, and few came from isolated communities. While most of the workers reported they felt safer and thought that their communities were safer, there was no independent evidence to support their claims. Their responses show that they saw better results in community safety and reduced problems in smaller (rather than larger) communities. This difference was not noted by the governments, which remain committed to developing the larger centres instead of smaller ones, which seem to function better. This view also emerged in the survey below. It is hard to see how these opinions validate the extension of the programs, given that there were reported criticisms about the loss of local control.
The Community Safety and Wellbeing Research Study (CSWRS) was an ambitious attempt to tap into local views. It covered 1411 residents in seventeen remote NT Indigenous communities. The emphasis was on opinions, so it might offer a credible record of some local views, but not any credible measures of actual benefits. It was not clear how the small local samples were chosen, which raises questions about the representative nature of the sample and the possible exclusion of dissident voices.
Interestingly, the report comments on the contradiction between respondents’ positive perceptions on indicators such as school attendance, despite evidence of static or even deteriorating performance. These studies are part of the Northern Territory Emergency Response Evaluation Report . In its introduction, it includes the following caveat :
‘While the report does have a strong focus on data, it is important to understand that there are only around 45,000 Indigenous Australians resident in the NTER communities. It can be difficult at times to observe trends in some outcome data for what is a relatively small population over a four-year period. It is also important to understand that the NTER is a very complex policy response that has many elements. It is not always possible to identify the additional impact of individual measures because so many changes, both NTER and other measures, were introduced at a similar time.’
This extraordinary admission is in the context of the decisions already made on income management and the new Stronger Futures initiatives. There is no evidence that the new Improving School Enrolment and Attendance through Welfare Reform Measure (SEAM) initiatives will work, and they have no clear mandate from the so-called consultations. The O’Brien Rich Stronger Futures Quantitative Analysis Report attempts to address this. The report is a statistical analysis of notes taken by FaHCSIA staff at consultations in the NT. These were notes taken by public servants as feedback rather than research data, so the recycling of them in this way is odd. Nor was any ethics clearance sought to use them as such. In this light, the report should be read as a summary of information recorded during the consultations, rather than a representation of the opinions of consulted communities.
The best view of the data available is that opinions of the community are diverse, but there still no hard data on program benefits. As there is other evidence that purchasing patterns, for example, have not changed significantly because of income management, a moratorium on extending programs would be a good starting point for serious interest evidence-based policy. Why spend lots of extra money on extending programs without evidence that they work?