Since its introduction in 2007, there has been much debate over the effectiveness of the Northern Territory Emergency Response (NTER) in improving the quality of life in remote indigenous communities. Public discussion on using good evidence for policymaking has so far not encouraged Indigenous Affairs Minister Jenny Macklin to improve the quality of Families, Housing, Community Services and Indigenous Affairs (FaHCSIA) data. Given the failure rate of policymaking in this area, these data deficits are problematic.
Questions on the value of many of the measures in the NTER have been raised in official reports as well as in independent studies. Jumbunna, the Indigenous House of Learning based at UTS, researched evidence on the effectiveness of income management. We also compiled a report on the data that the minister was using to justify maintaining and extending the program in The Journal of Indigenous Policy. We concluded that existing studies showed no reliable evidence of benefits to individuals or communities.
Income management should be one of the easier programs to measure as there are records of financial transactions. This data could be used to assess changes in savings and purchasing patterns, but none of these have been included in official studies. Statistics collected on school attendance, education, crime, health and child welfare should be able to offer evidence of changes in wellbeing and safety. However, this data — collected since 2007 — has shown scant improvement in wellbeing.
The best evidence offered by government media releases is in the form of anecdotal reports. Official surveys in Western Australia and the Northern Territory included many positive accounts from individuals, but lacked any independent confirmation. Moreover, there are more questions about the validity of the minister’s claims of evidence, which are presumably intended to support legislation in the Senate for extending NTER measures. There are two more research reports and a report that has been designed to support the legitimacy of the Stronger Futures Consultation Report. All show serious flaws in design, and fail to produce clear evidence for benefits that would support legislation in the Senate.
The Committee on Community Affairs is calling for submissions for the three Bills and holding hearings until the beginning of February. If approved, this legislation will continue and extend programs without any serious evidence that the current programs have improved the situation of NT communities covered by the intervention.
Official documents, produced in late 2011, try to claim claim that research proves the benefits of the intervention. The reports included an online survey of service providers and a commissioned study of people living in some of the targeted communities. The material below looks at these studies in more detail.
The survey covered the opinions of nearly 700 service providers, most of which were government-run or funded. Less than 9% were Indigenous, and few came from isolated communities. While most of the workers reported they felt safer and thought that their communities were safer, there was no independent evidence to support their claims. Their responses show that they saw better results in community safety and reduced problems in smaller (rather than larger) communities. This difference was not noted by the governments, which remain committed to developing the larger centres instead of smaller ones, which seem to function better. This view also emerged in the survey below. It is hard to see how these opinions validate the extension of the programs, given that there were reported criticisms about the loss of local control.
The Community Safety and Wellbeing Research Study (CSWRS) was an ambitious attempt to tap into local views. It covered 1411 residents in seventeen remote NT Indigenous communities. The emphasis was on opinions, so it might offer a credible record of some local views, but not any credible measures of actual benefits. It was not clear how the small local samples were chosen, which raises questions about the representative nature of the sample and the possible exclusion of dissident voices.
Interestingly, the report comments on the contradiction between respondents' positive perceptions on indicators such as school attendance, despite evidence of static or even deteriorating performance. These studies are part of the Northern Territory Emergency Response Evaluation Report . In its introduction, it includes the following caveat :
‘While the report does have a strong focus on data, it is important to understand that there are only around 45,000 Indigenous Australians resident in the NTER communities. It can be difficult at times to observe trends in some outcome data for what is a relatively small population over a four-year period. It is also important to understand that the NTER is a very complex policy response that has many elements. It is not always possible to identify the additional impact of individual measures because so many changes, both NTER and other measures, were introduced at a similar time.’
This extraordinary admission is in the context of the decisions already made on income management and the new Stronger Futures initiatives. There is no evidence that the new Improving School Enrolment and Attendance through Welfare Reform Measure (SEAM) initiatives will work, and they have no clear mandate from the so-called consultations. The O’Brien Rich Stronger Futures Quantitative Analysis Report attempts to address this. The report is a statistical analysis of notes taken by FaHCSIA staff at consultations in the NT. These were notes taken by public servants as feedback rather than research data, so the recycling of them in this way is odd. Nor was any ethics clearance sought to use them as such. In this light, the report should be read as a summary of information recorded during the consultations, rather than a representation of the opinions of consulted communities.
The best view of the data available is that opinions of the community are diverse, but there still no hard data on program benefits. As there is other evidence that purchasing patterns, for example, have not changed significantly because of income management, a moratorium on extending programs would be a good starting point for serious interest evidence-based policy. Why spend lots of extra money on extending programs without evidence that they work?
Join the conversation
Comments (12)
Jon Altman
(Research Professor in Anthropology at Australian National University)
I fail to see what is uncomfortably political about Eva Cox's article for two comment authors. Cox raises an important question for policy formulation for a government committed to evidence based policy making: if the $2 billion spend on the NTER has not delivered measurable and sustainable positive outcomes then shouldn't thought be given to an alternate approach? This seems to be the Gillard government preferred approach, for example, with gambling reform where there is to be a trial for a year…
show full comment
Matt Stevens
Senior Research Fellow/Statistician (logged in via email @gmail.com)
Hi Jon, I do understand that the NT Indigenous sample has its limitations, but this is typical of your comments to trash any available data that does show an improvement in conditions in communities. At this stage I have only looked at the Australia level by remoteness, and nearly all personal stressors (particularly those related to social transgressions that I mentioned) go down significantly from 2002 to 2008. While this cannot be fully attributed to the intervention, I think it is possible that…
show full comment
Thomas Marshall
Architectural Assistant (logged in via email @gmail.com)
One of the most amazing acts of the NTER was to completely disregard all of the research and programs being done by Healthabitat. Housing For Health (document) was given to the Federal Government, and dismissed.
Yet Housing For Health was fixing houses for 10% of the cost that the Government's people were doing it. Not only that, but the Housing For Health projects were achieving their objectives of improving sanitation, while the Government's building program was doing no such thing.
Go figure.
If you're interested to know more, take a look: http://www.healthabitat.com/housing-for-health/results/how-the-houses-perform
Matt Stevens
Senior Research Fellow/Statistician (logged in via email @gmail.com)
There is no reliable evidence that the housing fixes done by health habitat, fix the health of the people. Good houses do not = good health. It is more to do with overcrowding, hygiene and nutrition, and the usual suspects of socio-demographic and socioeconomic variables, particularly with child health. Sustainable housing will only occur though a reduction in crowding, the slow change of cultural change (like a didactic) which I mean by adaptation over generations to living a sedentary lifestyle…
show full comment
Bernie Masters
environmental consultant (logged in via email @iinet.net.au)
Jon: Eva Cox finishes her article by stating: "a moratorium on extending programs would be a good starting point for serious interest evidence-based policy. Why spend lots of extra money on extending programs without evidence that they work?". I view this to be a political statement since the corollary is to ask: why not spend lots of extra money when there is no evidence to suggest that the programs do not work? Neither statement is based on evidence and hence neither statement deserves to be put…
show full comment
Bernie Masters
environmental consultant (logged in via email @iinet.net.au)
The absence of verifiable data does not mean that the NTER hasn't provided valuable benefits to the women and children which the intervention was designed to protect. The author should stop playing political games and instead go out and obtain credible data to prove one way or the other that NTER has or hasn't worked.
Matt Stevens
Senior Research Fellow/Statistician (logged in via email @gmail.com)
There is some survey data from the 2002 and 2008 NATSISS (the 2008 survey started about 6-12 months after the NTER started), which show improvements across a range of "personal stressors", including witness to violence, being abused or in a violent crime, gambling problems, alcohol or drug problems..to name a few. Some of these measures have dropped by 10% or more. The high levels of personal stressors experienced by indigenous people living in remote communities has a huge impact on the way people function in their day to day lives and a reduction in these measures is a significant change.
However, while I don't like the politics of the article, I do agree that is pretty pathetic that more and better quality evaluation data has not been collected or made available for analysis.
Victor Martin
Retired (logged in via email @gmail.com)
Having lived in remote communities over years I notice that there has been endless streams of reorganisations, surveys, data collection, analysis, political comment etc generated by the NTER. And little has changed. May be it is about time everybody stopped whipping them selves. Apply law and order, supply health support, education support, employment for those who want to work, housing and centrelink as in most communities in Australia and leave it up to the people to decide if they want to use the system.
Matt Stevens
Senior Research Fellow/Statistician (logged in via email @gmail.com)
Basic understanding of how the white system works is required for Aboriginal people to use it. Unfortunately for many remote community residents in the NT, that basic understanding is not there...we all know why that is the case.
Victor Martin
Retired (logged in via email @gmail.com)
Hi Matt Stevens, They may or may not have basic understanding but as long as we patronise them they will not progress. They certainly have no problems understanding things they really want
Matt Stevens
Senior Research Fellow/Statistician (logged in via email @gmail.com)
I didn't mean to sound condescending, but clearly the lack of education and an inability to read at an adult level has enormous implications on access to information etc.
jim morris
(logged in via email @yahoo.com)
I recently read Bad Dreaming by Louis Nowra. Everybody should read it, especially if they doubt the necessity for intervention.