” Federal Indigenous affairs bureaucrats have released a draft of their new evaluation framework, eight months after the Commonwealth committed $40 million over four years to evaluate policies in the portfolio and put a highly regarded university professor in the driving seat.
The draft sets out processes to look more objectively at national policies to support Aboriginal and Torres Strait Islander communities and contribute to Closing the Gap, which have been led by the Department of the Prime Minister and Cabinet for the past few years.”This is intended to align with the role of the Productivity Commission in overseeing the development and implementation of a whole of government evaluation strategy of policies and programs that effect Indigenous Australians,”
PM&C sets high standards for Indigenous affairs evaluation see PART 1 Below
” It’s been widely known for fifty years that the health of Aboriginal people lags far behind that of other Australians. Despite that and the expenditure of billions of taxpayers’ dollars, serious gaps persist between Indigenous versus non-Indigenous health and wellbeing.
There is compelling evidence that social factors are potent determinants of the health of populations. In the simplest of terms these are (a) social disadvantage, and (b) the relationship of Indigenous Australians to mainstream society. Associated with these are basic issues already mentioned; these include education, housing standards, employment and socio-economic status. These must be addressed if health disadvantages are to be overcome. Until this happens the poor health outcomes of Indigenous Australians will persist.
It’s easy to identify medical problems, perhaps because they can be classified and measured. It is tempting then to decide that these problems are ‘medical’ and, therefore, should respond to ‘medical’ interventions or approaches in isolation. This is dangerously misleading.
It’s time for clinicians to realise and publicly acknowledge that most of the important issues which determine the health status of Indigenous people have ‘non-medical’ roots and need vigorous ‘non-medical’ approaches in order to be corrected.
Part 1
Around the same time as the new evaluation funding was announced, Malcolm Turnbull sought out indigenous health expert Ian Anderson to take over as deputy secretary leading the PM&C indigenous affairs group, which is also the only group within the central department overseen by an associate secretary, Andrew Tongue.
Anderson’s first major task was a review of the Closing the Gap target framework, which focuses attention on particular indicators of disadvantage. A few months into the job he set out some of his thoughts in a public speech at a special event marking 50 years since the referendum that effectively created this area of federal policy.
The framework notes good evaluation is “planned from the start, and provides feedback along the way” (referencing the audit office’s 2014 better practice guide to public sector governance).
“Good evaluation is systematic, defensible, credible and unbiased. It is respectful of diverse voices and world-views.
“Evaluation is distinct from but related to monitoring and performance reviews. Evaluation may use data gathered in monitoring as one source of evidence, while information obtained through monitoring and performance reviews may help inform evaluation priorities.”
The credibility of future evaluations depends on demonstrating their independence. To this end, the framework says a new external advisory committee, membership so far unknown, will “support transparency and ensure the conduct and prioritisation of evaluations is independent and impartial” by overseeing how the new framework is applied, checking the annual evaluation plan and with “ongoing advice, quality assurance and review”.
A “commitment to transparency” is also included. The committee will publish “all high priority evaluations” and reviews of them. Others will be randomly reviewed and summarised in an annual report.
“At the three year mark an independent meta-review of IAG evaluations will be undertaken to assess the extent to which the Framework has achieved its aims for greater capability, integration and use of robust evaluation evidence against the standards described under each of the best practice principles.”
All the actual evaluation reports will be published as well, at least in summary form, including “where ethical confidentiality concerns or commercial in confidence requirements” apply. Indigenous communities that have participated in evaluations will get to see the results too and additional “knowledge translation” efforts are proposed:
“Evaluation findings will be of interest to communities and service providers implementing programs as well as government decision-makers. Evaluation activities under the Framework will be designed to support service providers in gaining feedback about innovative approaches to program implementation and practical strategies for achieving positive outcomes across a range of community settings.”
The draft framework says it aims to:
- generate high quality evidence that is used to inform decision making,
- strengthen Indigenous leadership in evaluation,
- build capability by fostering a collaborative culture of evaluative thinking and continuous learning across the IAG and more broadly across communities and organisations, and
- place collaboration and ethical ways of doing high quality evaluation at the forefront of evaluation practice in order to inform decision making.
Higher quality evaluation that is “ethical, inclusive and focused on improving outcomes” is more likely to have impact, the draft points out. “It aims to pursue consistent standards of evaluation of Indigenous Advancement Strategy (IAS) programs but not impose a ‘one-size-fits-all’ model of evaluation.”
The guide calls for best-practice evaluation to be “integrated into the cycles of policy and community decision-making” in a way that is “collaborative, timely and culturally inclusive.”
“Our approach to evaluation, as outlined in this Framework, reflects a strong commitment to working with Indigenous Australians.
“Our collaborative efforts centre on recognising the strengths of Aboriginal and Torres Strait Islander peoples, communities and cultures.
“Fostering leadership and bringing the diverse perspectives of Indigenous Australians into evaluation processes helps ensure the relevance, credibility and usefulness of evaluation findings. In evaluation, this means we value the involvement of Indigenous Australian evaluators in conducting all forms of evaluation, particularly using participatory methods that grow our mutual understanding.”
Indigenous Advancement Strategy evaluations will look at how well programs meet three criteria:
Do they build on strengths to make a positive contribution to the lives of current and future generations of Indigenous Australians?
Are they designed and delivered in collaboration with Indigenous Australians, ensuring diverse voices are heard and respected?
Do they demonstrate cultural respect towards Indigenous Australians?
Four elements of good evaluation
The draft framework lists four elements of good evaluations — they are robust, relevant, credible and appropriate, which is to say they are “fit for purpose” and done in a timely fashion — and explains in detail how each of these ideals is to be achieved in Indigenous affairs through higher standards.
“Evaluation needs to be integrated into the feedback cycles of policy, program design and evidence-informed decision-making,” explains a chapter on relevance. “Evaluation feedback cycles can provide insights to service providers and communities to enhance the evidence available to support positive change. This can occur at many points in the cycle.”
While not being too prescriptive, the framework aims to set a high standard for the evidence that is used to judge the impact of programs.
“A range of evaluation methodologies can be used to undertake impact evaluation. Evaluations under the Framework will range in scope, scale, and in the kinds of questions they ask. Measuring long-term impact is challenging but important. We need to identify markers of progress that are linked by evidence to the desired outcomes.
“The transferability of evaluation findings are critical to ensure relevant and useful knowledge is generated under the Framework. High quality impact evaluations use appropriate methods and draw upon a range of data sources both qualitative and quantitative.
“Evaluation design should utilise methodologies that produce rigorous evidence and make full use of participatory methods. Use of participatory approaches to evaluation is one example of demonstrating the core values of the Framework in practice.”
Perhaps the moves to take a more academic approach at the federal level will allow for more open discussion of what works, in a portfolio where this year the minister has seen fit to publicly attack researchers in the field, and blast the independent audit office for doing its job instead of helping him attack the opposition.
Part 2 : Aboriginal health: An embarrassing decades-long saga
It’s been widely known for fifty years that the health of Aboriginal people lags far behind that of other Australians. Despite that and the expenditure of billions of taxpayers’ dollars, serious gaps persist between Indigenous versus non-Indigenous health and wellbeing.
Recognition of an Aboriginal Health Problem
When these inequities were recognised in the 1960s the very high rates of Aboriginal childhood malnutrition and infections and high death rates of infants and young children brought home the unpalatable fact that Australia had a so-called ‘Third World’ health problem. This is a feature of poverty-stricken nations. This was clearly unacceptable in our otherwise affluent and healthy country. There was a public outcry which stirred the federal government into attempts to remedy this embarrassing state of affairs.
In 1979 the Commonwealth Parliamentary Committee on Aboriginal Affairs found that . . .
‘the appalling state of Aboriginal health’ . . . ‘can be largely attributed to the unsatisfactory environmental conditions in which Aboriginals live, to their low socio-economic status in the Australian community, and to the failure of health authorities to give sufficient attention to the special needs of Aboriginals and to take proper account of their social and cultural beliefs and practices’ . . .
The Committee criticised governments for their lack of recognition of these factors and commented on the need for Aboriginal people to be much more closely involved in all stages of planning and delivering their own health care. Notwithstanding some improvements in Indigenous health which occurred over the almost forty years that followed, many of that Committee’s findings and criticisms are still valid.
Efforts to Improve Indigenous Health
In 1981 a $50 million Aboriginal Health Improvement Program was launched with the aim of upgrading environmental health standards, such as better housing and community and family hygiene conditions. Government funds were allocated and State and Territory health departments implemented strategies and programs and deployed clinical and allied staff in order to achieve better Indigenous health.
An important objective was to provide more accessible services for Indigenous people. Some positive health gains followed; for example, better pregnancy outcomes, fewer maternal deaths, fewer infant and young child infections, suppression of vaccine-preventable illnesses through immunisation, and lower infant death rates.
This should have helped Indigenous youngsters to negotiate the rough ride through early life that would otherwise have been their lot. However, health and disease statistics for Indigenous Australians generally stayed well behind those of other citizens in the years that followed.
Strategies to ‘Close the Gap’
The persisting poor standards of Indigenous health prompted the Federal Government in 2008 to ‘Close the Gap’ for Indigenous Australians in a range of health outcomes and other facets of life and wellbeing so that they and other Australians would have ‘equal life chances’. The then Prime Minister Rudd anticipated within a decade halving the widening gap in literacy, numeracy and employment opportunities for Indigenous people. The Statement of Intent also anticipated better opportunities for Indigenous children so that within a decade . . . “the appalling gap in infant mortality rates between Indigenous and non-Indigenous children would be halved and, within a generation, the equally appalling 17-year life gap between Indigenous and non-Indigenous when it comes to overall life expectancy” . . . would be gone.
These aspirations seemed commendable and were well received by the public. However, their feasibility was questioned soon after they were announced. The target of closing the gap in life expectancy was said to be “probably unattainable” and the capacity to extinguish the risk of chronic diseases (like heart disease, diabetes and kidney disease) and related deaths was considered publicly by a renowned medical expert to be “implausible” in the 22-year timetable set out by the government. This is pertinent because those chronic diseases are the main contributor to the discrepancy in Indigenous versus non-Indigenous deaths. Those reservations were well founded.
Obstructions to Closing the Gap
Indigenous Australians now have very high rates of chronic diseases, as already mentioned. These are aggravated by smoking- and drug-related disorders. These conditions are long-term and have permanent complications, such as visual loss or blindness, or severe limitations on mobility. These cannot be reversed and, therefore, restrict prospects for longevity. In many Aboriginal communities a third or half of adults 35 years or over have one or more of these problems. Nationally, these diseases and accidental or intentional injuries, including suicide and homicide, are several times more prevalent in Indigenous Australians than in the total Australian population.
This well-documented and widespread heavy burden of illnesses, disabilities and related excess premature deaths among Indigenous Australians makes it virtually impossible to remove, within a generation, the inequalities between this pattern and the better outcomes which prevail in the rest of the population. This is made more difficult because some of these problems are trans-generational and can have their origins during intra-uterine development.
There are practical impediments in bringing better health to the Indigenous population. Inadequate access and maldistribution of facilities, personnel and services can be serious drawbacks, particularly in rural and remote areas. Of course, improving access to services does not necessarily lead to their appropriate utilisation.
And compliance with treatments and follow-up supervision and medications can be problematic. Similarly, altering health knowledge and modifying risky personal lifestyles are difficult among many people whether they are Indigenous or not. There have also been serious problems with management and governance of clinical services for Indigenous people whether they are Indigenous-specific or mainstream services.
This has tended to weaken their impact on health service delivery and waste limited financial and other resources. Collectively, all of these factors have diluted the much-needed positive outcomes of efforts to close the gaps in Aboriginal health standards and statistics.
Indigenous Health: the current situation
Some indicators of the current situation are revealing: death rates of Indigenous children under five years are more than double the national rates; their low birth weight rate is about double the overall national rate; hospitalisation rates are almost three times the national rates; hospital admission rates for potentially preventable conditions are almost four times higher; deaths from complications of diabetes at 35 to 55 years are approximately twenty times higher; and dementia rates are about five time higher than in non-Indigenous Australians and the condition starts earlier in life. The Australian Institute of Health and Welfare estimated that among Indigenous Australians born from 2010 to 2012 life expectancy would be about nine to ten years shorter than for other Australians. These indicators of health status, illness patterns and life expectancy are disgraceful and require urgent attention.
Where to from here?
Social Dimensions which affect Health
There is compelling evidence that social factors are potent determinants of the health of populations.
In the simplest of terms these are (a) social disadvantage, and (b) the relationship of Indigenous Australians to mainstream society. Associated with these are basic issues already mentioned; these include education, housing standards, employment and socio-economic status.
These must be addressed if health disadvantages are to be overcome. Until this happens the poor health outcomes of Indigenous Australians will persist.
It’s easy to identify medical problems, perhaps because they can be classified and measured. It is tempting then to decide that these problems are ‘medical’ and, therefore, should respond to ‘medical’ interventions or approaches in isolation. This is dangerously misleading. It’s time for clinicians to realise and publicly acknowledge that most of the important issues which determine the health status of Indigenous people have ‘non-medical’ roots and need vigorous ‘non-medical’ approaches in order to be corrected. This means, of course, that non-medical sectors of governments must accept more responsibility and become more actively involved in issues which ultimately determine the health of populations which they are expected to serve. This will require a major shift in thinking within Federal and State governments and bureaucracies and wider acceptance among the Australian community.
The challenges are daunting but the need is urgent. Surely it is within our collective capabilities to turn around this sad and long-standing saga into a success story.
Michael Gracey AO is a paediatrician who has worked with Indigenous children, their families and communities for more than forty years. He was Australia’s first Professor of Aboriginal Health and for many years was Principal Medical Adviser on Aboriginal Health to the Western Australian Department of Health. He is a former President of the International Paediatric Association.
Reblogged this on World4Justice : NOW! Lobby Forum..