by Rajith Vidanaarachchi, Maneesha Perera, Sangeetha Chandrashekeran, Brendan Kennedy, Saman Halgamuge, University of Melbourne
As Australia marks 50 years of NAIDOC Week, honoring the world's oldest living culture, humanity's newest technology has yet to reckon with a simple principle: "nothing about us, without us." The concern is that artificial intelligence (AI), like so many technologies before it, will become another extractive force that "takes" Indigenous Knowledges without consent, credit or return.
The tension behind that question has reached the global stage. In his first encyclical, Magnifica Humanitas, Pope Leo XIV cast AI as one of the defining moral tests of our age.
He talked about the dangers of AI becoming a tool for oppression, emphasizing its potential to discriminate against minority groups, marginalized populations and Indigenous peoples.
He ultimately said we need to confront fundamental questions about the ownership, governance, accountability and distribution of the spoils of AI.
Our research found a dual reality: While AI is empowering, it also carries significant risks of entrenching existing and historical biases.
The aim of our research, based on a First Nations-led research partnership that predates the industry's now-visible reach, was to explore the intersection of Indigenous Knowledge Systems and AI.
We also wanted to weigh this against the risks and opportunities for Indigenous peoples around the world.
Our review of 53 academic studies published between 2012 and 2023 identified four ways AI can affect Indigenous Knowledge holders and communities.
We found AI can entrench existing biases, accelerating harm rather than preventing it.
So protecting communities requires a new, engaged and consensual approach to training Large Language Models (LLMs). Indigenous Knowledge must be ethically embedded, properly recognized and genuinely valued as part of this process.
But there are also positives. AI can be a tool to preserve knowledge for community benefit and to design solutions that address Indigenous-identified challenges.
In fact, researchers around the world are already using AI to revitalize endangered languages, map culturally significant land and document oral traditions.
In Western Australia, Curtin University has developed a prototype app that identifies wildflowers in the area, then plays audio of their names in the Noongar language, helping with pronunciation.
In California, researchers are partnering with the Amah Mutsun Tribal Band, a sovereign Indigenous nation of more than 20 historic clans. Together, they are using machine learning to map plant distributions critical to the tribe's cultural revitalization.
Computational tools like this can support Indigenous Knowledge holders in documenting, preserving and keeping their cultural heritage alive.
AI is also helping address challenges prioritized by Indigenous communities themselves.
In regional and remote Western Australia, Aboriginal medical clinics are trialing AI-assisted screening to combat diabetic retinopathy, a leading cause of preventable blindness.
It's an initiative that directly addresses a critical gap in areas where screening rates have historically lagged.
Other Australian health care providers are using AI tools to improve ear health by detecting middle-ear diseases in Aboriginal and Torres Strait Islander children.
Beyond Australia, in Taiwan, health care programs are using predictive modeling to assess fall risks for Indigenous elders—bridging critical gaps caused by distance, skills shortages and high costs.
But these examples are not the whole picture.
The same systems that support can also disrespect and damage Indigenous peoples and their Knowledge systems.
A child protection AI used in Aotearoa New Zealand disproportionately targeted Indigenous families. Research found that the algorithm referred Māori children to protection services at rates far higher than actual case numbers justified.
In Australia, an AI eye-screening tool misdiagnosed Aboriginal patients with high rates of false positives. Because the system was trained exclusively on data sets from Chinese clinics, it's likely the algorithms failed to account for natural variations in retinal pigmentation.
These are not glitches. They are the predictable consequence of building systems on generalized data that doesn't reflect the needs of specific populations.
But a deeper concern underlies these examples. Open data harvesting is central to how AI developers build technology.
For Indigenous peoples, who have been the subject of extractive research for centuries, this approach violates the principles of Free, Prior and Informed Consent.
It directly denies communities the right to govern their own information.
Two Indigenous-led global frameworks are explicit on this.
Most AI systems in use today were not designed with these principles in mind.
Critics have long warned that AI can reinforce existing structures of power. Eurocentric ideologies shape AI, privileging individualism and Western ways of seeing the world, and the systems then reproduce those assumptions in the categories they learn.
Many researchers say AI must be examined through a decolonial lens, one that reveals the historical and ongoing operations of colonizing power.
Indigenous Knowledge Systems offer a different starting point.
In Aotearoa New Zealand, Mātauranga Māori centers the relationships between humans, non-humans and te taiao (nature and all living beings). In sub-Saharan Africa, the Ubuntu philosophy centers on collective well-being.
These are not flavors to add at the end of an AI design process. They are alternative foundations on which new AI could be designed.
The work ahead is one of regulation, not goodwill.
Regulators must move principles like CARE and OCAP from aspiration into law, making Indigenous Data Sovereignty a binding standard that every AI system using Indigenous data must meet.
Australia's voluntary AI Ethics Framework is one place this work can begin.
The technology industry must accept that co-design with communities is a precondition for deployment, not an afterthought. Companies building systems for health care, child protection and other high-stakes settings must train their models only on data Indigenous peoples deem valid.
This involves centering and valuing Indigenous Knowledges in the data training process through recognition, respect and financial recompense.
This goes to a larger set of questions that need clarification.
Who owns the training data behind the AI we use? Who benefits when it works? Who pays the cost when an algorithm gets it wrong?
As AI booms, the patterns we found have only sharpened.
We are building AI now and locking in its assumptions and biases. The only question is whether it reflects one way of seeing the world, or many.
BA art history, MA material culture. Former museum editor, paramedic, and transplant coordinator. Editing for Science X since 2021. Full profile →
This article was first published on Pursuit. Read the original article here.
Originally reported by Phys.org.