Pope Leo takes aim at big tech in sweeping encyclical on AI

"Disarming is not enough, we must build," he added, calling for broad participation in the programming, regulation and benefits of AI.

The pope presented the document alongside high-ranking Vatican prelates, Catholic theologians and Chris Olah, the co-founder of Anthropic, the American AI company behind Claude that has been pushing to brand itself as safety-oriented and ethical.

Speaking at the event, Olah said that AI development "operates inside a set of incentives and constraints that can sometimes conflict with doing the right thing," citing commercial concerns, geopolitical pressure and pride and ambition.

For that reason, he said, "we need more of the world – religious communities, civil society, scholars, governments – to do what His Holiness has done here: to take this seriously, to look closely, and to push events in a better direction."

At the heart of the encyclical is the insistence that human beings take precedence over artificial intelligence. The dignity of the human person "does not depend on a person's abilities, wealth or position in life, nor on the right or wrong choices made," but simply by virtue of existing, the encyclical argues.

In a time of AI chatbots, Leo wrote that the risk is not just that someone interacting with an AI agent might believe they are talking to a person, but that they might lose the desire to seek other people at all. And handing over decision-making to machines may "encourage excessive reliance and the search for ready-made answers, and weaken personal creativity and judgment."

In the document, the pope recognizes the positive impact AI innovation can bring to human society and to the care of the environment, while also warning of its dangers. "Technology is never neutral," Leo wrote, adding that it's an expression of the interests and stakeholders behind it.

"When such power is concentrated in the hands of a few, it tends to become opaque and evade public oversight, increasing the risk of distorted forms of development that give rise to new dependencies, exclusions, manipulations and inequalities," Leo wrote.

Leo argued that applying moral and ethical principles to AI models cannot happen after it has wreaked havoc on society — such values must be applied in its construction. "For AI to respect human dignity and truly serve the common good, responsibility must be clearly defined at every stage: from those who design and develop these systems to those who use them and rely on them for concrete decisions," he wrote.

Pushing back against executives who resist restrictions on AI development, Leo explained that "calling for prudence, rigorous evaluation and even, at times, a slower pace in adopting AI does not mean opposing progress; instead, it is an exercise of responsible care for the human family."

In a nod to AI models that have adopted ethical constitutions — such as Anthropic — the pope said such frameworks must still be discussed and subjected to criteria of shared social justice. "A more moral AI is not enough if that morality is determined by a few," he wrote.

The document also takes on popular Silicon Valley philosophies, transhumanism and posthumanism, which the pope defined as a series of assumptions "that interpret progress as surpassing the human condition."

These philosophies view limitations — including illness, disability, old age and vulnerability — as something to overcome, Leo wrote, but "we must remember that humanity flourishes not despite limitations, but often through them," adding that a life without limitations would ultimately mean not being human.

Cardinal Víctor Manuel Fernández, who heads the Vatican's doctrinal department, said at the presentation of the document that unlike these philosophies, claiming "that humanity has reached its expiration date and must simply be replaced," Catholic teaching believes that "every human being has infinite dignity."

In contrasting the dangers of AI, Leo pointed to truth as "an essential element of democracy" and urged for people to receive education on AI, especially the young. While AI promises to alleviate the burdens of work, Leo warned that "it frequently forces workers to adapt to the speed and demands of machines, rather than machines being designed to support those who work."

In this fourth industrial revolution, the priority must be "the protection of employment opportunities and the irreplaceable role of the individual," Leo wrote.

As global economic disparity grows, the pope warned "technological progress will inevitably produce structural inequalities." Leo called for tax systems that ease the burden on the most vulnerable and demand more from those with greater resources, while the benefits of innovations should be transparent and shared with the entire community.

The pope warned against "new forms of slavery," highlighting the trail of human and environmental exploitation behind AI — from the models training on copyrighted material to the extraction of rare minerals used in AI hardware. Leo also reflected on the history of the church's behavior toward slavery, which was not fully denounced until the 19th century by Leo XIII, highlighting the slow evolution of Catholic doctrine over time.

"For this, in the name of the Church, I sincerely ask for pardon," Leo wrote.

There is a "new face" of colonialism, Leo argued, which doesn't only dominate bodies, but also appropriates data: health flows, epidemiological profiles, genetic maps and demographic data. "These have become the new rare earths of power," Leo said, adding that AI technology in the hands of few, profit-oriented individuals or groups represents a new form of colonial dominion.

"Here lies one of the most urgent moral challenges of our time: to ensure that shared knowledge becomes a true common good rather than an instrument of dominance. This requires restoring to individuals not only the data that describes them, but also the ability to decide how it is used, by whom and for whose benefit," he wrote.

A lengthy portion of the document reflects on the topic of war, which the pope has denounced repeatedly in his first year as pontiff. The military use of AI "must be subject to the most rigorous ethical constraints," he wrote, adding that responsibility for military and especially lethal action should remain with human beings, not machines.

He called for traceability within the decision-making processes of warfare, for human oversight, and for the creation of international laws to address the increased use of automated weapons and their consequences.

The document is based on the contributions of past pontiffs, starting with Pope Leo XIII, and draws on Catholic social teaching, the church's tradition of applying moral principles to social, economic and political life, with human dignity and the common good at its center.

Solidarity and caring for one's "neighbor" are key aspects of Leo's reflection, with particular concern for the next generations.

Social justice "is not merely about the fairer distribution of resources or the correction of current injustices, but also assumes a restorative dimension," Leo wrote. Applying the same principle to the digital realm, the pope warned against "new forms of exclusion and deprivation of freedoms," such as invasive surveillance, communities denied access to basic technologies and groups harmed by opaque algorithms that perpetuate discrimination.

In this context, he also addressed migration as a "litmus test for social justice today."

In the introduction, Leo draws from the Bible to juxtapose the city of Babel, with its pursuit of power and self-sufficiency, and Nehemiah's rebuilding of Jerusalem, oriented on prayer and the participation of families and communities. This reflects the teachings of the pope's religious order, founded on the principles of St. Augustine, that contrast love for the earthly city, structured around concepts of power and dominance, with the longing for the heavenly city of God.

"As throughout history, these two loves continue to contend for dominance in our hearts today. The age of AI is no exception: the construction of Babel or the rebuilding of Jerusalem begins within each one of us," he wrote.

"This is a landmark opportunity for the world to look at a new technology and really think about what it is for," said Brian Patrick Green, director of technology ethics at Santa Clara University's Markkula Center for Applied Ethics.

"What is the purpose of this technology? What is it supposed to do in the world? How can it help people? What do we need to do in order to make sure that this technology does the best that it can do for the most people in the world?"

This story was produced via a partnership between NPR and Religion News Service.