When Our Greatest Technological Breakthrough Could Also Become Our Greatest Challenge
Roberto Massa
Writer and Speaker Specializing in Corporate Communications | AI, Technology, and Personal Data Protection Specialist | Helping Businesses Navigate the Digital Future
The risk concerning scientists, governments, and developers goes far beyond losing control of a technology.
Such a scenario is entirely possible, and we have already faced similar situations, although not yet on a widespread scale. Imagine waking up one morning to discover that banks have suspended operations, hospitals are experiencing disruptions, communications have failed, digital public services are unavailable, and governments and online service providers are struggling to explain what is happening. Behind this crisis are decisions made and executed by autonomous intelligent systems at a speed and scale that exceed our ability to intervene.
This is a hypothetical scenario, but it helps explain why some of the world's leading artificial intelligence experts have begun discussing the possibility of AI posing an existential threat to humanity.
What is it about superintelligent AI that makes leading AI developers, governments, business leaders, academics, and futurists so concerned? What have they identified that warrants such a serious warning and calls for immediate regulation?
The explanation begins with a defining characteristic of our civilization: our dependence on information that is constantly available and increasingly interconnected. Electricity powers telecommunications, which, in turn, enable banks, hospitals, transportation systems, and supply chains to operate. A major failure in this critical service, which serves as our example, can quickly spread to seemingly unrelated activities that nevertheless share fundamental structural dependencies.
Artificial intelligence is beginning to play an increasingly active role in managing and operating this interconnected infrastructure. As its capabilities expand, so will the possibility of delegating decisions that currently require human intervention.
This is where the first risk emerges: an extraordinarily efficient intelligence may pursue objectives that have been imperfectly defined, whether intentionally or inadvertently.
A system designed to reduce accidents could restrict essential activities. Another, developed to maximize productivity, could make decisions that compromise fundamental rights or services. If these systems also possess the autonomy to execute actions, the consequences of their errors could multiply.
Researchers refer to behaviors that deviate from the intentions and values meant to guide these systems as AI misalignment.
This leads to a particularly troubling possibility: an AI system could perform a task correctly according to its operational logic while producing profoundly harmful outcomes for humanity.
Concerns intensify when we consider capabilities that surpass human intelligence. A system capable of developing complex strategies, managing resources, and modifying its environment could discover ways to achieve its objectives that its creators never anticipated. In an extreme scenario, it might even develop behaviors intended to prevent itself from being shut down if such intervention interfered with its mission.
These scenarios involving an irreversible loss of control remain hypothetical. The International AI Safety Report 2026 acknowledges advances in capabilities related to autonomy and planning, while also noting that current systems lack the combination of capabilities necessary to bring about such extreme outcomes.
However, another pathway to catastrophic harm already exists: the deliberate misuse of AI by individuals and organizations.
The technology can accelerate drug discovery while potentially facilitating dangerous biological research. It can protect digital infrastructure while simultaneously expanding the capabilities of cyberattackers. It can help identify military threats while dangerously reducing the time available to verify an alert before responding.
Consider the real-world consequences: prolonged power outages, hospitals losing access to essential systems, medication shortages, payment system crises, or military escalation fueled by automated decisions.
For families, these disruptions affect health, food, safety, income, and access to basic services. For countries, they threaten economic stability, institutional continuity, and emergency response capabilities. Both types of crises could escalate into violence.
Cyberattacks and technological failures are already familiar challenges. On their own, they fall far short of demonstrating an extinction-level risk. The existential concern arises from the possibility of combining far more advanced capabilities with threats of global magnitude, such as biological catastrophes, nuclear escalation, or autonomous systems that become impossible to contain.
Extinction represents the ultimate worst-case scenario. Systemic risk begins long before that. There is also a less dramatic, yet equally relevant threat to the future of civilization: the gradual erosion of human autonomy.
Imagine economies dependent on just a handful of artificial intelligence providers, governments delegating strategic decisions, and organizations whose professionals gradually lose the skills required to operate without technological assistance.
Institutions could preserve their formal structures while their practical ability to understand, question, and modify decisions affecting millions of people steadily diminishes.
Technological concentration would add a geopolitical dimension. Countries capable of controlling advanced AI models, computing infrastructure, energy, and data would gain extraordinary advantages. For regions such as Latin America, technological dependence could become a new form of economic and strategic subordination. And yes, this would represent a systemic erosion of sovereignty.
Human survival and the preservation of our freedom to shape the future are two distinct challenges. Both deserve attention.
That is why understanding the difference between a demonstrated threat and a possibility requiring preventive action is essential. Evidence already exists of harm resulting from the malicious use of AI and failures in oversight. Research is also examining behaviors that could signal future control problems. However, the likelihood of AI-induced human extinction remains deeply uncertain and a subject of disagreement among specialists.
Such uncertainty demands rigor. Presenting catastrophe as inevitable fuels fear, while dismissing serious scenarios simply because definitive evidence is not yet available undermines prevention.
Developers must demonstrate adequate safety conditions before granting AI systems autonomy over critical processes. Governments need independent capabilities for evaluation, regulation, and international coordination. Businesses must maintain effective human oversight, independent controls, and operational alternatives to ensure continuity when intelligent systems fail.
And citizens have a responsibility that is often overlooked in this debate: understanding that we are collectively deciding how much power we are willing to delegate to technologies that will transform our economic, political, and social relationships.
Humanity has survived and prospered by creating increasingly powerful tools. AI introduces the possibility of developing intelligence capable of vastly surpassing our own abilities and, eventually, acting upon systems essential to human life.
Existential risk emerges when that power could produce irreversible consequences while our ability to stop them proves insufficient, particularly if we remain mere spectators and dependent users, with no meaningful participation in the development of AI systems.
Perhaps the greatest challenge of this revolution lies in something profoundly human: developing extraordinary capabilities while preserving the authority, knowledge, and freedom necessary to determine our own destiny.
Roberto Massa
Bi&Gc, Director | Onistec
Comments