News in recent weeks about AI and AI agents, statements and open letters from top politicians and public figures, the frantic drive to regulate everything, the recent "hacking" of Australian health data and so on have generated a considerable amount of public and political debate. Unfortunately, much of the public discourse on the subject is fairly problematic.

First, there are no rogue agents from OpenAI and Anthropic (and other companies providing LLMs, or large language models) roaming around, breaking into systems of their own accord and wreaking havoc. Framing the situation this way does not reflect reality. What we have instead are automated applications built by someone for a specific purpose, given insufficiently detailed instructions and constraints and carrying out the commands they have been given. Software. Capable software, certainly.

These applications have no morality of their own. They do not make ethical judgments or operate within the context of laws and societal rules. Various considerations and rules can be prescribed for these applications as soft constraints, but those boundaries have gaps because, due to the particular characteristics of the technology, the applications may not comply with all the requirements imposed on them.

If the task given to an application was (deliberately) described too broadly, its security measures were (deliberately) inadequate, its monitoring was (deliberately) inadequate, its reporting mechanisms were (deliberately) inadequate — or all of the above, combined with carelessness and disregard for best practices — then there is little point in casting an accusatory eye at the application that was launched.

The technology underlying these applications has no experience and no concept of legality or permissibility. The probabilistic nature inherent in it does not operate in the same way as what we have previously been accustomed to.

You can, for example, throw a ball for your Labrador and shout, "Fetch!" and then reasonably expect, though not know with 100 percent certainty, that the dog will probably jump over a low fence (rather than ram it down or set it on fire), swim across a shallow stream (rather than cut down the surrounding forest to build a bridge), dodge a child playing in its path (rather than trample the child) and bring back the ball you threw (rather than collect every ball thrown to other dogs within 10 square kilometers, because its owner obviously likes balls).

An AI agent, or automated application, by contrast, may, if it has access to the necessary tools, take steps in various situations resembling those described in parentheses because, based on its training material, weights and configuration, the application "concludes" that the very step that would seem absurd or dangerous to humans will help achieve the objective described in the task.

An AI agent does not behave like a human, or like an animal. Even the verb "behave" is clearly conditional here because we are talking about an application that performs operations rather than behaves. Because of the particular characteristics of the technology and the nature of human language (and the human mind), LLM-based applications have simply ended up in a role in which it is easy to treat them as conscious beings. Much as, in some religions of the past, the wind, sun, rain and thunder were treated as such.

In light of recent news, it should also be said that there was no "evil AI agent hack" of Australian healthcare information systems, despite how the situation was initially presented to the public.

There was an automated application that, based on what is known so far, either used an unexpected method to access public data (in which case the problem is an inadequately thought-out or documented healthcare system architecture) or exploited a security vulnerability to access confidential data (in which case the problem is inadequate prior security testing).

The situation is not fundamentally different from a citizen interested in data finding that data at an undisclosed but publicly accessible web address or from a cracker (or hacker, though that distinction requires a longer discussion) discovering a security vulnerability in a system.

There is, however, an important difference in the intensity with which such incidents will begin occurring in a world of AI agents and in the potentially dangerous impact those incidents may have.

In practical terms, every individual, group, company and country now has the ability to automate many tasks that previously would have taken a great deal of time or would not have been done at all because they would have required too much time and money. Some of these tasks may lead AI agents, or automated applications, down a path that a human would not have chosen — for example, one that violates the law. Though, of course, humans come in all kinds too.

In some cases, likely an increasing number of them, breaking the law is exactly what the person launching the application wants. Theft of personal data, theft of trade secrets, discovering and exploiting security vulnerabilities in the systems of other countries' government agencies or critical service providers — all of this can now be done several orders of magnitude faster than before.

In none of these cases, however, is this an "AI agent did it" situation when it comes to responsibility. "The hammer did it" or "the ax did it"? No. Even if that hammer falls off scaffolding and hits someone on the head, someone first carelessly put the hammer there or installed the scaffolding badly. The hammer did not choose to fall.

Yes, AI agents are technologically far more complex than a hammer or an ax because a hammer generally does not strike things on its own and cannot order an ax to aim its blade. Behind every blow and every movement is a human being, directly and immediately.

With automated solutions, however, there is no longer a human being directly behind every individual step. Modern production lines perform a series of operations without human intervention. In fact, we are already familiar with this from the digital world, for example in data analysis and report generation. Even a formula in an Excel cell automatically performs calculations when the input data changes.

The difference with AI agents is significant, however, because we have never before had such readily accessible ways to tell automated solutions that, while completing a task, they may leave their environment when necessary, use various external tools and keep looking for ways to improve results until they succeed if those results happen to fall below the desired level.

Figuratively speaking, we can give an Excel formula the tools and capabilities — for example, by exploiting vulnerabilities in third-party systems — to start changing the trend in its input data in order to improve margins, bankrupting a competitor in the process or steering a competitor's CEO's self-driving car off the road and into a ravine. The report for the company's supervisory board may be delayed slightly, but the charts will look great.

Many of the risks discussed in connection with the use of artificial intelligence are real risks. Not all of them, however, because we still do not have superintelligence whose risks are also discussed in the same breath, and presidential decrees will not bring it into existence.

The way artificial intelligence risks are discussed, however, is at times absurd. Rapid automation presents both opportunities and risks. Malicious actors have already begun using it. But it is these actors who commit crimes, not AI agents — software portrayed almost as mystical beings.

In my view, new regulations governing development would not have a significant effect. The technology in question has already advanced to the point where further development can, if there is sufficient interest, take place covertly. Regulations governing its use would work to some extent, but they would affect only those seeking to avoid breaking the law, not those who misuse the technology.

Given the capabilities of this technology that are already known, as well as those that can currently only be glimpsed, it is also unlikely that countries with political systems and civil liberties somewhat different from ours and that seek to improve their international standing in competition among states would, first, go along with such regulations or, second, actually comply with them even if they appeared to do so.

The various percentages attached to scare stories and cited in support of regulation are pure fiction. "A 10 percent probability that ..." and "in 50 percent of future cases ..." remain inventions even when they are backed up with citations to pseudoscientific papers and studies that do not, in fact, support those inventions. We currently lack the ability to make such forecasts reliably. There is too much we do not know.

When considering predictions of all kinds, it is worth remembering that AI psychosis is already extremely widespread. Unlike some other dangerous technologies, artificial intelligence in particular has attracted a disturbingly large number of quasi-religious people, as well as people who do not appear troubled by the ethical dilemma of whether it makes sense to actively contribute to humanity's destruction. Any possible destruction, I emphasize, would not be brought about by an LLM application that suddenly became conscious and began to hate humanity, but by people using LLM applications.

Some people have fallen into the trap of misleading information simply out of ignorance, including, unfortunately, most politicians and public figures who speak on the subject because they do not know how to distinguish other people's falsehoods from facts. This group often tries to do what it believes is good, but unfortunately things turn out the way they always do. This group is primarily afraid and acts out of that fear.

There are also those who are not afraid at all. This group naively and hopefully believes all the marketing announcements, bogus studies and pseudoscientific papers and thinks that LLM-based applications either are already conscious or soon will be and that we are almost certainly already in AGI (Artificial General Intelligence, or general artificial intelligence) or even ASI (Artificial Super Intelligence, or artificial superintelligence) territory. I cannot say what motivates these people.

And then, of course, there are people whose income and future wealth depend directly on how many people believe in the supreme capabilities of LLM-based applications. Some of these people develop LLM technologies (and may also be preparing for an IPO), some have raised investors' money to fulfill promises that are proving far more difficult to deliver than expected, while others have found a subject about which other people (whether frightened or hopeful) know as little as they do, creating an opportunity to convert this exceptionally favorable energy into revenue. See also: snake oil salesmen, gold rush, NFTs.

If additional legislation may not deliver results, falling into the trap of LLM mania is surprisingly easy and the situation is being exploited by scammers, charlatans and bad actors ranging from small-time operators to those with an international reach, then what can we do?

The following is certainly not a definitive or comprehensive list of what might help in the current situation.

• We urgently need to make our digital solutions even more resilient because the volume, speed and systematic nature of attacks will, in the near future, be several orders of magnitude greater than what we have been accustomed to.
• More effective monitoring systems will help detect more of these attacks more quickly, so we need to invest even more in them as well.
• More broadly, we must assume that it may not be possible to prevent a successful attack or data breach, which means we must also have a plan for what comes afterward: how we restore operations, how we communicate and how we learn.
• To make better use of the opportunities offered by AI technologies ourselves, both as a country and as businesses, we need to build stronger frameworks for automation and continuously strengthen and improve them as we learn from experience.
• We must preserve the ability to continue experimenting, testing, hypothesizing and making mistakes, but we must create that opportunity and capability as safely as possible — so-called sandboxes in which, however, we do not actively ignore everything that information security professionals have discovered and addressed over the past several decades.
• With AI-based applications (including LLM-based applications), as well as the agreements and policies concerning them, we must keep people at the center of it all. Applications (and regulations) work for people, not the other way around. This is easy to forget and the desire for authoritarianism is already emerging very strongly in current debates, proving remarkably attractive across almost the entire political spectrum.
• To avoid, or at least reduce, tomorrow's problems, we need to devote even greater attention to education. While the ability to use AI technologies and applications is also very important, even more important are general analytical skills, the ability to recognize and respond to threats, the courage to say "no" and the courage to make informed choices. It is crucial that efforts to ensure and improve these capabilities reach every group in society.

Finally, neither individually nor collectively can we give in to cognitive laziness and surrender our thinking and creativity to machines simply because it seems easier or because someone in a position of authority said this is our future and that we should welcome the Brave New World with open arms, lest we be considered backward or even criminal.

Let history be a source of lessons for us on AI as well.

P.S. LLM (Large Language Model) applications are one subset of the AI world. LLM-based applications are not all-encompassing artificial intelligence. Nor are LLM-based applications Skynet, the Terminator, HAL, etc., familiar from books and movies. It is unlikely, not to say exceedingly unlikely, that LLM technology specifically is the path that will lead to the world of AGI/ASI and conscious AI. The very fact that LLMs are equated with AI in many discussions — and often not with present-day AI but with sci-fi AI — is itself a source of many of the social and political problems associated with AI.

--

Follow ERR News on Facebook and X and never miss an update!