AI Firms ‘Unprepared’ for Dangers of Building Human-Level Systems

• AI firms are deemed ‘fundamentally unprepared’ for risks of human-level systems.
• None of the firms scored higher than a D in existential safety planning.
• Anthropic received the highest safety score at C+, OpenAI at C.
• Max Tegmark suggests their safety plans resemble a nuclear plant without safeguards.
• The pace of AI advancement continues to outstrip preparations for AGI.

Concerns About AI Firms and Human-level Intelligence

AI Companies Are Falling Short on Safety Planning. A recent report from the Future of Life Institute (FLI) has raised serious alarms about the preparedness of artificial intelligence companies in tackling the potential dangers associated with developing systems that can think at human levels. According to the safety group’s findings, none of the firms managed to earn a score higher than a D in existential safety planning. This worrying conclusion was supported by one of the five experts involved in the review, who emphasized that, while these companies are on the trajectory to create artificial general intelligence (AGI), there is a glaring lack of coherent and actionable plans in place to ensure that these advanced systems can be controlled and operate safely.

Unexpected Risks from Artificial General Intelligence

AGI and its Implications for Society. So, what exactly is AGI? It’s a concept describing an AI’s capability to perform any intellectual task just as a human can. The warnings from safety advocates are quite serious, indicating that such advancements could present existential risks—potentially allowing machines to operate beyond human control and possibly leading to catastrophic scenarios. The FLI’s report clearly stated, “The industry is fundamentally unprepared for its own stated goals.” This is rather alarming considering that many of the companies mentioned are claiming that they’re on the path to achieving AGI within the next decade, yet none of them have established robust plans for managing that development from a safety perspective.

Industry Ratings and Expert Opinions

Safety Ratings of Leading AI Developers. The index evaluated prominent players in the field, including Google DeepMind, OpenAI, Anthropic, Meta, xAI, and two companies from China, Zhipu AI and DeepSeek, across six specific domains, focusing on factors like current harms posed by their technologies and overall existential safety practices. Among them, Anthropic topped the safety scores with a C+, followed closely by OpenAI at a C, and Google DeepMind trailing with a C-. It’s important to note that this evaluation was executed and scrutinized by a panel of AI experts, including the renowned computer scientist Stuart Russell and Sneha Revanur of the advocacy group Encode Justice. Despite these grades, Max Tegmark, a co-founder of FLI and a professor at MIT, expressed his disbelief that such advanced AI companies aim to create systems with human-level intelligence without having any comprehensive plans to confront the risks involved. He likened the situation to constructing a large nuclear facility in a major city without safety measures in place—a precarious and ill-advised endeavor.

In sum, a startling report suggests AI companies may be inadequately prepared for the risks posed by the development of human-level intelligence systems. With the FLI reporting that none have excelled in existential safety planning, the concerns expressed by experts such as Max Tegmark highlight the urgency of creating effective safety measures. Overall, the current trajectory raises serious questions about whether companies can responsibly develop AGI without putting society at risk.