When generative artificial intelligence first captured the public’s imagination, the cybersecurity community simultaneously braced for a future where sophisticated, autonomous malware could be developed and deployed by AI with terrifying efficiency. This initial wave of concern, which emerged nearly three years ago, painted a grim picture of an imminent and dramatic escalation in cyber warfare. However, a deep analysis of the specialized, malicious large language models (LLMs) that have since appeared, often dubbed “dark LLMs,” reveals a reality that is far more subdued. Investigations into leading platforms like WormGPT 4 and KawaiiGPT show a significant disconnect between the early, breathless hype and their actual, observable capabilities. Rather than acting as revolutionary weapons for advanced adversaries, these tools have found a niche as force multipliers for low-skilled criminals and have, for the most part, proven to be technically underwhelming, failing to fundamentally alter the cyber threat landscape as many had feared.
The Reality of Dark LLM Capabilities
Empowering Novice Attackers
The most significant and practical application of dark LLMs lies in their ability to assist novice hackers and cybercriminals who lack technical expertise or face language barriers. Their primary strength is not in creating novel attack vectors but in refining existing ones, particularly in the realm of social engineering. These models excel at generating persuasive, grammatically impeccable text, allowing attackers to craft convincing phishing emails, business correspondence, and professional-sounding ransom notes. This capability is especially valuable for threat actors who are not native speakers of their target’s language, as it helps eliminate the tell-tale spelling errors and awkward phrasing that often betray less sophisticated scam attempts. By smoothing out these operational kinks, dark LLMs can substantially increase the potential success rate of basic social engineering campaigns, making them appear more legitimate and harder for the average user to detect at a glance. Beyond improving communication, these malicious AI platforms also serve to democratize the creation of simple malware, effectively lowering the barrier to entry for cybercrime. Models like WormGPT 4 have demonstrated the capacity to produce functional malicious code snippets upon request, such as a basic locker for PDF files that can be configured to target other extensions. Similarly, KawaiiGPT can generate simple yet effective Python scripts designed for data exfiltration or assist an attacker with lateral movement within a compromised Linux environment. In this capacity, the LLMs act less like an evil genius and more like an interactive guide, walking a “script kiddie” through the various stages of a standard attack chain. This allows individuals with minimal coding knowledge to generate the tools they need for their attacks, essentially providing a step-by-step tutorial for carrying out rudimentary cyber offenses without requiring them to understand the underlying technical complexities of the code they are deploying.
The Underground Marketplace
The emergence of dark LLMs has carved out a new commercial and developmental frontier within the cybercrime ecosystem, a trend that gained momentum in the summer of 2023. This market was largely sparked by the introduction of a malware-as-a-service (MaaS) product known as WormGPT, which was explicitly marketed as a boundary-free AI alternative to mainstream models like ChatGPT. While there is little evidence to suggest that the original WormGPT had a significant real-world impact, it served as a successful proof-of-concept that ignited the imagination of the underground community and inspired a wave of imitators. The current market is now characterized by a blend of commercial and private development efforts. For instance, tools like WormGPT 4 operate on a tiered subscription model, charging users anywhere from tens to hundreds of dollars per month for access and boasting a dedicated Telegram community of over 500 subscribers, indicating a stable commercial interest.
This flourishing market is not limited to paid services, as competitors have entered the space with different business models, further diversifying the landscape. Its main rival, KawaiiGPT, has managed to cultivate a modest but active user base of over 500 registered individuals by offering its services entirely for free, suggesting that monetization is not the only driver of development. According to security technologists, this ecosystem is actively growing, with various hacker groups competing to develop and release new and improved tools. In parallel to this public-facing market, a more discreet trend has emerged among skilled and well-resourced threat actors. These sophisticated groups are increasingly choosing to bypass commercial offerings entirely, opting instead to build their own proprietary AI models. By integrating these custom-built LLMs directly into their local infrastructure, they gain greater control, enhanced secrecy, and the ability to tailor the models to their specific operational needs without a third-party provider.
Why Dark LLMs Fall Short
Significant Technical Flaws
Despite their utility in assisting amateurs, the overarching consensus among cybersecurity researchers is that dark LLMs are technically unimpressive and fall far short of their hyped potential. One of the most fundamental flaws inherent in this technology is the phenomenon of “code hallucination.” This occurs when an LLM generates code that appears plausible, well-structured, and syntactically correct but is, in reality, factually incorrect, contains critical errors, or is simply non-functional. The AI can produce scripts that look like they should work but will fail upon execution, rendering them useless without significant manual correction. This unreliability is a core limitation of current generative AI, meaning that the outputs of these malicious models cannot be trusted to function as intended out of the box. This single issue significantly diminishes their value for creating anything beyond the most basic and well-documented types of malicious code. Compounding the problem of hallucination is the fact that these models lack the abstract, contextual knowledge required to create genuinely sophisticated and effective malware. Building a complex, multi-stage attack tool requires a deep understanding of network environments, operating system internals, and defensive evasion techniques—a level of abstract reasoning that current LLMs cannot replicate. They struggle to construct a fully functional, complex malware sample from scratch because they are essentially pattern-matching machines, not creative strategists. Consequently, their outputs are not fire-and-forget solutions that can be deployed autonomously. Human intervention remains absolutely essential to debug the generated code, check for hallucinations, and adapt the simplistic scripts to the specific nuances and security configurations of a target’s network environment, a task that still requires a considerable degree of human expertise.
Overblown Impact on Cybersecurity
The final and most crucial finding from recent analyses is that, despite their growing availability and the surrounding media buzz, there is a distinct lack of hard evidence to suggest that dark LLMs are having a widespread or significant impact on the overall cyber threat landscape. Senior threat intelligence directors candidly admit that it is nearly impossible to track their adoption rates with any degree of accuracy. This difficulty stems primarily from the fact that cybersecurity researchers lack the specialized tools needed to reliably detect AI’s involvement in malicious artifacts. Unless attackers explicitly reveal their methods or leave behind obvious clues, distinguishing AI-generated code from human-written code is exceptionally challenging. This evidentiary gap means that many of the dire predictions about an AI-fueled cyber-pocalypse remain purely speculative, unsupported by concrete data from real-world attacks. As a result, the much-discussed arms race between AI-generated malware and AI-powered defenses has been largely premature and, so far, has failed to materialize. Because the outputs of dark LLMs are overwhelmingly based on known malware samples and common, well-documented attack techniques, existing cybersecurity infrastructure has remained effective. These models are not innovating new threats; they are merely repackaging and automating the creation of old ones. The malware tricks, obfuscation methods, and ransom note styles they generate are tired and unoriginal, copied directly from existing artifacts and public code repositories. Security vendors already have the tools, signatures, and behavioral detection mechanisms in place to detect and mitigate the threats these models produce. The reality was that while dark LLMs lowered the barrier to entry for petty criminals, they did not produce novel threats capable of bypassing modern, established defense mechanisms.