Artificial intelligence companies are tightening safety controls to prevent hackers from misusing powerful AI models, but cybersecurity experts say those same restrictions are increasingly making it harder for legitimate security researchers to protect digital systems.
Researchers involved in offensive cybersecurity—whose job is to discover vulnerabilities before criminals do—argue that overly strict AI guardrails are slowing vulnerability research, exploit analysis and defensive security work, while pushing many professionals toward unrestricted open-source AI models.
Why AI Companies Introduced Guardrails
Over the past year, companies such as OpenAI and Anthropic have introduced stricter safeguards to prevent AI models from being used for:
- Malware development
- Cyberattacks
- Exploit generation
- Phishing campaigns
- Unauthorized hacking
To access fewer restrictions, approved organizations must join special programs such as:
- OpenAI’s Trusted Access for Cyber
- Anthropic’s Cyber Verification Program
These initiatives are designed to ensure that advanced AI capabilities are only available to vetted cybersecurity professionals.
Researchers Say Guardrails Also Block Defensive Work
Security experts argue that cybersecurity is unique because the same techniques used to attack systems are also necessary to defend them.
When researchers discover a software bug, they often need AI assistance to determine whether that vulnerability can actually be exploited.
If AI refuses to analyze or demonstrate the exploit because of safety filters, researchers say it becomes significantly harder to verify security flaws.
Chris Anley, Chief Scientist at cybersecurity firm NCC Group, explained that asking AI to exploit a bug is often the fastest way to confirm whether developers need to fix it.
According to him, offensive and defensive cybersecurity are inseparable.
“The same tool that helps attackers also helps defenders.”
He compared AI to a hammer—capable of building a house or being used as a weapon depending on who holds it.
Guardrails Pushing Researchers Toward Open Models
Several cybersecurity professionals say that when commercial AI models refuse legitimate security requests, they increasingly switch to open-source AI models running locally.
Unlike cloud-based frontier models, open-source models generally:
- Have no cybersecurity restrictions
- Can be modified freely
- Operate entirely offline
- Avoid sending sensitive research to external servers
Researchers argue that this gives them more consistent assistance without risking confidential vulnerability data.
Concerns Over Cloud Privacy
Another reason many security researchers avoid using commercial AI for vulnerability discovery is confidentiality.
Paolo Stagno, Chief Technology Officer at Crowdfense, said his team avoids uploading sensitive exploit research to cloud-based AI systems because doing so could expose valuable security information.
Instead, his team uses locally hosted open-source models for offensive research while relying on frontier AI mainly for reverse engineering tasks.
Some Researchers Prefer Human Expertise
Not every security expert wants AI to automate vulnerability discovery.
Independent researcher Giuseppe Cali said he uses AI only to understand unfamiliar code and build supporting tools.
Finding vulnerabilities and developing exploits remains something he prefers to do himself.
As he put it:
“I still want to own the actual bug discovery.”
He believes AI should accelerate research—not replace human expertise.
Inconsistent AI Responses Create Frustration
Another major complaint is inconsistency.
Chris Thompson, CEO of cybersecurity company RemoteThreat, said frontier AI models often produce different responses to identical security questions.
Sometimes the models provide useful technical analysis.
Other times they suddenly refuse to continue because internal safety systems classify the request differently.
Researchers say this forces them to spend valuable time negotiating with AI instead of investigating vulnerabilities.
US Restrictions Driving Researchers Elsewhere
Some experts warn that excessive restrictions may unintentionally encourage researchers to rely on unrestricted foreign AI models.
Chinese open-source models such as GLM have become increasingly popular among cybersecurity professionals because they:
- Require no approval process
- Have minimal safety restrictions
- Can run completely offline
- Allow full control over security analysis
According to Thompson, legitimate researchers are gradually being pushed away from heavily regulated American AI systems.
Debate Over AI Safety Continues
The discussion comes amid growing concern about AI-powered cyber threats.
Governments and AI companies fear advanced models could eventually enable:
- Automated cyberattacks
- Large-scale phishing
- Vulnerability discovery at unprecedented speed
- Malware generation
Cybersecurity researchers, however, argue that defenders need access to equally capable AI tools if they are expected to keep pace with increasingly sophisticated attackers.
Many believe the solution is not removing safety measures entirely, but creating more reliable and transparent access systems for verified security professionals.
Balancing Security and Innovation
Experts say AI companies now face a difficult balancing act.
On one hand, they must prevent malicious actors from weaponizing powerful AI systems.
On the other, excessive restrictions risk slowing the work of ethical researchers responsible for discovering and fixing security flaws before cybercriminals can exploit them.
As AI becomes more capable, finding that balance may prove critical for the future of cybersecurity.
Author
About the Author
