This article discusses how Craig, a penetration tester from Black Hills Information Security, uses AI, specifically GitHub Copilot, to enhance his pentesting process. He shares experiences in using AI for payload generation and vulnerability analysis, detailing the successes and challenges he faced, while advocating for thoughtful, ethical usage of AI in real-world applications.
In the second installment of his series on enhancing penetration testing with AI, Craig, a seasoned red teamer at Black Hills Information Security, shares practical insights from his experience using GitHub Copilot. With a background in software development, Craig’s unique perspective focuses on employing large language models (LLMs) for improving productivity in pentesting tasks. He initially leveraged AI for generating out-of-band command injection payloads, utilizing a variety of sources like PayloadAllTheThings to build a comprehensive list, modifying them as necessary to fit his testing needs.
Craig’s methodology takes a tech-savvy twist as he details how he crafted a Python script to automate the tedious process of swapping out specific IPs and URLs for a Burp Suite Collaborator server. Using Visual Studio Code, he simply instructed Copilot to replace those addresses with placeholder text, enabling him to quickly generate modified payloads. After a few moments, Craig ran the script and confirmed that it performed well, turning lengthy procedures into mere seconds of coding magic—a genuine time-saver in an often convoluted field.
Building on this success, he challenged Copilot once again, asking for a script to link two files—one containing placeholders and another filled with Collaborator URIs. The output was as expected, with newly crafted payloads ready for injection tests. Not the most glamorous of tasks, but refreshing to see how AI can lighten the load for pentesters—making what once took ages a flash in the pan.
However, the journey with Copilot wasn’t all smooth sailing. When Craig attempted to explore potential vulnerabilities in the Juice Shop’s code, he discovered that AI could be quite strict about what it would share. He quickly learned that a little creativity in his prompts could coax out valuable insights. By asserting his role as an ethical researcher, he managed to get Copilot to list some JavaScript methods that could be dangerous and flagged mentions of eval() and innerHTML within the code.
Pushing the envelope further, Craig experimented with some advanced jailbreak techniques he’d encountered online. This led to Copilot providing more in-depth analysis of exploitable vulnerabilities—albeit with some playful interruptions in its responses. Each time he pressed for more detail, particularly for proof of concept (PoC) code, he faced a bit of resistance. It was like negotiating through layers of red tape.
After refining his requests and reapplying jailbreak prompts, Craig finally managed to extract thorough exploit instructions from the model. This is a crucial point—while the Juice Shop is built with vulnerabilities for educational purposes, one must tread carefully in real-world testing scenarios, protecting sensitive client data. Ideally, he noted, using a local LLM would be more prudent when employing AI in genuine penetration tests.
In closing, Craig hopes his journey through AI-assisted pentesting proves valuable to others in the field, simplifying workflows and offering new efficiencies. He encourages fellow pentesters to adapt their methods and consider incorporating AI in their toolkits as they tackle increasingly complex security challenges. Interested parties should mark their calendars for his upcoming webcast on integrating AI into pentesting methodologies, offering further exploration of this promising intersection.
Craig’s exploration of AI in penetration testing highlights how tools like GitHub Copilot can streamline tedious processes, from payload generation to vulnerability assessment. While navigating the quirks of AI presents challenges, the potential to enhance efficiency and effectiveness is clear. As he emphasizes, a responsible approach, especially in the context of client work, remains paramount. By blending technology with traditional skills, penetration testers can continuously evolve in an ever-changing digital landscape.
Original Source: www.blackhillsinfosec.com
