trace hunter labs day 2 - teaching AI agents how to hack

Last post

The last post was a manifesto. This one is more of a build log.

The first step in building out systems to use AI for security is building out a lab. The agents will need something to scan, something to test malware against, something to find vulnerabilities in.

I worked with Codex + Hermes to develop a plan. The big delineation here is which parts are "agentic" (run by or primarily concerning the AI, which operates non-deterministically) and which parts are programmatic (deterministic and linear). The core decision we arrived at is a general approach of "we build programmatic systems whose parts are usable by the agents." This means we're building tools for the agent to use, but those tools should execute deterministically: running a given tool with a given input should give you a deterministic/set output.

So, instead of telling an agent "you have operating system privileges, set up a windows machine and find 0days" you say "here is a tool to set up the Windows virtual machine according to a known-good approach with these goal sets in mind." The agent can then determine when it should set up that machine, and can configure it in a deterministic way.

The approach starts with creating some ground rules and configuration files. We need the experiments (called cases) to be fairly reproducible. If we're trying to determine a way to teach AI agents to hack systems, we need those systems to be pretty reproducible. If we're trying to teach agents how to reverse engineer malware, we need that malware to be able to transfer to the right target systems.

I created a basic schema for config files that defines the set up and tear down protocol, or how to initialize the system and how to destroy it. I also created a way to inject commands on the system while it's running. Finally, I created a way for the system to be connected to the SIEM.

A SIEM is a Security Information and Event Management platform. It takes all of the logs and event information from a system, centralizes it and makes it easy to search and analyze. This will be super important later when we're doing vulnerability research and malware exploration, but it's important now to have a system that has logs of what we're doing to confirm that we are, in fact, running the commands that we're supposed to. Later, this will be important in a lot of things:

• Confirming that an AI isn't hallucinating running commands (this happens a lot)
• Confirming that the commands ran successfully and had a desired impact
• Training data for future models - when training models, you need to have data that shows the action an agent took and as much information about the result as possible. So showing "the agent ran this command, then it had that effect on the system" will be vital.

So essentially these YAML config files tell the Runner system how to set up a system, how to connect it to the virtual network, how to connect it to the SIEM and what commands to run. Then it tells the runner how to shut down the system after it's done running.

The runner manages the whole lifecycle of a case. Cases are based on immutable case files (those YAML configs) and if a case file is changed, it's considered a new "version" of that case. Runs are archived according to cases and their versions, and are instances of a run. Basically what this means is that any given case should never change, and if it does it becomes a new case that's associated with the other version. Runs are instances of a case being run, and they'll comprise all the logs and stored information on that run.

This part is working, for the most part. There are some name collision issues that make it to where connecting to the SIEM is more difficult, but it works, for the most part. After that, I'll be giving tool call permissions to the agents so that they can have limited interactions with the operating system. That'll be a solid end point for the day.

For now, though, I have a lot of code to review...