We build runtime security for AI agents. The playground started as an internal tool that we used to test our own guardrails. But we kept finding the same types of vulnerabilities because we think about attacks a certain way. At some point you need people who don't think like you.
So we open-sourced it. Each challenge is a live agent with real tools and a published system prompt. Whenever a challenge is over, the full winning conversation transcript and guardrail logs get documented publicly.
Building the general-purpose agent itself was probably the most fun part. Getting it to reliably use tools, stay in character, and follow instructions while still being useful is harder than it sounds. That alone reminded us how early we all are in understanding and deploying these systems at scale.
First challenge was to get an agent to call a tool it's been told to never call.
Someone got through in around 60 seconds without ever asking for the secret directly (which taught us a lot).
Next challenge is focused on data exfiltration with harder defences: https://playground.fabraix.com
Comments URL: https://news.ycombinator.com/item?id=47392677
Points: 6
# Comments: 0
The company is reportedly delaying the launch as its engineers and lawyers work to avert further legal issues.
Aether OS puts a full-fledged desktop in your browser that ties directly into the AT Protocol. That means it connects to your Bluesky account and other public records. It offers a pretty full suite of apps, 42 in total, covering text editing, task management, and social media. There's even a rudimentary tracker for making chiptunes, […]