Aginiti Redteam – Adaptive AI Security Red-Teaming Engine
An open-source adaptive red-teaming engine for agentic AI and RAG systems that implements published attack research and outperforms fixed-checklist approaches by re-ranking attacks against a learned evidence model.
Comprehensive Technical Write-Up Available
Explore the peer-reviewed methodology, system architecture diagrams, benchmark comparisons, and academic citations.
Project Overview
Aginiti Redteam is an open-source adaptive red-teaming engine for agentic AI and RAG systems, built as a Forward-Deployed AI Engineering Intern at DevNeuron under Dr. Ahmad Faraz Khan. Rather than following a fixed attack script, it keeps an evolving model of everything it has learned about a target and re-ranks candidate attacks at every step using a weighted scoring formula (expected new information, potential impact, prior failures), mapping every finding to the OWASP LLM Top 10. It implements four attacks drawn directly from published security research spanning three threat models: IKEA (ICLR 2026) and SECRET (IEEE TIFS 2026) for RAG knowledge-base reconstruction, the Interrogation Attack (ACM CCS 2025) for membership inference, and SPE-LLM (ICLR 2026) for system-prompt extraction. Every finding is verified through a three-step process: a plain-text scan for obvious matches, an LLM check against a specific list of what a real leak looks like, and a final direct comparison against the actual source data.
Problem It Solves
Manual or fixed-checklist red-teaming of agentic AI and RAG systems is slow, generates excessive false positives from the target's own defenses, and does not adapt as new information about a target is uncovered. Enterprises deploying RAG and agentic systems need a systematic, reproducible way to discover real vulnerabilities mapped to recognized frameworks (OWASP LLM Top 10), with findings that are independently verified rather than taken on a single model's word.
Tools & Technologies Used
Skills Involved
Challenges
The central challenge was designing an adaptive planner that could reason over an evolving evidence graph rather than a static script, while keeping the architecture provider-agnostic across multiple LLM backends. Verifying findings rigorously enough to avoid false positives required a three-step verification pipeline (pattern scan, LLM-based check, direct source comparison) rather than trusting a single model's judgment. Benchmarking against a hardened target running 8 independent defenses (access control, output filtering, rate limiting, and more) also required building a realistic enterprise-grade test environment rather than an undefended lab setup.
Learnings
This project deepened my understanding of translating published, paper-level security research into working, reproducible attacks, and of designing rigorous verification pipelines that hold up against an industry-standard scanner (NVIDIA's garak). I gained hands-on experience with adaptive planning under uncertainty, provider-agnostic system design, and benchmarking methodology (attempts-to-success, filter-trigger rates) that produces credible, comparable evidence of real security impact.