Idea
API detecting adversarial and jailbreak prompts in large language models for AI developers and platform providers.
Research Paper
Core Innovation
This paper presents CoCoTen, a method that uses latent space features from Contextual Co-occurrence Tensors to detect adversarial inputs in LLMs. It requires minimal labeled data and significantly outperforms existing baselines in accuracy and speed. This approach enables more efficient and reliable detection compared to prior methods relying on extensive labeled datasets or slower processing.
Market Size (TAM)
$2–10B TAM, $1–2B SAM; assumption: growing adoption of LLMs in enterprises and increasing demand for AI security solutions.
Potential Customers & Pain Points
- AI Developers Needing Robust Adversarial Detection
- Enterprises Deploying Large Language Models Safely
- Security Teams Preventing Jailbreak Attacks on AI Systems
Business Model
Subscription-based API access with tiered pricing for enterprise usage and custom integration support.
Competitive Landscape
- OpenAI Moderation API
- Hugging Face Safety Tools
- AI21 Labs Security Solutions
Implementation Challenges
- Integration complexity with diverse LLM architectures
- Evolving adversarial attack techniques
- Need for continuous model updates
Validation Strategy
- Pilot integration with select AI platform providers
- Benchmark against existing adversarial detection tools
- Collect user feedback to refine detection accuracy and speed
Research Paper Overview
CoCoTen: Detecting Adversarial Inputs to Large Language Models through Latent Space Features of Contextual Co-occurrence Tensors
Summary
This paper introduces a novel method to detect adversarial and jailbreak prompts targeting Large Language Models by leveraging latent space features of Contextual Co-occurrence Matrices and Tensors. The approach achieves a high F1 score of 0.83 using only 0.5% labeled data, outperforming baselines by 96.6%, and offers significant speed improvements, making it effective and efficient for safe LLM deployment.