Idea
Multi-task safety classification models reducing LLM content risks with efficient, scalable real-time detection of toxicity and harmful prompts.
Research Paper
Core Innovation
This paper presents Opir, a multi-task encoder-based safety classification system built on GLiClass architecture, trained on a comprehensive taxonomy with nearly 1000 categories. It combines diverse training data including adversarial negatives and multilingual examples, delivering competitive performance with fewer parameters than existing models. Opir also provides open-source evaluation tools supporting multiple backends.
Why It Matters
Real-time safety filtering is critical for deploying large language models responsibly, preventing harmful or unsafe content from reaching users. Opir's efficient models reduce computational costs and enable scalable safety monitoring across diverse content types and languages. This improves user trust and compliance while lowering operational overhead for AI service providers.
Market Size (TAM)
$2–10B TAM for AI content safety and moderation; $500M–$1B SAM from AI platform providers and social media companies. Driven by increasing regulatory scrutiny and demand for responsible AI deployment.
Potential Customers & Pain Points
- AI platform providers – Need scalable cost-effective safety filters
- Social media companies – Require real-time detection of toxic and harmful content
- Enterprises deploying LLMs – Need reliable guardrails to prevent misuse
- Developers of conversational AI – Need lightweight models for edge deployment.
Business Model
Subscription-based API access for real-time safety classification; licensing of edge model variants for on-premise deployment; consulting and customization services for enterprise clients.
Competitive Landscape
- GLiNER2
- OpenAI Moderation API
- Perspective API
- HateSonar
- Google Jigsaw
Implementation Challenges
- Rapidly evolving adversarial attack techniques requiring continuous model updates
- Balancing false positives and negatives to avoid over-censorship or missed harmful content
- Integration complexity with diverse LLM platforms and deployment environments
Validation Strategy
- Benchmark Opir against leading safety classification models on public datasets
- Pilot deployments with AI platform providers to measure real-world performance and cost savings
- Collect user feedback on false positive/negative rates and adjust taxonomy coverage
- Expand multilingual and adversarial robustness testing
Research Paper Overview
Opir: Efficient Multi-Task Safety Classification for Toxicity, Jailbreaks, Hate Speech, and Harmful Content
Summary
Opir introduces a family of encoder-based guardrail models for real-time safety filtering in LLM applications, detecting unsafe prompts, toxic language, jailbreak attempts, and harmful content with a smaller deployment footprint. It supports multi-task classification across a detailed taxonomy and offers edge variants under 100M parameters. Opir outperforms or matches leading open-weight baselines on multiple safety benchmarks while enabling efficient, scalable safety monitoring.