Idea
Sparse MoE model delivering dense-level AI performance with lower compute and storage for edge devices.
Research Paper
Core Innovation
This paper introduces DECO, a sparse MoE architecture that uses ReLU-based routing with learnable expert-wise scaling and a novel NormSiLU activation to stabilize expert activation and increase sparsity. It simplifies MoE design by using non-gated MLP experts and achieves dense-comparable performance with only 20% expert activation, plus a 3x inference speedup via a specialized kernel.
Why It Matters
End-side devices require AI models that balance high performance with low computational cost and minimal storage. DECO addresses these constraints by reducing active parameters and accelerating inference, enabling advanced AI capabilities on resource-limited hardware. This can transform workflows in mobile, IoT, and embedded systems by making powerful models more accessible and efficient.
Market Size (TAM)
$20–50B TAM for AI model deployment on edge and mobile devices; $2–10B SAM from mobile manufacturers and IoT providers. Driven by demand for efficient on-device AI and edge computing growth.
Potential Customers & Pain Points
- Mobile device manufacturers – Need efficient AI models for on-device processing
- IoT solution providers – Require low-latency low-power AI inference
- Edge computing platforms – Demand scalable models with minimal memory footprint
- AI application developers – Seek cost-effective deployment without performance loss
Business Model
Licensing the DECO architecture and acceleration kernel to device manufacturers and AI platform providers; offering consulting and integration services for optimized deployment.
Competitive Landscape
- Google Edge TPU
- NVIDIA Jetson
- Qualcomm AI Engine
- Microsoft Azure Percept
Implementation Challenges
- Integration complexity with existing AI frameworks
- Hardware compatibility and optimization challenges
- Adoption inertia due to established dense model usage
Validation Strategy
- Benchmark DECO on diverse end-side devices against dense and existing MoE models
- Partner with hardware vendors for real-world deployment pilots
- Collect performance
- power consumption
- and user feedback data to refine the model and kernel
Research Paper Overview
DECO: Sparse Mixture-of-Experts with Dense-Comparable Performance on End-Side Devices
Summary
DECO is a sparse Mixture-of-Experts architecture that matches dense Transformer performance while reducing computation and storage, enabling efficient deployment on end-side devices. It activates only 20% of experts and achieves a 3x speedup in inference with a specialized acceleration kernel.