Idea
A compression method for large language models that improves accuracy and efficiency using sparse dictionary learning and calibration data.
Research Paper
Core Innovation
This paper introduces CoSpaDi, which replaces rigid low-rank approximations with a flexible structured sparse factorization using a dense dictionary and sparse coefficients. It uniquely optimizes compression using calibration data to minimize output activation errors rather than just weight differences. This approach achieves better model fidelity without fine-tuning and supports efficient sparse computations and quantization.
Market Size (TAM)
$20–50B TAM for AI model compression and deployment; $2–10B SAM from cloud providers and enterprises deploying large language models. Driven by growing LLM adoption and demand for cost-efficient inference.
Potential Customers & Pain Points
- AI Model Developers Needing Efficient LLM Deployment
- Cloud Providers Seeking Reduced Inference Costs
- Enterprises Running Large Language Models with Limited Hardware Resources
Business Model
Licensing compression software to AI developers and cloud providers; offering consulting and integration services for LLM deployment optimization.
Competitive Landscape
- LoRA
- AdaLoRA
- SparseGPT
Implementation Challenges
- Integration with existing LLM deployment pipelines
- Hardware support for structured sparse operations
- Adoption resistance due to new compression paradigm
Validation Strategy
- Benchmark CoSpaDi on diverse LLM architectures and datasets
- Demonstrate latency and memory improvements in real-world deployments
- Collaborate with cloud providers for pilot integration and feedback
Research Paper Overview
COSPADI: Compressing LLMs via Calibration-Guided Sparse Dictionary Learning
Summary
This paper proposes CoSpaDi, a training-free compression framework for large language models that replaces low-rank weight approximation with structured sparse dictionary learning. It represents weight matrices using a dense dictionary and a column-sparse coefficient matrix, enabling a union-of-subspaces representation for greater expressiveness. CoSpaDi uses a small calibration dataset to minimize functional reconstruction error by matching output activations of compressed layers to original ones, preserving model accuracy without fine-tuning. The method supports efficient sparse-dense multiplication and post-training quantization, showing superior accuracy and perplexity over state-of-the-art low-rank methods across multiple Llama and Qwen models at 20-50% compression ratios.