Idea
A platform enabling integration of black-box large language models via API queries for improved combined model performance.
Research Paper
Core Innovation
This paper introduces Evo-Merging, a novel method for merging black-box large language models using only inference APIs. It innovates by applying sparsity-based denoising to remove redundant information and sign-aware scaling to dynamically weight models, overcoming the limitation of inaccessible model parameters. This enables effective model integration without requiring internal model details.
Market Size (TAM)
$20–50B TAM for AI model integration platforms; $2–10B SAM from enterprises using multiple LLM APIs. Driven by growing adoption of LLM services and demand for unified AI capabilities.
Potential Customers & Pain Points
- AI developers lacking access to model weights
- Enterprises using multiple LLM APIs needing unified capabilities
- Language-Model-as-a-Service providers seeking model combination without parameter sharing
Business Model
Subscription-based API platform charging for model merging services and query usage; Enterprise licensing for large-scale deployments.
Competitive Landscape
- Hugging Face
- OpenAI
- Cohere
Implementation Challenges
- Limited access to model internals restricts merging techniques
- High cost of API queries for optimization
- Complexity in scaling to very large model repositories
Validation Strategy
- Benchmark merged models on diverse NLP tasks against baselines
- Pilot integration with LLM API providers and enterprise users
- Measure cost-efficiency and performance improvements in real-world scenarios
Research Paper Overview
Black-box Model Merging for Language-Model-as-a-Service with Massive Model Repositories
Summary
This paper addresses the challenge of merging large language models provided as black-box APIs by proposing Evo-Merging, a derivative-free optimization framework using evolutionary algorithms. It features sparsity-based denoising to filter irrelevant information and sign-aware scaling to optimize model combination weights. The approach achieves state-of-the-art results without requiring access to model parameters.