Idea
Visual token pruning platform cutting cloud MLLM inference costs by 87.5% while maintaining 95% accuracy for edge-cloud AI.
Research Paper
Core Innovation
This paper introduces LAST, which leverages the last query token's attention in a compact edge-side vision-language model to guide query-dependent visual token pruning before cloud inference. Unlike prior methods, it requires no cloud model access or complex aggregation, enabling efficient, training-free token selection that balances accuracy and computational cost.
Why It Matters
Edge devices face high costs transmitting dense visual tokens to cloud MLLMs, limiting scalable multimodal AI applications. LAST reduces cloud inference load by pruning irrelevant tokens early, preserving accuracy and lowering latency. This efficiency enables broader deployment of multimodal AI in resource-constrained environments, transforming workflows in vision-based cloud services.
Market Size (TAM)
$10–20B TAM for cloud AI inference optimization; $2–5B SAM from edge-cloud multimodal AI deployments. Driven by rising multimodal AI adoption and edge computing growth.
Potential Customers & Pain Points
- Cloud AI providers – High inference costs
- Edge device manufacturers – Limited compute and bandwidth
- Enterprises deploying multimodal AI – Need scalable cost-effective inference
- Autonomous systems – Require low-latency visual processing.
Business Model
Licensing the LAST pruning technology as an edge-side SDK or API to cloud AI providers and device manufacturers, with potential for SaaS-based inference cost reduction services.
Competitive Landscape
- Token pruning frameworks
- Edge-cloud AI optimization platforms
- Multimodal inference accelerators
Implementation Challenges
- Integration with diverse cloud MLLMs
- Edge device hardware variability
- Adoption resistance due to workflow changes
Validation Strategy
- Benchmark LAST on real-world edge-cloud AI deployments
- Partner with cloud AI providers for pilot integration
- Measure cost savings and accuracy retention in production scenarios
Research Paper Overview
LAST: The Last Query Token Guides Visual Token Pruning for Edge-Cloud Collaborative MLLM Inference
Summary
LAST is a training-free framework that uses the last query token's attention to prune visual tokens on edge devices before cloud MLLM inference, reducing cloud computation while preserving accuracy. It enables query-dependent token selection without accessing the cloud model or costly operations, achieving strong performance across benchmarks with minimal token retention and low overhead.