Idea
Dense retrieval platform enhancing short-video search relevance by integrating diverse positive signals to reduce filter bubbles and improve user engagement.
Research Paper
Core Innovation
This paper presents CroPS, a retrieval data engine that incorporates cross-perspective positive samples from query reformulations, system engagement, and knowledge-level signals. It introduces a Hierarchical Label Assignment strategy and H-InfoNCE loss to optimize relevance-aware training, outperforming existing dense retrieval methods in large-scale commercial short-video search.
Why It Matters
Short-video platforms face challenges in retrieving relevant content due to reliance on limited historical user interactions, causing narrow search results and poor user experience. CroPS expands training signals with diverse positive samples, improving retrieval accuracy and reducing user query reformulations. This scalable approach enhances content discovery and engagement for hundreds of millions of users daily.
Market Size (TAM)
$10–20B TAM for video search and recommendation platforms; $2–5B SAM from short-video and social media companies. Driven by rapid growth in short-video consumption and demand for improved content discovery.
Potential Customers & Pain Points
- Short-video platforms – Limited retrieval relevance and user engagement
- Search engine providers – Filter bubble bias reducing content diversity
- Content recommendation systems – Inadequate training signals limiting discovery.
Business Model
Licensing the CroPS retrieval engine as a SaaS platform or API to video platforms and search providers, with tiered pricing based on query volume and feature set.
Competitive Landscape
- Google Video Search
- TikTok Search
- YouTube Search
- Bing Video Search
Implementation Challenges
- Integration complexity with existing search infrastructure
- Dependence on large-scale user interaction data
- Balancing diverse positive signals without noise
Validation Strategy
- Conduct A/B testing on partner short-video platforms to measure retrieval accuracy and user engagement improvements
- Benchmark against leading dense retrieval baselines on large-scale datasets
- Gather customer feedback to refine integration and feature offerings
Research Paper Overview
CroPS: Improving Dense Retrieval with Cross-Perspective Positive Samples in Short-Video Search
Summary
CroPS introduces diverse positive training samples from multiple perspectives to improve dense retrieval in short-video search, reducing filter bubble effects and enhancing relevance. It leverages user query reformulations, engagement data, and large language model knowledge, optimized via a hierarchical label assignment and H-InfoNCE loss. Deployed at scale on Kuaishou Search, it significantly boosts retrieval performance and lowers query reformulation rates.