Idea
Testing platform catching pre-production bugs in large-scale backend systems to prevent serious failures and reduce review overhead.
Research Paper
Core Innovation
This paper introduces Just-in-Time catching test generation that focuses on generating tests meant to fail to reveal bugs before code integration. It leverages code-change-aware methods to improve candidate catch generation significantly over traditional hardening tests and uses rule-based and LLM-based assessors to reduce false positives and human review effort.
Why It Matters
Large-scale backend systems face critical risks from undetected bugs that can cause serious failures in production. This approach surfaces bugs before code lands, reducing costly post-deployment issues and improving software reliability. It scales to hundreds of millions of lines of code, making it suitable for industrial applications with high development velocity.
Market Size (TAM)
$10–20B TAM for software testing and quality assurance tools; $2–5B SAM from large enterprises and tech companies. Driven by increasing software complexity and demand for faster, reliable deployment.
Potential Customers & Pain Points
- Large tech companies – Need to prevent production failures in massive codebases
- Software development teams – Need to reduce false positive test failures and human review load
- Enterprises with complex backend systems – Need scalable bug detection before deployment
Business Model
Subscription-based SaaS platform targeting large enterprises and tech companies, with tiered pricing based on codebase size and usage volume.
Competitive Landscape
- Snyk
- SonarQube
- Coverity
- DeepCode
- Diffblue
Implementation Challenges
- Integration complexity with existing development pipelines
- Balancing false positive reduction with bug detection sensitivity
- Adoption resistance due to changes in developer workflows
Validation Strategy
- Pilot deployments with large-scale backend teams to measure bug detection rates and false positive reduction
- User feedback collection to optimize assessor accuracy and developer experience
- Longitudinal studies to track reduction in production failures and development drag
Research Paper Overview
Just-in-Time Catching Test Generation at Meta
Summary
This paper presents Just-in-Time catching test generation at Meta, designed to detect bugs in large-scale backend systems before code lands. It improves bug detection efficiency by using code-change-aware methods and reduces false positives with rule-based and LLM-based assessors, significantly lowering human review effort and preventing serious production failures.