Idea
A scalable multicalibration algorithm improving model fairness and performance for machine learning practitioners and enterprises
Research Paper
Core Innovation
This paper introduces MCGrad, a multicalibration method that does not require manual subgroup specification and scales to web-scale datasets. Unlike prior methods, MCGrad maintains or improves other performance metrics such as log loss and PRAUC. It has been successfully deployed in production at Meta, demonstrating practical scalability and effectiveness.
Market Size (TAM)
$20–50B TAM for AI fairness and model calibration tools; $2–10B SAM from large enterprises and tech companies deploying ML models. Driven by regulatory pressure and demand for ethical AI.
Potential Customers & Pain Points
- Enterprises deploying ML models needing fair and calibrated predictions
- ML practitioners struggling with subgroup specification
- Companies requiring scalable fairness solutions without performance trade-offs
Business Model
Offer MCGrad as a SaaS API or integrated platform for ML model fairness and calibration; enterprise licensing and consulting services
Competitive Landscape
- Fairlearn
- AIF360
- Google What-If Tool
Implementation Challenges
- Integration complexity with existing ML pipelines
- Convincing enterprises to adopt new fairness methods
- Handling diverse and evolving data subgroups
Validation Strategy
- Pilot deployments with select enterprise customers
- Benchmark against existing multicalibration tools on public datasets
- Collect feedback and performance metrics from production use cases
Research Paper Overview
MCGrad:: Multicalibration at Web Scale
Summary
We propose MCGrad, a novel and scalable multicalibration algorithm. Multicalibration - calibration in sub-groups of the data - is an important property for the performance of machine learning-based systems. Existing multicalibration methods have thus far received limited traction in industry. We argue that this is because existing methods (1) require such subgroups to be manually specified, which ML practitioners often struggle with, (2) are not scalable, or (3) may harm other notions of model performance such as log loss and Area Under the Precision-Recall Curve (PRAUC). MCGrad does not require explicit specification of protected groups, is scalable, and often improves other ML evaluation metrics instead of harming them. MCGrad has been in production at Meta, and is now part of hundreds of production models. We present results from these deployments as well as results on public datasets.