Idea
A framework and dataset that improve large language models' multi-step data analysis for data scientists and AI developers.
Research Paper
Core Innovation
This paper introduces NbQA, a large dataset of tool-based data analysis tasks extracted from Jupyter notebooks, enabling better training and evaluation. It also presents Jupiter, a novel framework that formulates data analysis as a search problem using Monte Carlo Tree Search to generate diverse solution paths, improving multi-step reasoning beyond prior LLM approaches.
Market Size (TAM)
$2–10B TAM, $1–2B SAM; assumption: growing demand for AI-powered data analysis and automation in enterprises and research.
Potential Customers & Pain Points
- Data Scientists Needing Reliable Multi-Step Reasoning
- AI Developers Seeking Enhanced Tool Use in LLMs
- Enterprises Requiring Automated Complex Data Analysis
- Researchers Working on LLM Benchmarking and Evaluation
Business Model
Subscription-based API access to Jupiter framework and NbQA dataset licensing for enterprise and research use.
Competitive Landscape
- OpenAI GPT-4o
- Google Bard
- Anthropic Claude
Implementation Challenges
- Complexity of integrating search-based reasoning with LLMs
- Data quality and diversity in extracted notebook tasks
- Computational cost of Monte Carlo Tree Search during inference
Validation Strategy
- Benchmark Jupiter against GPT-4o and other agents on diverse data analysis tasks
- Pilot deployments with data science teams to measure productivity gains
- Collect user feedback to refine search strategies and dataset coverage
Research Paper Overview
Jupiter: Enhancing LLM Data Analysis Capabilities via Notebook and Inference-Time Value-Guided Search
Summary
Large language models struggle with multi-step reasoning and tool use in complex data analysis. This paper introduces a scalable pipeline to extract high-quality, tool-based data analysis tasks and solutions from Jupyter notebooks, creating NbQA, a large dataset of task-solution pairs. It also presents Jupiter, a framework that formulates data analysis as a search problem using Monte Carlo Tree Search to generate diverse solution paths and improve multi-step reasoning. Experiments show strong performance surpassing GPT-4o and advanced agents on diverse reasoning tasks.