Lenny's Podcast: Product | Career | Growth

Why experts writing AI evals is creating the fastest-growing companies in history | Brendan Foody (CEO of Mercor)

September 18, 2025

Key Takeaways Copied to clipboard!

  • The "era of evals" is critical for AI model development, acting as the product requirement document and sales collateral by defining and demonstrating model capabilities. 
  • Mercor's rapid growth from $1 to $400 million in revenue run rate in 16 months highlights a significant market shift towards sourcing and vetting high-caliber professionals to evaluate and interpret AI model capabilities. 
  • The future of work will involve humans leveraging AI to enhance productivity, creating new job categories and increasing demand in fields like software development and product management, rather than widespread job displacement. 
  • The continuous need for expert-created evaluations and data is crucial for improving AI models, even as training methodologies evolve. 
  • While superintelligence may be further off than some predict, the next decade will see significant automation of knowledge work and advancements in areas like medicine, law, and education driven by AI. 
  • Building and launching products, rather than just consuming information, is the most effective way to drive innovation in the current AI landscape. 

Segments

The Era of Evals
Copied to clipboard!
(00:00:05)
  • Key Takeaway: Evals are the new product requirement documents for AI models, crucial for measuring success and guiding development.
  • Summary: The conversation introduces the concept of ’evals’ as a critical new phase in AI development, where the focus shifts to measuring model capabilities. This is driven by the need for AI labs to understand what success looks like for their models, making evals essential for both research and demonstration.
Mercor’s Rapid Growth
Copied to clipboard!
(00:10:06)
  • Key Takeaway: Mercor’s unprecedented growth is fueled by identifying and serving the critical need for high-caliber human experts to train and evaluate AI models.
  • Summary: Brendan Footie discusses Mercor’s explosive growth, attributing it to a strategic pivot from general labor to specialized expertise for AI labs. He highlights the transition from crowdsourcing basic data to sourcing professionals who can rigorously evaluate and improve complex model capabilities, leading to a significant market opportunity.
Future of Work & Skills
Copied to clipboard!
(00:17:10)
  • Key Takeaway: The rise of AI will create new job categories and increase demand in elastic industries, emphasizing the importance of learning to leverage AI tools.
  • Summary: The discussion explores the impact of AI on the labor market, moving beyond job displacement narratives to focus on new opportunities. Brendan emphasizes that skills in leveraging AI tools will be paramount, and industries with elastic demand, like software development, will see increased opportunities as AI boosts productivity.
Building a Legendary Company
Copied to clipboard!
(00:39:01)
  • Key Takeaway: Mercor’s success is built on ambitious goals, exceptionally high standards for talent, and an intense, output-oriented culture.
  • Summary: Brendan outlines the core tenets behind Mercor’s hypergrowth: setting audacious goals (‘can-do attitude’), maintaining a rigorous hiring bar (‘high standards’), and fostering an intense, results-driven work environment (‘intensity’). He also touches on the importance of customer obsession and identifying market leading indicators.
The Future of AI Evals
Copied to clipboard!
(00:57:33)
  • Key Takeaway: Expert-created evaluations and post-training data are essential for continued AI model improvement and capability development.
  • Summary: The discussion emphasizes that evaluations are evergreen and will always be necessary to measure model success across various domains, even as training methods change.
AI Progress and Superintelligence
Copied to clipboard!
(00:58:01)
  • Key Takeaway: The path to superintelligence is a longer road than some executives predict, with progress relying more on data-efficient post-training datasets than simply more pre-training data.
  • Summary: The speakers agree that the pace of AI progress might be slowing down regarding superintelligence, but acknowledge the extraordinary capabilities of current models and the potential for automating knowledge work within the next decade.
AI’s Societal Impact
Copied to clipboard!
(00:59:44)
  • Key Takeaway: AI is poised to create significant abundance, leading to improvements in healthcare, legal access, product development, and education over the next ten years.
  • Summary: The conversation highlights the positive societal impacts of AI, including increased access to information and personalized learning experiences, drawing parallels to the evolution of online educational resources.
Personal AI Use and Innovation
Copied to clipboard!
(01:00:48)
  • Key Takeaway: AI tools like ChatGPT can serve as valuable thought partners for problem-solving and creative tasks, and the ease of AI development encourages proactive building and customer engagement.
  • Summary: The speaker shares personal uses of AI for writing and problem-solving, discusses the potential of voice-based AI assistants like Parrot GPT, and encourages listeners to take initiative in building AI products.
Lightning Round and Life Philosophy
Copied to clipboard!
(01:03:24)
  • Key Takeaway: Focusing on leveraging individual strengths, rather than solely on improving weaknesses, is a key management principle that can lead to greater creativity and success.
  • Summary: The lightning round covers book recommendations, favorite media, product discoveries, and a life motto, culminating in a discussion about the speaker’s dyslexia and how it has informed their management philosophy of focusing on strengths.