Episode 15: Why Good Metrics Still Lead to Bad Decisions — and How to Fix It
Eoin O'Mahony—data science partner at Lightspeed, former Uber science lead, and one of the early architects of the system that kept NYC’s Citi Bikes available across the city—argues that positive metrics are meaningless if you don’t understand the mechanism behind them. At Uber, he was careful to make sure his launches both looked good on paper and made sense in practice. Now in venture, he’s applying that same rigor to unstructured data—using GenAI to scale a kind of work that’s long resisted systematization.
LINKS
Eoin's page at Lightspeed Ventures (https://lsvp.com/team-member/eoin-omahony/)
Ramesh Johari on How to Build an Experimentation Machine and Where Most Go Wrong (https://high-signal.delphina.ai/episode/ramesh-johari-on-how-to-build-an-experimentation-machine-and-where-most-go-wrong)
Chiara Farronato on Data Science Meets Management: Teamwork, Experimentation, and Decision-Making (https://high-signal.delphina.ai/episode/data-science-meets-management)
Delphina's Newsletter (https://delphinaai.substack.com/)
--------
54:17
Episode 14: Why Most Companies Aren’t Actually AI Ready (and What to Do About It)
Barr Moses—co-founder and CEO of Monte Carlo—thinks we’re headed for an AI reckoning. Companies are building fast, but most are still managing data like it’s 2015. In this episode, she shares high-stakes failure stories (like a $100M schema change), explains why full-stack observability is becoming essential, and breaks down how LLM agents are already transforming data debugging. From culture to tooling, this is a sharp look at what real AI readiness requires—and why so few teams have it.
LINKS
2024 State of Reliable AI Survey – Monte Carlo (https://www.montecarlodata.com/blog-2024-state-of-reliable-ai-survey/)
Delphina's Newsletter (https://delphinaai.substack.com/)
Unity’s $100M Data Error – Schema Change Gone Wrong (https://www.theregister.com/2021/11/11/unity_stock_plunge/)
Citibank’s $400M Fine for Risk Management Failures (https://www.reuters.com/article/us-citigroup-fine-idUSKBN26T0BK)
Google’s AI Recommends Adding Glue to Pizza (https://www.theverge.com/2024/5/23/24162896/google-ai-overview-hallucinations-glue-in-pizza)
Chevy Dealer’s AI Chatbot Agrees to Sell Tahoe for $1 (https://incidentdatabase.ai/cite/622/)
The AI Hierarchy of Needs by Monica Rogati (HackerNoon) (https://hackernoon.com/the-ai-hierarchy-of-needs-18f111fcc007)
Data Quality Fundamentals by Barr Moses, Lior Gavish, and Molly Vorwerck (O’Reilly) (https://www.oreilly.com/library/view/data-quality-fundamentals/9781098112035/)
Delphina's Newsletter (https://delphinaai.substack.com/)
--------
51:58
Episode 13: The End of Programming As We Know It
Tim O’Reilly—founder of O’Reilly Media and one of the most influential voices in tech—argues we’re not witnessing the end of programming, but the beginning of something far bigger. He draws on past computing revolutions to explore how AI is reshaping what it means to build software, why real breakthroughs come from the edge—not incumbents—and what it takes to learn, teach, and build responsibly in the age of AI.
LINKS
The End of Programming as We Know It by Tim <--- Read this! (https://www.oreilly.com/radar/the-end-of-programming-as-we-know-it/)
WTF? What’s the Future and Why It’s Up to Us (https://www.oreilly.com/tim/wtf-book.html)
The fundamental problem with Silicon Valley’s favorite growth strategy (https://qz.com/1540608/the-problem-with-silicon-valleys-obsession-with-blitzscaling-growth)
AI Engineering by Chip Huyen (https://www.oreilly.com/library/view/ai-engineering/9781098166298/)
Delphina's Newsletter (https://delphinaai.substack.com/)
--------
1:23:09
Episode 12: Your Machine Learning Solves The Wrong Problem
Stefan Wager—Professor at Stanford and expert on causal machine learning—has worked with leading tech companies including Dropbox, Facebook, Google, and Uber. He challenges the widespread assumption that better predictions mean better decisions. Traditional machine learning excels at prediction, but is prediction really what your business needs? Stefan explores why predictive models alone often fail to answer critical “what-if” questions, how causal machine learning bridges this gap, and provides practical advice for how you can start applying causal ML at work.
LINKS
Stefan's Stanford Website (https://www.gsb.stanford.edu/faculty-research/faculty/stefan-wager)
Machine Learning and Economics, Stefan and Susan Athey's lectures for the Stanford Graduate School of Business (https://www.youtube.com/@stanfordgsb)
Causal Inference: A Statistical Learning Approach (WIP!) (https://web.stanford.edu/~swager/causal_inf_book.pdf)
Mastering ‘Metrics: The Path from Cause to Effect by Angrist & Pischke (https://www.masteringmetrics.com/)
The Book of Why: The New Science of Cause and Effect by Judea Pearl and Dana Mackenzie (https://en.wikipedia.org/wiki/The_Book_of_Why)
Causal Inference: The Mixtape by Scott Cunningham (https://mixtape.scunning.com/)
A Technical Primer On Causality by Adam Kelleher (https://medium.com/@akelleh/a-technical-primer-on-causality-181db2575e41)
What Is Causal Inference? An Introduction for Data Scientists by Hugo Bowne-Anderson and Mike Loukides (https://www.oreilly.com/radar/what-is-causal-inference/)
The Episode on YouTube (https://www.youtube.com/watch?v=f9_Lt5p8avU&feature=youtu.be)
Delphina's Newsletter (https://delphinaai.substack.com/)
--------
54:40
Episode 11: What Comes After Code? The Role of Engineers in an AI-Driven Future
Peter Wang—Chief AI Officer at Anaconda and a driving force behind PyData—challenges conventional thinking about AI’s role in software development. As AI reshapes engineering, are we moving beyond writing code to orchestrating intelligence? Peter explores why companies are fixated on models instead of integration, how AI is breaking traditional software workflows, and what this shift means for open source. He also shares insights on the evolving role of engineers, the commoditization of AI models, and the deeper questions we should be asking about the future of software.
LINKS
Peter Wang on LinkedIn (https://www.linkedin.com/in/pzwang/)
Anaconda (https://www.anaconda.com/)
Mistral Saba (https://mistral.ai/news/mistral-saba)
Peter chatting with Hugo several years ago about the beginnings of PyData, NUMFOCUS, and Python for Data Science (https://vanishinggradients.fireside.fm/7)
Delphina's Newsletter (https://delphinaai.substack.com/)
Welcome to High Signal, the podcast for data science, AI, and machine learning professionals.
High Signal brings you the best from the best in data science, machine learning, and AI. Hosted by Hugo Bowne-Anderson and produced by Delphina, each episode features deep conversations with leading experts, such as Michael Jordan (UC Berkeley), Andrew Gelman (Columbia) and Chiara Farranato (HBS).
Join us for practical insights from the best to help you advance your career and make an impact in these rapidly evolving fields.
More on our website: https://high-signal.delphina.ai/
Polska