From Prompts to Runtime Signals: Making Open-Source AI Systems More Trustworthy
MCLD 3002 | Sat 08 Aug 11:45 a.m.–12:30 p.m.
Presented by
-
Rishabh Banga
https://rbx-labs.io/
Rishabh Banga is an AI product leader, builder, and ecosystem contributor whose work sits at the intersection of applied AI, product systems, and real-world execution. Over the past decade, he has built and led products across startups, enterprise platforms, and community-driven technology ecosystems, with a focus on making complex systems practical, reliable, and usable beyond demos.
Rishabh has also volunteered his time to give back through judging, mentorship, and founder support across initiatives such as MIT $100K, MIT Solve, the Presidential AI Challenge, and the Verizon Disaster Resilience Accelerator. Across developer, student, and startup ecosystems including Hacker’s Tribe, IntelliGen, and Intel programs, he has helped build communities and trained 5.8K+ students and professionals across AI, IoT, and product development.
Rishabh Banga
https://rbx-labs.io/
Rishabh Banga is an AI product leader, builder, and ecosystem contributor whose work sits at the intersection of applied AI, product systems, and real-world execution. Over the past decade, he has built and led products across startups, enterprise platforms, and community-driven technology ecosystems, with a focus on making complex systems practical, reliable, and usable beyond demos.
Rishabh has also volunteered his time to give back through judging, mentorship, and founder support across initiatives such as MIT $100K, MIT Solve, the Presidential AI Challenge, and the Verizon Disaster Resilience Accelerator. Across developer, student, and startup ecosystems including Hacker’s Tribe, IntelliGen, and Intel programs, he has helped build communities and trained 5.8K+ students and professionals across AI, IoT, and product development.
Abstract
Open-source AI systems are getting easier to assemble: pick a model, add retrieval, wire up a few tools, and you can get something impressive working quickly. But getting that same system to behave reliably in production is a different problem. This talk is about the gap between “it works in a demo” and “it can be trusted in use.”
Attendees will leave with a clearer way to think about trust layers for open-source AI systems, where these systems tend to fail in practice, and what design patterns can make them more usable beyond demos.
Using lessons from building applied AI systems, I’ll show why prompts alone are not enough, why offline evaluations often miss real-world failure modes, and why trust in AI systems has to be treated as a systems problem rather than just a model problem. The session will focus on practical ways to make open-source AI workflows more trustworthy through runtime signals such as retrieval quality checks, confidence proxies, verification layers, fallback logic, and failure detection across multi-step workflows.
Open-source AI systems are getting easier to assemble: pick a model, add retrieval, wire up a few tools, and you can get something impressive working quickly. But getting that same system to behave reliably in production is a different problem. This talk is about the gap between “it works in a demo” and “it can be trusted in use.”
Attendees will leave with a clearer way to think about trust layers for open-source AI systems, where these systems tend to fail in practice, and what design patterns can make them more usable beyond demos.
Using lessons from building applied AI systems, I’ll show why prompts alone are not enough, why offline evaluations often miss real-world failure modes, and why trust in AI systems has to be treated as a systems problem rather than just a model problem. The session will focus on practical ways to make open-source AI workflows more trustworthy through runtime signals such as retrieval quality checks, confidence proxies, verification layers, fallback logic, and failure detection across multi-step workflows.