Adam Kamor (@tonicfakedata, Founder & Head of Engineering) talks about building RAG (Retrieval Augmented Generation) systems in production for AI data.
SHOW: 992
SHOW TRANSCRIPT: The Cloudcast #992 Transcript
SHOW VIDEO: https://youtube.com/@TheCloudcastNET
NEW TO CLOUD? CHECK OUT OUR OTHER PODCAST: "CLOUDCAST BASICS"
SHOW NOTES:
Topic 1 - Adam, welcome to the show. Give everyone a brief introduction.
Topic 2: Our topic today is RAG systems, specifically RAG in production. Let’s start with customization sources and types. When it comes to customizing off-the-shelf LLMs, RAG is one option, as is an MCP connection to a SQL database, and there is pre- and post-training, as well as fine-tuning. How does an organization decide what path is best for customization?
Topic 3 - RAG came on the scene as the savior for organizations that want to use customer AI without the need for fine-tuning and additional training. It has either gone through or is currently still in the trough of disillusionment. What are your thoughts on RAG's evolution and the challenges it faces?
Topic 4 - Let’s walk through the basics of validation. Once you set up RAG, how would an organization know it works? How is accuracy measured and validated? Are you looking for hallucinations? Context quality?
Topic 5 - What is Tonic Validate, and where does it fit into this stack? Is it in band? Out of band? Built into the CI workflow?
Topic 6 - Accuracy is one aspect, but we hear more and more about ROI for Enterprises. How should ROI, risk, and compliance be measured?
Topic 7 - Where and how does security fit into all of this? Also, your thoughts on synthetic data for training vs. real data?
FEEDBACK?
FEEDBACK?