Building a Podcast Chatbot for Voxgig
This post is part of a series.
At Voxgig we have now recorded 140 podcast episodes (as of Feb 2024). That's a lot of chatting about developer relations. It's great if you are a regular listener (thanks!), or have the time to listen to our back catalogue. But what is frustrating is that there is so much wonderful knowledge about developer relations locked up in an audio format that, while enjoyable to listen to, is not accessible or usable in an efficient manner.
When you need to know something about developer relations, and you want to tap into the wisdom of one our guests, you are out of luck. Or at least, you were. We decided to turn our coding skills to this little problem and write our own AI chatbot to let you ask our guests your questions directly.
If you go to the Voxgig podcast page, you can now see a little chatbot widget, and when you type in a question, you get a reasonably good answer!
Now this is a very crude initial version, and we are actively improving it. The chatbot uses Retrieval Augmented Generation (RAG) to ingest all our podcast episodes, and let you ask questions of our guests directly.
We decided to open-source the entire project. Building a RAG ingestor and query engine is a natural fit for microservices (there are lots of little asynchronous tasks). So if you'd like your own podcast chatbot, just cut and paste our code: github.com/voxgig/podmind.
We've had such an enthusiastic reaction to this fun little project we've decided to take it further and extend the chatbot to handle all sorts of developer content. We're also adding the ability to easily experiment with the prompt and the "chunker" (stay tuned if you want to know all about that stuff).
We're going to "build in public" with this project. You'll be able to follow along at a source-code level as we make it more and more useful.
To get started, here are the microservices and what each one does:
audio- Transcribes audio to dialogue text (using Deepgram).auth- User identity - for the user analytics dashboard.chat- Perform chat queries.chunk- The "chunker" that splits the transcript into chunks for the vector db.embed- Embed the chunks as vectors so you can do similarity searches.entity- General purpose persistent entity operations.ingest- Ingestor to orchestrate the ingestion process.monitor- Monitoring and debugging of the system, provides a REPL.store- Store audio and RSS files.user- User management.widget- The API for the embeddable chat widget (which is a proper web component).
We run this system as a "local monolith", but deploy it as Lambdas on AWS.
A lot of our development is done in a REPL with hot reloading, so we can alter business logic and experiment on the fly, even with our deployed Lambdas. We'll get into this in later posts, but here is an example of submitting a question to the live chat service Lambda:
$ seneca-repl aws://lambda/pdm01-backend01-dev-monitor?region=us-east-1
> @voxgig/podmind-backend@0.3.3 repl-dev
> seneca-repl aws://lambda/pdm01-backend01-dev-monitor?region=us-east-1
Connected to Seneca: {
version: '3.34.1',
id: 'axjz/monitor-pdm01-dev@0.3.3',
when: 1708436725968
}
axjz/monitor-pdm01-dev@0.3.3> aim:chat,chat:query,query:'what is devrel?'
{
ok: true,
why: '',
answer: "DevRel, short for Developer Relations, is a field within the tech industry that primarily focuses on building and maintaining relationships between companies and developers. DevRel professionals often act as intermediaries between developers and the companies they serve, bridging the gap between technical know-how and the business needs of a product. They help developers understand how a company's product works and are always on the lookout for opportunities to improve the overall developer experience.",
context: {
hits: [ ... ]
}
}
We're a TypeScript/JavaScript shop, so that's what we're using right now. I would not be surprised if a Python microservice or two appears in the future, but for now, we'll stick to what we know best. This is the start of a series - I'll add links to more posts as we go.