We helped write the sequel to "Cracking the Coding Interview". Read 9 chapters for free

System Design: Online Bookstore Backend API

Watch someone solve the online bookstore backend api problem in an interview with a FAANG engineer and see the feedback their interviewer left them. Explore this problem and others in our library of interview replays.

Interview Summary

Problem type

Online Bookstore Backend API

Interview question

A brick-and-mortar bookstore wants to launch an online presence and needs a complete backend API built from the ground up. The system must support browsing and searching a catalog of up to 10 million books, handle millions of reads per hour, manage inventory levels accurately, and eventually incorporate AI-driven semantic search and an agentic chat experience to help users discover books.

Interview Feedback

Feedback about Double Hermit (the interviewee)

Advance this person to the next round?

Yes

How were their technical skills?

4/4

How was their problem solving ability?

4/4

What about their communication ability?

3/4

Strengths - Really strong technical knowledge across different types of tech and clean way of laying out the diagram. Good job including monitoring and observability. - Good job thinking of edge cases and possible failure scenarios. - Responded really well to the AI follow-up. Didn't just go down the gen ai path immediately but laid out two options and proposed that the first (vector db and open ai search) would likely be the most useful. Shows that you understand tradeoffs and how to integrate pieces into the system. Things to improve: - Be careful saying things like "that's not that interesting to talk about but if you want, we can discuss" - you never know what skill the interviewer may be digging into and you don't want to put down a part of a design. LIke I mentioned, it may be deceptively simple so they can introduce a twist or a curveball later in the interview. - Especially at the senior level of a design, I expect you to drive a lot more of the design without as much intervention from me. This was more a situation in the first half of the interview versus the second, where you really got into the groove. It's good to check-in but not overly so at that level - Make sure to explain your decisions and assumptions, even if they seem deceptively simple. WHy did you chose a message queue, why postgres (relational vs. not) etc., it shows up much better than just specific technologies - Be sure to wrote out some ballpark numbers for non-functional requirements, especially number of users, number of items, amount of data, SLAs etc. It's important to show you have that understanding up front. - On the AI follow-up: there can be a ton of complexity when you add in AI systems, be careful to not get too overwhelmed in the design. Start with the simple piece first and then then start going through the edge cases/error scenarios.

Feedback about Wheedling Zebra (the interviewer)

Would you want to work with this person?

Yes

How excited would you be to work with them?

4/4

How good were the questions?

4/4

How helpful was your interviewer in guiding you to the solution(s)?

4/4

Interview Transcript

Wheedling Zebra: Perfect, so welcome. Before we get started, I always like to ask, have you done an interview with interviewing.io before?

Double Hermit: Yes, I have.

Wheedling Zebra: Yeah, okay, perfect. So you should know your way kind of around the system, it's sort of the same. And then just to confirm, this is for a system design interview. Is that correct? Awesome. So how I generally like to run these, we spend about 45 to 50 minutes actually doing the interview. Then I like to leave at least 10 minutes at the end. You can ask me any questions kind of about the interview, if you had any questions about whatever the problem is we went through. And then that also gives me a chance to give you a little bit of feedback live. That way you can do any follow-ups if there is anything you want to clarify. I still do the write-up at the end of course anyways, but like it just gives you a chance to ask me anything just in case. Is that all fine, [REDACTED]? Awesome. So we are going to start with a pretty basic prompt which is, your team owns a bookstore, or maybe better is your team's been hired by a bookstore. They're brick and mortar and they've decided that they want to open an online business. And so they basically need to build out an entire backend API for clients to be able to retrieve information about their bookstore and what they have in inventory and all that sort of fun stuff.

Double Hermit: Okay. I'm sorry, wait, did you say low-level design or system design?

Wheedling Zebra: Uh, System Design.

Double Hermit: Um, should we swap to the whiteboard, or— okay. Yeah. All right, let me just copy this and then paste it in. Okay, sorry, I think I cut you off.

Wheedling Zebra: No, that's basically it.

Double Hermit: Okay. So yeah, I'd like to go over the functional and non-functional requirements. So from what I gather, I think this is probably going to be like pretty straightforward. I would say, so if we go over like the functional/non-functional requirements, I think we probably want to be able to get book information. We probably want to be able to write book information as well. And we probably need some kind of way to search for books by, like, the author name or title name. Does that kind of sound good for the functional requirements?

Wheedling Zebra: Yeah, sounds good to me.

Double Hermit: Okay, so if we're going to the non-functional requirements, I would say we probably want to have like low latency, high availability, and we probably want to have like freshness within a few seconds. So say we write a book, we probably want this to be like and the overall timeout dates. So I think that should be good for the non-functional requirements. Is there anything you would probably add?

Wheedling Zebra: Well, talk to me a little bit about why low latency and why high availability.

Double Hermit: I would say— Low latency, meaning like if we search for a book, we want it to be within like a few milliseconds that we get a result.

Wheedling Zebra: Awesome. And why high availability? Like why do we think that's important?

Double Hermit: So say we have like high QPS or like high traffic, like we want to be able to give some result back. Okay.

Wheedling Zebra: And when we are thinking of like traffic and a lot of potential users, what are you thinking in terms of scale for what we are looking at?

Double Hermit: Yes, I mean, I would say we probably want to design this for a higher scale because that's a more interesting question. I think if we consider like a normal library, like even in big cities, the scale is not going to be like that big, but say maybe this is like Barnes Noble or something like that is what I'm thinking.

Wheedling Zebra: Yeah, so if we are looking at something like Barnes Noble, what scale are we looking at?

Double Hermit: I would say it's going to be a pretty big scale, like maybe millions of reads, you know, an hour or so, and maybe we have like about like 1 to 10 million books. Does that sound good to you?

Wheedling Zebra: Yep, sounds good to me.

Double Hermit: Okay, so I guess there's two paths we can go from now. It's like, do we want to get into the high-level design, or do you want to go over like the API and write that out as well?

Wheedling Zebra: Let's start with the high-level design, and then we can drop into the API level.

Double Hermit: Okay, sounds good. So we'll just have our client, which is going to be our front-end application. I mean, this could be like a mobile app or like a browser or anything like that. And then we want to hit some kind of load balancer and it's gonna be responsible for like authentication, rate limiting, and whatnot as well. So the kind of path that I'm thinking is like we're probably gonna have pretty high read traffic and not very high write traffic. Um, so I think it's important to separate our read and write services, um, to be two different services. That way they can scale independently. So, um, we can have like add book and we can have a get book. So to satisfy our low latency on search requirement, it's going to be beneficial to have something like Elasticsearch or Typesense, where essentially we're loading the books into memory. But for now, let's just go over like the, the basic path. It kind of brings into the question, what kind of database do we want to use? And I think if we consider like Cassandra or Bigtable, like we can have Well, if we think about our access patterns, like, we can probably have two separate databases. So like, we can have one for like the catalog database and one for like our inventory, and then obviously have our search. I think for now we can just keep it simple and go with like Postgres. I mean, maybe we come back and like touch on the database here in a second.

Wheedling Zebra: Yeah, sounds good to me.

Double Hermit: So let me add some to-dos. I have some things that I want to touch on. So I would say probably like Elasticsearch, and then probably Splunk or database paths. Um, before I get into these, is there anything that you think I'm missing or that you wanted to go deeper on?

Wheedling Zebra: Uh, I think let's start here and see where we get to.

Double Hermit: Okay. Um, so I think we can probably go with Elasticsearch first. Um, so what Elasticsearch is It's essentially going to load all of our inventory data into RAM and let us do like very quick searches. But then the question becomes like when we write to the database, how do we know how to put it into Elasticsearch? And we can use change data capture for that. And then when we get a book, we, we want to get it from Elasticsearch. And let me just fix this typo. So let me just think this through. So it depends how we want to search for, how we want to search for authors, right? Like, I mean, what kind of metadata do we want to search on? If we want to search for title alone, I think we can have— I think we have the book title, like index on book title in Postgres. And if we're just searching by title, we can probably just get it from Postgres. But if we want to search by metadata, we can use Elasticsearch for that.

Wheedling Zebra: And what do you think the trade-offs are there on timing-wise of one over the other? Like, why the split?

Double Hermit: So I'm thinking anything we want to do that's like fuzzy, we use Elasticsearch for because it's going to be quicker. And I guess like now that you mentioned that, we can have fuzzy searches in Postgres as well. Like, people might want to firstly search the title. So it might be the case where we do titles in Elasticsearch as well, and maybe we use like the ISBN, and I think that's the correct name for it, but if it's something like ISBN where you have like exact data, then we search for it in Postgres.

Wheedling Zebra: Yeah, that feels a little more reasonable because I can imagine, right, like if a customer doesn't exactly write the title perfectly, which is very likely if you think of the use cases when people are searching for stuff, having that fuzzy match can be really helpful.

Double Hermit: Yeah, that makes sense. All right, so I guess we could go deeper into like how we want to design the database. Um, which— this is the question, like, do we want the primary key to be the title or the ISBN? I think either ISBN or like some kind of book ID, um, would be ideal here because they could probably have multiple titles. Um, so I think I would design this like where we have like, um, book ID as like the primary key, and then we have like the title, and this book ID could be the ISBN as well if we decide to use that. We would just have like the title, the description, maybe like some kind of summary about the book, the author. We could basically have all of the metadata, so like the publisher, Then we consider like these publishers and authors, like they can have their own table as well. So we can have like an authors table where we have like the author ID. So this would be an author ID, which we can join on and get like the author metadata. So that would be like the first name, last name, and all of that. Is there anything you want to go deeper in on the database design?

Wheedling Zebra: No, I think this is good.

Double Hermit: All right, so kind of at this point I want to think through like some failure scenarios. Like where can this actually break? So I think one path would probably be the availability of Postgres and Elasticsearch. And we have like pretty high read-throughput or like load, like our read path that's very like load-bearing, right? So I think if we have like a big Traffic spike. All right. Yeah, let's say if we have like a big traffic spike, we can— I mean, we could get into scaling the database and like scaling, like setting up scaling rules. I don't really think it's like a super interesting conversation. Is there like a particular area you think we should go into?

Wheedling Zebra: Yeah, maybe not scaling. I mean, I agree it's not as interesting. But you know, there are times when high traffic may be high traffic, but it's very similar traffic. So if you think of a scenario when a very popular book comes out, you might be getting a lot of traffic, but you are getting a lot of traffic for the same thing.

Double Hermit: Okay, yeah. That's a good point. So what would that look like? Yeah, so say like we have a popular artist, we have some kind of like hotkeying issue. One way you can mitigate that is with Redis. So we could just put popular artist in Redis.

Wheedling Zebra: And what would the benefit be to that?

Double Hermit: So if we have like, say it's like the new Harry Potter book, all of these lookups are not going to be actually hitting the actual database. I mean, we could do more than just like title queries. Like we could just do— we could just keep that querying and then return the result. So this would be like title, author, essentially like whatever the query ends up being. And we could put this in front of— I put this in front of Postgres. But it's not necessarily always going to hit Postgres. I would say the only situation where we'd have a hotkey is if they're searching by ISBN. Because if they're searching by, like, title and whatnot, it's gonna be hitting Elasticsearch. So we can add the same cache to Elasticsearch, but Elasticsearch is already, like, in memory and pretty fast. So I'm not really sure how much advantage we would be getting, but we still do save, like, a lot of our— so we won't be protecting Elasticsearch from, like, a hot keying issue, but we will still be protecting it from, like, the very high repeated writes.

Wheedling Zebra: Right. And what are your thoughts on how you manage that cache about when you invalidate it, maybe refresh it?

Double Hermit: So this is pretty static information. Like, we don't expect this information to change often, so we can do like a 30-minute eviction, or we can even do longer. Like it depends how many unique searches we are getting, like how much we want to evict. So like we could do, like as it gets to the end of memory and it hasn't been accessed in a while, we can evict it. Or we could just do like a standard, like 30-minute eviction.

Wheedling Zebra: And what do you think the, like think of this scenario of like it being a bookstore. So sometimes We're adding new books, maybe we're running out of inventory, right? Like this is also sort of attached to our inventory levels, especially if we're talking about let's say the same scenario like a very popular book like a Harry Potter book. What might those scenarios look like in that cache?

Double Hermit: So you're saying like we're actually checking out books?

Wheedling Zebra: Yeah, like it's not just, you know, we're adding or removing books. Like this is also around our inventory. So there may be a—

Double Hermit: Okay, I see. So you're saying like we're showing how many books we have available. So you're saying like if we catch up for too long, we could be showing inaccurate information.

Wheedling Zebra: Exactly. So how do you keep that contract of what you— like, what is a contract that you expect to be able to fulfill of like how accurate that information needs to be?

Double Hermit: It's a good question. So I guess there's two situations, right? It's like one is out of stock. We can probably cache it longer if it's out of stock unless we expect it to come back into stock. But if it's in stock, we probably want to do a shorter cache, like say 10 seconds. Or maybe a minute.

Wheedling Zebra: Yeah, sounds good. So looking at this, I mean, what else do you think we need to think about?

Double Hermit: That's a good question. So I guess it's the situation, like, it's not gonna be users adding books. It's gonna be more of like an admin user. We can go into like authentication and stuff as well. I wouldn't say it's super interesting, but we can do like a JWT where we have like claims and we're checking roles, making sure it's like an admin user who's actually adding this book. So like our middleware would be like, is it admin? Before they can actually access this add book route. I would say that there's probably like reservation semantics, like does somebody have this book reserved. It really depends if we want a system to support that. We could talk about like observability as well. So like one thing we probably want to track is like the Redis. So like what is our cache hit percentage? What is our end-to-end latency on the whole system? What is our query time on the database? I mean, we can set up alerting rules if any of this happens. We can also set up like scaling rules. So like imagine this is Dockerized and we can scale it like put it on like EC2 and then scale it as needed. So say this is probably like a pretty minimal application, I would say, for just reading books. Maybe your bottleneck is a CPU. So we can scale like as we get to like 80% CPU usage on this service maybe we preemptively scale it, keeping in mind there's like scaling delay. I mean, we pay a price to scale.

Wheedling Zebra: Yeah, this all looks great. So maybe let's shift over and look a little bit at the API design.

Double Hermit: Okay. So we want to get the book. Well, let's do the search because I think it's more interesting is how we are going to do the search. And I guess the question for you is like, does a user need to be logged in to search?

Wheedling Zebra: Well, what would you expect? If you go to somewhere like Barnes Noble, do you expect to have to log into an account to be able to search for what they have in stock?

Double Hermit: That's a good question. I'm trying to think like what Amazon does. I'm pretty sure you can search without having an account. So maybe we want to do no authentication on the search path, which is something we need to consider can be like an abuse point. So we still need to rate limit by like IP address since we don't necessarily have like a JWT token. So this website is frozen for some reason. Can you still hear me?

Wheedling Zebra: Yeah.

Double Hermit: OK. Give me one second.

Wheedling Zebra: Yeah, no worries.

Double Hermit: All right, what happens if I refresh the page?

Wheedling Zebra: I think it just brings you back in. You should be fine.

Double Hermit: Okay, I think I'm good. So yeah, so what I was saying was our rate limiting. So we need to rate limit this slightly differently, probably like on the IP address to prevent abuse. IP address has its trade-offs, like it's not always going to be a good mechanism to really limit on, but I think for the sake of this discussion, it's probably good enough. I would say the way we want to do search is we want to have some kind of endpoints, so like books/search, and I think we passed the search parameters with query parameters, and that's going to be easiest for caching. So we could just do query would be like the book title. And then we can have like author as like an optional thing. Maybe even we can do whatever we want, select the ISBN or whatever we support searching by. We can even search by description. And this would be a GET request. And then we could go into our more authenticated paths. Yeah. So we could have a POST books, so books/upload, and this can take like the author. So this will just basically take all the information we're going to be writing into the database. The book ID. If it's not the ISBN here, we probably want to do like a UUID for this. We wouldn't generate this on the client, we would generate this in the Add Book service. So I'll just leave this off. We want this to be JWT authenticated, so only users with the role in the JWT token of like admin or like book owner Or whatever we decide to call the role, can upload books. And then the question is like, do we want to support images on these books? Because I think in an application like this— Probably. You would want to upload like the book image. Yeah. In that case, we need to introduce like an object storage here. We can use something like S3 for the images. And when we do add book, we can get a pre-signed URL which gets sent back to the client, and that lets them upload the image directly to S3 using the pre-signed URL. I mean, S3 is going to give us back an ID for that image, which we can store that ID and the metadata database. So like, our book would have S3 image ID. And I think there's probably a little bit more to get into with that, is like, how big do we want these files to be and things like that. I would say we can like do limiting on the file size Um, maybe we only allow like 50 or like maybe a gigabyte file for the most part. Um, and then we could create thumbnails or something, uh, for the actual website and like compress it from there. Um, that kind of adds a little bit of complexity to our system though.

Wheedling Zebra: Well, complexity that is potentially useful in terms of the use case for the user we're building for, right?

Double Hermit: Yeah, I would say it's useful and we could go into it if you want. Yeah, let's do it. So I would say once we upload— let me just move these out of the way. So once we upload this, we want to place the image on a queue. And this could be something like Kafka where we have events. And these events are going to some kind of worker pool. And essentially what these workers are doing is like running the computation required to get these in different sizes and compressing it. So these would be like worker pool, and we use Kafka consumer groups for this where we're sending certain images to certain workers. So we can have like compression and resizing, which is like— maybe I'll write that somewhere else. Which is going to ultimately end up sending these back to S3, but we can bucket them so we can have like— all right, this would probably be like the same S3 instance, but imagine we have buckets for each one. So like the small bucket, medium, for each thumbnail. And we just send them to this bucket. And then we get the ID and we can send it to the database, which changes our schema slightly where we have like a small, medium, and large ID. I would say as well, like, it's important to note, like, we need to scale these workers based on, on the backpressure and like our latency requirements. So we have like a SLA of like 5 seconds when you upload a book, we want it to be seen, we need to scale these workers. I would say like this probably is not going to be an issue that we run into because we're not going to have a lot of writes. So I think in most cases, like yes, this will be a synchronous path, but these workers will be scaled down most of the time. Maybe if there's some kind of like bulk book upload, like we just acquired another bookstore and we want to add all of the books to our inventory at once, then we would have some kind of burst traffic, but I don't foresee, don't foresee it being something that we'd like, we need to overdesign around, so to say. I would say maybe it's worth talking about this Kafka schema. So we want to have exactly-once in this situation. So we can have like an identity key of like the book ID, and then we can have like the— the raw image. And this would be the S3 path. I then— when these workers get it, they can download it, do the transformations, and then put it back into S3. And we're still preserving the original image if we ever need it at some point.

Wheedling Zebra: Yeah, and you talked a little bit about— you mentioned the SLA. Do you think this is something that needs to happen quickly, or is this something that can sort of happen It depends how quickly you expect the user to see the book.

Double Hermit: I would say we want it to happen relatively quickly, within the magnitude of like a minute or so. I'm not including network time, so the time that it actually takes to get into S3, but from the time it's in S3 to the time that we make it available on the website, I would say maybe 1 minute is a good SLA for that.

Wheedling Zebra: Yeah. Might even, even if it's longer than that, right? Like if you think of where our priorities are, if we're adding a new book, does it need to be at 8:58 versus 9 AM? Like maybe not, right? So there we have a little bit of flexibility on the timing of that. We may even want to batch a lot of this when we don't expect a lot of people to be looking at information on our bookstore. If we have, certain spikes in usage, right?

Double Hermit: That's a good point. We can do this as a batch as well. Makes sense. Cool.

Wheedling Zebra: So one tangent we can take here, sometimes I, at the 30-minute mark, it kind of depends where your experience is. For the design, because a lot of design problems now are starting to incorporate some aspect of AI into them. And so kind of at the halfway point, I like to ask, like, is that something you want us to pivot and look at for this, or is that not really something that either your experience or your— what you're practicing aligns to?

Double Hermit: Yeah, I'm okay with that. When you say AI, do you mean like how can we incorporate AI into the system?

Wheedling Zebra: Yeah, like now what if the bookstore owner is like, "Hey, this is all well and good, right? But like I've heard there's now these new fancy AI systems, like a chat system I can use to actually help users find books or surface new books for me, for the users." And they just come to you and they're like, AI that's designed for me in a way that's actually useful.

Double Hermit: Okay.

Wheedling Zebra: What are the questions you ask and what do you start trying to figure out where or what parts of the system may actually benefit from having some of that?

Double Hermit: So there's probably two paths we can take to the AI question. One is like, do we want to use generative AI where you just type to an agent and it downs a book for you. I think to do that we can probably create like an MCP server, um, and then have the agent like, uh, reinforcement trained on some of these books. Um, I think probably a better approach would be to have like a vector database and we store description embeddings. Um, and then we can have the AI do like a semantic embedded search and see like whatever is closest to your query. Um, and then It's not really like, we don't really need generative AI for that, but we can have generative AI on the system like telling the user like, "Oh, I found these books and I think you might like them." Yeah, so let's dig then into like the vector, the second option with the vector database in the sense, what would that look like? Okay, so we need to again have some kind of pipeline and this could be a batch pipeline. This time it doesn't need to be something that happens in real time. But essentially we have some kind of ETL job that pulls these descriptions out and embeds them. So if you go to like a third-party embedding API like OpenAI Embedding, and they also have batch embedding which is like drastically cheaper. So OpenAI embedding, uh, and then we send it to them, and then at some point they send it back to us. This could be like through a webhook. So say we have like a webhook service. So when they're done with their embeddings— I've never actually done this, but I'm assuming that's how it works— when they're done with their embeddings, somehow they call back to our system. Um, the other question is like What if OpenAI is down? We can have like a circuit breaker or something like retrying requests and we could do like a circuit breaker that slowly opens and tests if they're back up. Since this is not real time, there's no need for like a dead letter queue or anything like that because we can just hold off on sending requests. So they send us back a webhook and we can send this into a vector DB. And I'm actually not too familiar with like the different types of vector DBs and trade-offs. I think a popular one is CockroachDB. I might be wrong on that, but I would say some kind of vector DB which lets us do search. And actually I think Elasticsearch supports vector searches, so we might not even need to search this DB directly. Like, we can, um, in turn load this data into Elasticsearch and let it do the vector search, if that makes sense.

Wheedling Zebra: Yeah, that makes sense. And do you— how do you imagine kind of a fallback scenario? So you talked about like, what if OpenAI is down? Um, do you think we want to take down our entire bookstore API in that case because our backend is down? Or do you think there's potentially a hybrid solution that lets us, um, potentially still be able to pass those results back?

Double Hermit: Uh, so in that situation, these books just won't be searchable on Live Sector until they get embedded. So say this happens at nightly, like a nightly, um, ETL job that will extract them, um, prepare them for embedding, send them to OpenAI, then put them into our system. I would say like if OpenAI is down, our system is not going to be down. There's no risk of that.

Wheedling Zebra: And is there a way to work around that?

Double Hermit: I wouldn't say there's a— maybe I don't understand the exact risk that you're talking about. Like you're saying if the embedding API is down?

Wheedling Zebra: Yeah, or if like our vector database goes down. Right, so we're, we're adding this piece, this like, uh, kind of AI piece by doing the vector embeddings and the search and then potentially routing that back. Um, but we had a working system before we decided to add this. So do you imagine there's a way to have a version of both as a fallback?

Double Hermit: Um, yeah, I mean, we can fall back to normal search. I think We can have like, well, we can fall back to the database directly, which would just give us slower search than Elasticsearch. So you're saying in the case that VectorDB is down, we would still have books loaded into Elasticsearch. So it's already like, I guess there was some redundancy built in because we're actually hitting Elasticsearch. If we want to fall back, we can fall back like embedded searches to the database directly and just be acceptable that it might be like 1 to 2 second searches instead of like millisecond searches.

Wheedling Zebra: Yeah.

Double Hermit: Great.

Wheedling Zebra: So this is one place. Where else do you think?

Double Hermit: I think probably the biggest risk is the webhook being down. Because, well, I would say two things. If our database is down and the webhook can't write, we have to do something with these messages. So I think we created a dead letter queue here, which would just hold the messages until our database is back up because we're paying for these embeddings and we don't want to lose them. There's probably some mechanism on OpenAI's side where we can have them replay these events. I would say probably the biggest risk is if our webhook is down because it's sending us data and it's just getting 500. In that case, OpenAI should stop. This is like a third-party provider, so there's not really much we can do, but I would assume if they're sending messages to webhook, they would just stop sending them until we fix it and then we could go back and re-enable the webhook. So they might send us some kind of like email notification being like, your webhook is down, go back in the dashboard to enable it.

Wheedling Zebra: Yeah, okay, so those are kind of our side scenarios with having these embeddings and having this hooked up to our Elasticsearch. Where else do you think we could hook up AI into the system?

Double Hermit: Yeah, so let's, um, let's do like a more generative AI. We have a separate request path for agentic-search. And we can have this agent call, like, an MCP server for our own application. I mean, essentially what an MCP server is, is it's just a list of tool calls that the agent can do. Let's do MCP. So my understanding of MCP, it's like injecting tools into the agent's context. So when it calls this server, We're going to send back a list of tools that it can use, and then this agent's going to be able to call routes on the server. And I'm envisioning this— well, it's going to be kind of weird to diagram, but the agent will talk to this, and then this will again go through the load balancer to actually hit our application. So say we have like a tool called for reading books. Let's call this semantic search. So we can pass a— this tool can basically just say we want to pass a— let's do semantic true on this. So semantic is true, then you can search by like whatever. And we could probably get into like the design of this a little bit better, but for now I think this is fine. We can even break semantic search into its own path as well because I can imagine it's, it's a lot more intensive than doing just a normal read on the database. Well, maybe that's not a true statement, but we can go into that later. Yeah, so it calls this, then we can have the agent take search. So it would have to be powered by like a third party. So say we use like OpenAI for this or Anthropic, but we just call the OpenAI endpoint and we can have like a system role and some kind of like system prompt for the agent. So it's like telling it that it's a bookstore agent and it's meant to help the user. I would say the question of streaming here comes into play. So if we want to stream back to the user, We need to do WebSockets. I think streaming is like the best experience for this. So we could break our WebSocket layer into an L4 load balancer, and essentially what an L4 load balancer does is it just forwards HTTP requests— sorry, UDP— and So this is probably too complex for this diagram, but there's the question of scaling these. So essentially these agent servers become WebSocket servers who are doing multiple tasks. So let's talk about the L4 load balancer real quick. If you have a L4 load balancer, since they're just forwarding UDP, they're not able to like read HTTP requests. So the question is like How do we know which— how to route them? And we do this at the DNS level. So we have websocket.domain.name. So that's how we know to go to the L4 load balancer as well on this domain, 'cause we can't just do, like, /websockets since there's no way to parse the URL. So it goes to this WebSocket server. I mean, the trade-off with using like an L4 load balancer is like we have to do authentication on the WebSocket server now. So I'm just going to rename this to WebSocket server because it kind of becomes like a WebSocket problem. The question is like when we have all of these services scaled out, how do we know which server you're being routed to? Like say we have 10 users doing like a WebSocket connection with ChatGPT, for example, we can have like a registry. Well, there's two things we can do, right? We can do like a broadcast where we're just sending like each thing to each server looking for the connection, or we can have like a registry, Redis registry, where we store each user's WebSocket connection. So when we go to send the message back to them, like streaming it through, these WebSocket servers can look up like where that user lives on the registry and send it back. And there's probably— there's a lot of complexity here. Like this becomes very complex very quickly. So for the sake of time, it's like any problem you want to dig in on, because I think there's also like the the third party like OpenAI can drop the stream and we have to reconnect the stream. I think there's a lot of edge cases in this.

Wheedling Zebra: Yeah, and we are at the— we are at 46, so we are almost done. Instead of going into that, I kind of want to talk about the final— so we've talked about different pieces, like we had like the initial design And then we had the embedding addition. We had the other going through kind of semantic search, the MCP server, the agent. So now you have to go back to your client and you're looking at this design and you're trying to give them a sense of what is the best path for how they should be thinking about this design in terms of priorities and where you think it best fits for them. So in order of like how important the different pieces are, like where do you think they should invest most of their time upfront first to build and like what pieces can they maybe scale out to third-party providers?

Double Hermit: I would say they definitely want to probably do embeddings first because, well, it's more of a product question. So I would just recommend them the trade-off is like if you want to get like an agentic experience where you have an agent who is, um, maybe you do like some kind of reinforcement on OpenAI side and you take that path where the agent already knows like all of the books in your inventory and you're just training on that and then you have it just do the search by title or whatnot. Um, you can do that as well. I think it's going to be a lot more expensive And every time you add a book to your inventory, you're going to have to reinforce the agent more. With the other path, I would say probably want to prioritize doing the embeddings because that's going to let any user search by normal text. So if they're like, I want a love story about, say, a dragon or something like that, like a medieval love story, they can just search that. And you're getting 98% of the user experience.

Wheedling Zebra: Yeah, that all sounds good. I'm actually going to stop us there because I think any other piece we dig into is just going to take a while looking at different pieces. So yeah, in general, first on your end, did you have any questions about— The problem or the different paths we went down?

Double Hermit: No, I don't think so. I think the AI section added a lot of interesting complexity.

Wheedling Zebra: Awesome. So yeah, let me talk— let me pull up my notes and I can talk through some of it. In general, I think you did really well. Good start with the functional versus non-functional requirements. One piece here, and I mean, we talked through it and I sort of— in the questions I was asking you, I got a sense that you knew, especially on the nonfunctional requirement side. But it's usually pretty common in system design to start talking about, like, daily active users, like, and actually writing them out as part of our nonfunctional requirements. Thinking about the scale of the books that we're going to be working with, the scale of how much data we plan on going back, and we talk about lowly and see like, what does that mean? Are we talking milliseconds? Are we talking seconds? Are we talking, you know, what our range is? So we were sort of going through it, just talking it through, but it's usually a lot easier if you sort of write those out from the beginning. And especially me not having to prompt you for them. I mean, half of them you sort of went there by yourself. Half of them we got through me just asking some follow-up questions. And that should be, like, very easy from the beginning, just like vocab and numbers that you can kind of spit out based on the scale of whatever problem we're working on.

Double Hermit: Okay.

Wheedling Zebra: In terms of the actual design, again, I think you did really well. Kudos, especially like making sure to mention auth and observability without me having to ask you. Those are pieces that I think a lot of people miss. Like they know about them, but it's not like the particularly interesting part of a design, right? And so really good job talking about those upfront.

Double Hermit: Front.

Wheedling Zebra: I think there was a difference in the first half and the second half of the design of like how much you were driving edge cases versus asking me kind of direction. And so I think in the first half when it was kind of just a simple like let's design this backend API, which granted, yes, it's not that interesting, but I expected you to drive a little more of the cases of like, you know, are we going to focus on search? Are we going to focus on the database next? Are we going to focus on the API? Like checking in and seeing, making sure like you're on track is great, but especially at a senior level design, like there's usually a lot less interaction from me as the interviewer aside from just like if you're going down a path I don't want you to, I might refocus you somewhere else. But I kind of assume you can drive a lot of that design without me having to kind of like guide you in the different directions. But that was completely different for the second half of the design. When we started talking about the AI piece, I feel like in there you were definitely driving a lot more of the like, okay, here's the scenario, here are the edge cases, here are the workflows, here are the things we need to think about for each of the pieces. And so again, I don't know if maybe that's just because that problem is a lot more interesting because there is just a lot more kind of complexity you can add there, a lot more like different things that you can talk about. But it would just be sort of like a warning to make sure you are doing that sort of in all parts of the design if that makes sense.

Double Hermit: Yeah, that makes sense. I did kind of panic because I was like, it's like a very simple design that we have. So I'm like, where can this fill? Stop and think.

Wheedling Zebra: Yeah, which is totally fine. I think I intentionally started out simple because people can focus in a lot of different areas. So especially if you get that feeling that you're like, maybe it's too simple, you already have the pieces, right? Like you were looking at the database, you knew you had to look at the API, you knew you had to look at the cache. Just start working through the pieces. Usually, especially at a senior level of design, there's going to be a point in the design where, you know, it may not shift to AI, which is the path I kind of took. But usually they want you to get like a single— a simple thing laid out, and then they'll add an edge case or a new requirement or a new kind of like sub-area. And kind of go from there. So I think you handled it really well, just kind of be ready for it on that end. And let me see if there's anything else I missed. You have really good knowledge of all the technology. I think that's really good. You know, we were talking about Kafka, we were talking about Elasticsearch, we were talking about caching. Different types of storage, I think that's great. Make sure just sometimes to explain your rationale for why you're choosing certain pieces, like what is the usefulness of having something like Elasticsearch? Why do we want and why do we think it's good to have an event-driven message worker pool situation? Even if it's all of like 30 seconds because it's like, well, this is kind of the standard, but the reason it's the standard is because You know, in scenarios like this, it's really good for batching, or it's really good to like distribute the work between a lot of workers, so you're not kind of hanging if something goes down, right? So even just a really simple like, "I'm choosing this thing and this is why," keeps me from then having to ask you the follow-up or wondering like, does he actually know why we should do this? Is this just like the path you chose and I have no kind of insight into kind of your design thought process? Okay. Yeah, other than that, I think you did great. Do you have any other questions, any other follow-ups you may want?

Double Hermit: Yeah, no, I don't think so. I think— Yeah, no, I don't think so.

Wheedling Zebra: Awesome. Well, we can go ahead and wrap there then. I know we're 4 minutes early, but I'll go ahead and write up my feedback for you as well. And yeah, good job.

Double Hermit: Okay, cool. Thank you. All right.