1. System Requirements and Traffic Estimation
Translating user requirements and monthly active user counts into concrete request-per-second (RPS) metrics by accounting for geographic and time distributions.
Hear it at 0:31 →Engineering video
34:51 · 29 May 2024
Translating user requirements and monthly active user counts into concrete request-per-second (RPS) metrics by accounting for geographic and time distributions.
Hear it at 0:31 →Separating front-end and back-end services, utilizing Content Delivery Networks (CDNs) for caching static assets, and implementing compression and cache busting.
Hear it at 4:41 →Scaling backend compute resources horizontally using stateless services and distributing incoming traffic via load balancers with static or dynamic algorithms.
Hear it at 6:47 →Structuring clean, resource-based URIs and properly utilizing HTTP verbs for actions while avoiding anti-patterns like putting verbs in the path.
Hear it at 9:39 →Decoupling shared databases, calculating read-to-write ratios, and implementing read-replica clusters and database sharding.
Hear it at 11:54 →Using in-memory caches like Redis to handle asymmetric product traffic and managing cache limits with least-recently-used (LRU) or least-frequently-used (LFU) policies.
Hear it at 19:10 →Transitioning from synchronous service-to-service calls to an asynchronous event-driven architecture using message queues for better scalability.
Hear it at 25:09 →Securing applications using the least privilege principle, firewall rules, dependency vulnerability scanning, rate limiting, and input sanitization.
Hear it at 31:34 →hi there Foxx it's POG here from the senior Dev today we will solve a real system design interview focused on the backend this interview I got myself for a senior backend position and I passed the interview the offer was around 100K in the European market if you're in the United States Market this would be around 180,000 so a very highend kind of salary and I will go through exactly what I did in my process and the final design uh that actually got me to pass the interview let's get going so basically what they've been presenting me with is this uh kind of initial sketch so I didn't have to care too much
about the functional requirements but the first thing you want to do when you're presented with a problem in system design is you want to make sure you extract concrete system specifications from the requirements they give you and you do this by asking smart questions so what they presented me was with this e-commerce system where we have a store service that has a front end and a back end together there's a pogress DB that is shared by another payment service which is a wrapper around the stripe API and it has the smaller re for the payments and then there's a warehouse system that we need to send notification from and we fetch
the inventory from and what they told me to do is to um kind of redesign the system as it would be uh if it scales so again if this would be a normal system design interview and you start from scratch make sure you number one um map out the uh functional requirements so this is basically just ask about um about the user stories so okay what do we need we need a back end they need to upload a picture we need a form and we need datab B cool so the functionality and the second is the nonfunctional and here there's a couple of them so there could be things like security um we have web performance if it's front end or
accessibility to another one and um and then there is scalability and for this interview the focus was scalability so they basically tell me can you make the system scale to a million users right and so the first thing when it comes to scalability you want to take that data they present to you and really break it down into a concrete system requirement so they might tell you uh and they told me in this case well we are going to have one million monthly users a a junior will completely overlook this but a senior candidate or more expert candidate will try to translate this into the exact request per second and
how would they do that well the first thing you want to ask is okay um what is the geography that they this users are around so is it mostly people in the US are they all coming from the same um geography or is different geography and the second thing would be kind of the time distribution so do we have this three distributions and here it's an e-commerce use case so it's for sure we have time distributions because ecomer it's all about promotions and things like Black Friday or Christmas so we do have PE traffic that's very important and third one is traffic is not Bor and equal so when it comes to e-commerce and
products usually have a lot more traffic on the most advertised products so we will have sort of a product um distribution in this case is products if it would be Instagram it would be or social media it would be post from the user so you have some viral posts they receive the most traffic keep this in mind we have 1 million monthly users most of them would be um uh you want to arrive with the final number this is 1 million now I divide that by um not really 24 hours because traffic is not equally distributed through the day let's say half of those visits would happen in an interval of one hour
between 8 and so I actually divide this by two and then by 60 minutes and then by 60 seconds and that would be my final number you ideally extract a RPS request per seconds and based on that you can then make your math as you see here I did some high school math nothing complex it's not about getting the exact number or the exact requirement it's about having a good mental process of course in production you would have number on this and this Google analytics or whatever analytics they have and tell you what the distribution is here but if you do this number you'll already pass so now that we have some rough traffic
number we could also estimate the throughput so if we have product images we could actually um extract how much Network capability we would need but we will focus here on later see uh because nowadays with Cloud you you don't need to calculate your put so aggressively like we used to have in the past when you had to build your own kind of thing so let's go in and just start scaling this now the first thing would be you want to in this case split the service between back end and front end because they can be scaled independently so we will have a front end store service that would be a single page application let's
say a react application or an angular and then we'll have a backand service and we will scale those independently let me get some space for us here we are let me zoom out I will move my requirements over here so step number one is done we have the requirements step number two you want to design a minimal minimum viable uh system we already have a minimum viable system in this case because they gave it to us this is a real interview that's what they gave me um so you start directly into okay let's scale the system and each one of the components so first thing number one split between front end and back end number two because we have
a content component one of the easiest way to scale it is to add a CDN so I'm going to add a CDN component here content deliver Network this will basically create a copy of your front end static assets and we'll get into a minute we'll uh we'll understand is this really static because we could probably save or render part of it but for now we'll assume there are static assets it's cash apply direct caching policy distribute them it cashes it distributes them and it also applies standard HTTP compression all the browsers will receive a compressed version of your CSS there it's an content encoding header
you'll get that says hey this is jip It's cached uh with a cash policy and ideally we also Implement cash busting in our front end if you don't know what it is make sure you look it up cash busting us make sure the users actually get the latest version and they don't keep the one that's cashed um and you might get a question about cash eviction cash eviction is how do you clean this up we'll get back to this when we talk about cashing the product right now it's not so important when we talk about static assets CDN things like Cloud fair or the AWS AWS has very good CDN solutions to very is to implement the
the first thing in scalability now let's move on to the back end so you want to really um go from the front end to the back end because a lot like like I know okay the static assets are scalable but then people start ordering and then it puts a lot of pressure on our backend service so how would this look like well there's two ways to scale services that need compute so they need CPU power they need a server they they need to execute something to give an answer and these two ways are horizontal scaling or vertical scaling vertical is basically we shut down the server and increase add more capability to it very not practical
nowadays also very expensive very tedious and it has downtime and is not elastic at all what do we mean by elastic elastic means we can automatically scale in or out so horizontal scaling we scale in or out with um as the traffic goes with vertical scaling we need to really shut down the machine and start with a new machine that has less gig ram so we go back to an eigh and then all the way to 32 very tedious and very expensive not recommended what you'll do nowadays mostly is use horizontal scalability what does this mean well you're going to spin the up into four instances let's say and then group it into a fleet of
instances it's extremely easy to do this with for example to like AWA and then put a lb a load balancer we have our load balancing component over here so all of the sudden the front end will redirect traffic to our load balancer and your load balancer will distribute traffic what kind of algorithm will you use for the load balancing so the default it's round robin make sure your look this one up around Robin uh this is kind of a static load balancing algorithm as they call it because it doesn't really need to know anything about the instances but it's also Dynamic uh algorithms like the list connections the load balancer will keep
track of which instance has how many connections and always redirect to the instance that has the least connections opened right now make sure you do know this uh these two so Dynamic load balancing in static round robbing there's a bunch uh there's a more more of them you don't need to know all of them in an interview you're not the working Wikipedia they don't expect you to know all that but you do want to know one or two and you do want to know this difference between static and dynamic what does it take for this to scale horizontally we need to have stateless services that means there cannot be any
valuable that we stores here in state and you need for the next connection so Connections in and request to this load balancing should be independent if a connection or a request arrives to this first in or to this for it should make no difference that's why all the state should live in the DB so we'll move our state to the database that's the only way we can scale horizontally and this is why we need to use a rest API rest is stess by default um so it's very easy to scale horizontally graphical can also work we'll discuss about that later important question in a interview in the backend interview if you go you want to
know this yes or yes they will ask you to out a rest endpoint so in this case they ask me about hey if you had a slash products how would the rest interface look like to get all the products I need to make a get request to SL products to get an individual product I will need to do this so basically laying out the URI for your standpoint so that would be how we get products uh to create a product I need to make a post post with the body of the new product and then you can choose between but slash patch to create a product um and this would either you know create if it doesn't exist and then there's a delete it's not
going to be open to the public um so that's how uh the rest end point will look like I was asked to do this in many backend interviews just write down the interface you don't need to code it but we'll ask you to know it big Pitfall here um you see people doing this like get products and they would say something like delete that's a big mistake in guust the um action it's indicated by the HTTP verb so make sure you don't do this or you don't do like products SL getet one that's a total unrestful way to design API and you'll use points in interview and uh the reason I mentioned this is because I
even saw a lot of YouTube videos very very V viral videos on system design and backend that we're doing this basic mistake you're not going to pass interview this mistake review how rest API should look like and also how a non restful URI looks like make sure you know this now once we have the rest interface and we have our load balancing component we talked about load balancing um different ways to load balance this um all this can be done with AWS cloud services or any other cloud services we end up putting pressure on the database so really important here I start with the traffic being here and I just move
it all the way down solving button neck as they appear and this is one of my most favorite parts so first of all share databases are a big anti pattern you don't want to have share databases that are extremely hard to manage so the first thing I would tell them is hey we are going to separate this into payment and the store and we will try to have you know just they can reference each other they can act these two Services can access each other through their apis but they won't access each other database because if one service breaks the database um they're both down and the other thing is it's very hard to
scale a database that is getting bigger we need databases are harder to scale so it's much better to have them separated and unified with their services now in this uh section we do know that a lot of the request here I made to read products because a lot more people will read products like view a product than actually Place orders or modify products that's why you want to extract really quickly a read to write ratio I'm going to add it here it's one of the most important read to write ratio one of the most important Concepts so once you know that it's much easier to scale different parts of your system so um the r part of
the backend story service will be much more demanding um and in a radical case if you really had to scale this to the Moon you could actually separate this and have a service that it's only a right service for our back office people or for the warehouse warehouse system and then a read Service and I completely separated and this one it's very optimized for scale and the other one is Maybe optimized for availability but it doesn't need so many resources because there's no more traffic so whenever you have an asymmetry of traffic on endpoints you can split those in independent pieces that can be scaled
independently what does this means to the database because there's so much pressure on the read versus the ride we can ideally for a long time still keep this in a single postgress instance but we will Implement a read replica pattern where we have a single a main database that is the one um where we can write product and then there are replicas in a DB cluster that get synchronized this would be our database Fleet whenever you just want to read you would go to the read replicas and because there's so many of them it's just super scalable so when you write you write to the main one and when you read you read here now do
you need to implement this by yourself do you need to know how to um set this up in uh pogress not really if you're using a service like IDs in in AWS IDs even by default they offer you the option to have a cluster with three read replicas in one main right so they do that by default for you so that's how you scale that's one way to scale the database now if you are talking about databases make sure you also look into database normalization it's a huge topic you want to look into DB indexing so one of the common questions here uh that you want to mention that will make you stand out as an expert it's they would ask you
well okay well we have a table um that has a million records in we have a very slow Full Table read and when they mention this is specifically because they want you to talk about indexing indexing is basically a way to create indexes for a specific table based on certain columns and it makes it much faster to read you can look into it uh but because you create an index basically the algorithm to read from the database it's much faster rather than going to um making a normal like an array look up you do something more similar to binary search I'm not going to get into in depth into that because
I'll spend you know one hour only talking about this but make sure you look this up um and the other thing is database caching so you could actually cach a different levels databases already have a lot of caching in implemented in them uh they cache their different pages internally in order to um minimize the operations from reading and writing to the disk but there is such things also as a caching layer you can add and a lot of those are also relatively easy to configurate in today's cloud services there it's a elastic cache in R based cach you can add to RDS for example uh so pretty easy to uh to scale and this would already be
a really good solution if this is not enough to scale we will need to look into things like sharding um sharding means we will split our database into three databases let say so we have like one two and three and we would hash certain properties so we' have consistent hashing imagine depending on the ID we have like let's say we hash based on ID well depending on ID you'll write the record here or here or there depending on where you're at and so you need a component that finds the right chart to write too it's a complex solution and it's only fit if you really scale a lot and you cannot like this kind of read and write split
doesn't work anymore um really important for you make sure you look up the cap theorum if you mention this in the interview you will stand out as a senior cap theum says we can only have consistency availability or partitioning so in this case right if we apply the cap torum we do know that we do have um availability we can be quite available um we do have petitioning because we distribute our database but consistency would be a bit harder uh that means we might have if you if we just update something and somebody reads the same record it will take a bit of time to update that record to all these replicas
so we will be not super super consistent again if you're a fullstack developer um you don't need to be a database engineer you that needs to know these things super in depth but database indexing normalization this way of read write kind of scalable replication pattern and sharding are the basics in the main Concepts that you'll ask if you know this you probably know 95% of everything you need to know about databases the other pattern I want to get into because you're still discussing about databases somebody passed the senior devil they will ask some mention using no SQL and SQL for in as a combination for actually
really finding out okay are there any parts that like of the of the data that are more to store them in a no SQL database and for example can I store other parts in the SQL database this all depends in the data how the data looks like um in our case we have products and prices and offers so it's very much a relational data but if you get a problem that's a more of a social media feed uh no SQL makes sense you can store your users in a relational database and anything else in a non- relational database because you don't need relationships there and no SQL it's much easier to scale it kind of scales out of
the box so make sure you know also that there is this alternative where you look at your data kind of take your data you map your schema of your whole data let's say and you kind of with the interviewer you can tell them hey you know what I would probably make this subset of the data live in a and this other subset in an SQL database um this is a very Advanced concept and it will make you stand out so let's riew our system so we kind of have a front end um decently scalable we'll get back to it our backend stores pretty um scales pretty well but remember we said that there would be there might be traffic
based on product so if our marketing department goes in and says we're going to promote this specific product zando does that all the time where you get an email with specific look and people will click on that and land on that page you can actually also have here a cach so the thing with a cash um let say it to leave here and that would be more of a red cach or whatever it would be that you don't really need to hit the backend store service to get a product you can just immediately get it from the cache um that's very useful when you have this kind of asymmetric traffic to specific products and you don't want to
scale the whole thing just to satisfy the need of that one record now regarding this cach over here you can specify red as a decent uh caching solution or M cach they're both available um in the cloud and pretty reasonably easy to deploy if they want to go very in depth make sure you look at Cash eviction uh this would be the way to empty this cash um what kind of cash eviction policy you want to set up you don't need to know all of them but lru which is basically L IU stands for the least on second so let me write them here list recently used recently and the list frequently used so that would be lfu
give me a second LF you that will be the least quently used so these two um basically mean okay do you want to delete from the cash the ones that were U read like a long time ago it was the last time we accessed that or the least like they're not good a lot maybe maybe somebody looked at that product today but in generally then look at it a lot and this depends on your use case for any Commerce use case of course you want to go for the least frequently used but if this would be a social media post a social media website um like Instagram it's much better if you go for the Leist recently why in social media things fade
really really fast so whatever video was popular three months ago is not popular anymore but in any comments use case it goes more for products products are more popular like 80% of the users would buy the same three products from from any store depending on the store so again you need to understand the data in the use case and then adapt those patterns to the specific use case um you just need to know the two of them and then really understand the context of your problem if you know these two things you should be fine now the last thing would be um basically we scale each we scale the database we scale our backend
Services we could apply the same recipe to any backend service we Define our rest API um you might get ask about graphical versus rest this would be a good case to switch over to graph qu because we have a very data reach interface uh these products can be very big you might have different queries and you want to give our front end the opportunity to query those independently that would be graph QA so this one again should be St less we should scale it independently um but it will actually get direct traffic to the specific Individual Services and then we can split the backst stream Services into
individual ones so that's one of the patterns you want to look at up the back end for front end um the cool thing about having this back end for front end is we can actually make it also handle things like https and make everything Downstream HTTP why would we do that and why do you want to mention this in the interview the reason for that is that HTTP is much faster uh the https handshake has three extra steps to make sure uh we establish a secure connection whereas HTTP it's a bit faster so if if you have a virtual private Cloud you could actually make it HTTP only and block the access only to
the outside and say okay whatever comes from the internet has to come over https but whatever it's inside my VPC I trust it to be over HTTP and that's how you decrease a bit the latency that you have between these two Services also make sure they are geographically close to each other in the same Cloud one of the questions they might ask when you go into https if you go for a really Advanced interview it's what is the difference what is asymetric encryption and how does it work in the context of TLS and HTTP you don't need to go too deep into this what basically happens is that the server has a private
and a public key the public key is containing the htps certificates that get sent to the browser the browser take that certificate and it generates a unique key it signs it with that certificate it sends it back to the server the server can decode that with a private private key and then uh that's where the asymmetric encryption ends and from now on it will be symmetric encryption everybody will encrypt their message with that key that was generated by the client so you might ask yourself why not we use asymmetric encryption for the whole process it's because this is very expensive computationally so you
just use it to exchange a secret key to switch over to symmetri encryption you then need to know much more about htps it protects us about a man in the middle attack make sure you look one up that's how you stand out as an expert so the difference between somebody that's junior is that Juniors will just kind of they know it's HTTP but they don't know this key concept on it or how does it work under the hood um again you don't want to go like you that no one knows these things to the bone no one knows the complete implementation but you want to have this mental model that I just described as you see this is we almost
complete with stage one which should be to scale our um first kind of each one of the components now if you look at the system as a whole and you unack this with your interviewer and tell them hey you know what I have a feeling if we want to scale this fighter we might have to make some considerable architecture changes and it's usually uh two changes you can make the number one change would be you can move into an event even so instead of having the services calling each other so you have all this the backend store calling the payment service and then calling the warehouse and then calling payments you move into
an event given kind kind of service where you'll have a center point this is our event queue and the services are just listening and Publishing events to it if you want us to make a video about it this is kind of an Advan it's it's a very wide topic by itself um and a lot of you um just like I had in the past you you don't necessarily get the exposure to it unless you work on a specific part of it because most companies don't necessarily implement this but basically the way to look is you have an sqs or an event queue and you publish things here like order placed so rather than calling each other
the services will listen to this so imagine uh there's an order placed and the payment service will listen to that when the payment is complete then the payment service will push another event in this Cube which would be payment complete and there might be some email service that's synchronous that's listening to all this and in parallel as synchronously sending emails so event event is very good for synchronous communication uh if you have something synchronous you still need your services to have against apis but a lot of the things that happen in any business application can be synchronous sending
emails analytics for example it then need to make a synchronous call analytics so this is a perfect use case to just plug Services into this Central Enterprise event Hub which can start very small with something like sqs make sure you look this one up or go deeper into something more Enterprise even like a ccon these Solutions all leverage the same an event queue that different services will subscribe to you can also have web Hooks and other methods I'm not going to get uh deep into them because of course I want to keep the scope reasonably small of this video but this is one of the most significant changes
you can make so you switch the architecture when you start with a functional system you start scanning the individual components but then it gets to a point where you can actually switch the whole architecture to make it more scalable and that's when you move from what we had here which was moving into microservices to microservices with event cues or event given microservices now we are pretty much done um a lot of the questions in a system design interview might also go around okay how would you deploy this and you want to have exposure you want to at least know how to deploy one single service um when you have
microservices the best way is to put them into something like Docker package them and then have one single pipeline that just deploys these Services as containers it will simplify immediately your deployment process uh if you never touch on this make sure you at least deploy one service yourself maybe the pipelines are already set up or you don't have the experience do it in MVP and don't use something like netlify or sell they're too easy and you cannot sell that as a senior you need to really do it to the cloud um AWS gives offers free tires just make sure you log in create a small application and deploy it
and then add more complex deployments now when it comes to the front end this is an e Comas use case so you do want to make sure that you spend enough time talking about this and in E Comas you want to ask them about SEO this is very specific but it's super necessary in an e-commerce use case and because we have SEO um and we want a lot of web performance so the problem in Ecom is you have all these product images so uh things get a bit more complex because you want to have high performance in SEO and this is when you look into server sign right rendering that means you render a lot of those pages on the
server and the other thing you could actually look into it's into static rendering that means you pre-render the HTML for all these product pages what is the advantage there it's super easy for the browser to interpret that you'll get a lot of performance the disadvantage is if you have a lot of changes in the inventory so remember we had this Warehouse system or you have changes in the price um you will have to kind of somehow clean the cash or reender those so in this case you might even look into different micr frontend solutions where you split this and um if you want us to make a video on micro frontends and how
am I making these choices let us know in the component in the in the comments but basically you split your product page into different micr frontends and you literally isolate the one the one part that displays the price in the inventory that one will not be will not have a lot of caching because you if you apply cash there you will lose consistency so uh you might get a lot of speed but maybe people see a different price and then when they go to the check out they get angry because the price change whereas what you want to do is to statically render all the other parts like product images and descriptions they don't
change that much but maybe keep one part Dynamic and you can do that with micro front end now remember this interview it's full stack so you don't want to talk too much about it especially if you're interviewing with a back end person they want they cannot really too much to that part of the uh of the infrastructure and if you don't know it too well just mention it as an idea look one of the ideas that I think my work here would be this one and get feedback from the interviewer uh don't pretend you know everything about it but you tell them hey look um I heard or in my company this How We Do It um so Ser
side rendering micro frent would help us scale our architecture as a side topic you might get asked about authentification in a typical system like this one you might also add an identity server uh you can look at uptic tools like OCTA orzo would allow all the services to tag to each other and it would allow you to implement a token based kind of jvt based authentification in the front end following um or two and it will basically give you authentification between all the services and your users there's so many tools you can use you don't need to code this one yourself there's open source ones or there's also to like OCTA or
Cognito they might ask you about a specific plow in in a system design interview there's not enough time to uh go too deep into it so don't worry too much about it um and the last topic would be security so we discussed https really quickly when you're building uh certain um cloud services you also want to mention the fact that you will apply the least privilege principle um and you can do this for example in AWS you would do that using um security groups Services should be should only give the per the minimum permissions to other services that they need to do their job not more so if we have a service like
this one um you would have a firewall around it that for example will limit all the outbound calls of the system to the ones that you actually or the inbound calls would be limited to the port of the front end where front end communicates with us and everything else would be closed so everything is closed by default and we only open the things we need that's the number one thing you also want to mention dependency scan dependency scan especially if you work with JavaScript and there's a lot of vulnerabilities that are introduced by nodejs packages and you can have this in your pipeline so every dependency that's
being added gets scanned uh and we have an idea of okay what are the dependencies in our code or we just don't add packages that have vulnerabilities we mention https and then there is things like rate limiting so you want to make sure that your services um would not get attacked uh with the DDOS attack the OS attack and you can do this with traffic filtering and load balancing or um yeah you can filter traffic from specific regions if you know you get attacked by then but again you don't need to like go super in depth about this but you need to know about it and the last thing for security would be things like input sanitize your
input and uh protect against SQL injection with an orm so if you have something like an SQL database and that's pretty much how it would all look like uh we have our event que that gets like different events let me put them in different colors so we can see and then all these other services kind of looking into it I'm going to move this up here this is more on the database discussion and that's pretty much how a good solution will look like what's important here is that you can actually go from the big picture to the small picture and you don't run out of things to say a real senior and a real expert could
probably spend three days on this uh design uh because they could go like service by service and go really in depth and you also want to get buying from your interviewer depending of how like you are sometimes interviewers are really cool sometimes they're not that cool but you do want to um work as a team so try to include them in a discussion and then make it a monologue what I've done here looked a bit more like a monologue because I'm talking to you but usually you want to ask questions all the time and say hey okay let's focus now on the database so let's talk about this what are the requirements here and have this kind of
up bit curious Spirit not the feeling that you're against the wall that's probably the the interview antipathy number one it's you just feel like okay they're asking me all this and and get nervous but they other be like let me think about this differently if you do this you should be in a very good position to pass the interview also make sure you check our playlist on real interview questions that we got or our M's got and I will see you in the next one