I’ve recently come across some challenges in educating developers on how to architect an application to enable it to actually scale. Scaling is used when the application starts to break under user-load.
What is scaling?
Typically most applications can be run on a single server. The database, cache, application all exist on this server. Single Server applications drastically reduce infrastructure complexity and application architecture. Unfortunately you cannot just copy/paste your application to a new server. The first choice for these more traditional applications should be “Vertical Scaling”.
Vertical Scaling
In short, vertical scaling just means add more resources to the server to meet the new demand. This usually comes in the form of migrating the application to a new server with more resources; although some cloud providers allow dynamic resizing. This typically is a short-term solution and becomes exponentially expensive as the “tiers” of server scale exponentially. For this reason it is much better to scale “Horizontally”.
Horizontal Scaling
This means splitting your architecture. You cannot just simply copy your server, that means you have 2 databases, 2 session stores… etc, this just doesn’t work; unless you do some janky stuff like (sticky sessions, or using sub-domains, … etc).
Much like code, infrastructure should be approached with using the “single responsibility principle” in mind and thinking in terms of abstraction. A simple way to start down this road is too look at tools that easily allow developers to develop these types of applications. AWS ElasticBeanstalk is a great starting place. Understanding how/why of building scalable applications requires a lot of study and practice, but you can get a lot for free by making smart choices at the start of a project.
N+1 Nodes
What developers who are not familiar with distributed applications (which any scalable application is) realize that much like an application, the infrastructure itself is very much structured like an onion.
Let us start with a box, and this box contains all of the components and code for our application. This box defines the boundary between the internet and our application. The component that user’s browsers interact with are HTTP Servers. This could be an ExpressJS App, or nginx configured to serve PHP; but this is the Front-End of our application.
We don’t “know” how many requests we are going to need to serve, but we expect a lot. When we expect that it will be too much for one server, then we need to immediately switch to thinking about a dynamic amount of front-end nodes. This immediately will change a lot of how we plan and architect our code. For our particular example lets assume that we have Logins to deal with, which means we have to have sessions.
So if we have say, 3 servers running our ExpressJS application. How do we handle database access? Guides around the internet will always just assume, “Localhost”. Well that’s not going to work, each of our servers has a different set of data? Seems bad. It is. So we need now 1 more server for database, that these 3 servers connect to.
What about logins/sessions? Well when you google session code again they assume “localhost” is fine. Well now on subsequent requests the session only exists on 1 of 3 nodes. What happens if a user connects to a node that doesn’t have that session? Oof. So now we need another server to act as a datastore for sessions. You can use your database, or something else like Redis, (hint redis is super good with this, databases are not). So thats another +1 for a server.
Well I might have forgot to tell you about another piece of equipment you need, which is a load balancer. When a person connects to your application it needs to be routed to one of your 3 servers, well that’s another piece. By now i’m sure you are thinking, omg that’s a lot of stuff to know… Yes it is. But fear not, once you figure out the how and whys it quickly becomes second nature.
That’s a lot…
Well let’s go back to ElasticBeanstalk on AWS. You get a lot of these components for free. Loadbalancer? check. Scalable Application Nodes? check. Logging? check. Database? check. That’s most of the battle right there. Beanstalk does a lot of the heavy lifting for you, which lets you just focus on your application.
Conclusion
Vertical scaling is a great short-term solution, however costs are typically exponential which makes it a not-great long term solution.
Horizontal scaling is great for starting down the road of building systems that have inherent redundancy, since it’s somewhat trivial to just add new nodes. You need to understand how each infrastructure components SDK works when things are not just Localhost