Vertical vs Horizontal Scaling: How to Choose

Vertical vs horizontal scaling is a choice between making one machine bigger and adding more machines. Vertical scaling, or scaling up, gives a server more CPU, memory or faster storage. It needs no code changes and stops at the largest machine you can rent. Horizontal scaling, or scaling out, runs several copies of the application behind a load balancer. It has no fixed ceiling and survives the loss of a machine, but it only works when no single server holds state the others need. Most systems should scale up first, then scale out the stateless parts.

Vertical vs horizontal scaling: the definitions

What is vertical scaling?

Vertical scaling means replacing a machine with a larger one. The application, its configuration and its architecture stay exactly as they were. You pick a bigger instance size from your cloud provider, the machine restarts, and the same software now has more cores and more memory to work with.

What is horizontal scaling?

Horizontal scaling means running more instances of the same thing and spreading the work across them. For a web application that usually means several identical application servers behind a load balancer, which sends each incoming request to one of them. Capacity grows by adding instances, and shrinks by removing them.

Horizontal v vertical: a side-by-side comparison

	Vertical (scale up)	Horizontal (scale out)
What changes	The size of one machine	The number of machines
Code changes	None	The application must be stateless, or made so
Ceiling	The largest machine available	No fixed limit for stateless work, until a shared resource such as the database fills up
If a machine fails	Everything is down until it comes back	The remaining instances carry on
Changing capacity	Usually a restart, so a short interruption	Instances join and leave with no interruption
Cost pattern	You pay for peak capacity all day	Capacity can follow demand, at the price of more parts to run
Operational effort	Low: one machine to deploy to and watch	Higher: load balancing, shared state, logs and deploys across many machines
Best fit	Databases, early-stage products, steady workloads	Web and API tiers, background workers, spiky traffic

The two are not rivals. Most production systems use both: a few reasonably large application instances in front of a database that has been scaled up.

Scale up vs scale out: what each costs and where each stops working

Where vertical scaling stops

Scaling up is the cheapest option in engineering time, and I think it is underrated for that reason. One change to an instance size can buy a team months. It runs out in four ways.

• The ceiling. Every provider has a largest machine. If your growth is steady, you can see the date you reach it.
• The restart. Resizing normally means stopping the machine and starting a larger one, which is a short interruption each time.
• The single point of failure. One large machine is still one machine. When it fails, or needs a kernel patch, the service is down. No amount of memory fixes that.
• Paying for the peak. Within one family of cloud machines the price rises roughly in step with the size, so twice the machine costs about twice as much. The catch is that a machine sized for your busiest hour is also what you pay for at four in the morning.

Where horizontal scaling stops

Scaling out costs more in engineering time and less in hard limits. The bill arrives as complexity.

• More moving parts. You now need a load balancer, health checks, a way to deploy to many machines without dropping requests, and logs collected in one place.
• Shared resources become the limit. Ten application servers that all wait on the same database are no faster than two. Adding instances moves the bottleneck. It does not remove it.
• Connections multiply. Each new instance opens its own connections to the database, the cache and other services. A database that was comfortable with two application servers can run out of connections at twenty.
• Bugs that only exist with more than one copy. A scheduled job that now runs on every instance, or a user whose next request lands on a server that has never heard of them.

Statelessness: the precondition for horizontal scaling

Horizontal scaling works when every instance is interchangeable. Any request can go to any instance, and any instance can be removed at any moment without a user noticing. An application with that property is called stateless. The state still exists, but it lives in shared services, not on an individual server.

Most applications that were built to run on one machine are not stateless, and the reasons are usually the same few:

• Sessions kept in memory. The user logs in on one server and appears logged out on the next. Move sessions to a shared store such as Redis, or use signed tokens that any instance can verify.
• Uploads written to local disk. A file saved on one server does not exist on the others. Put uploads in object storage.
• In-process caches. Each instance builds its own cache and they disagree. Use a shared cache, or keep local caches for data where a slightly stale answer is harmless.
• Scheduled jobs. A job that ran once on one server now runs once per instance. Give it a lock, or run a single scheduler that places work on a queue.
• Long-lived connections. WebSockets tie a user to one instance, so messages meant for that user need a shared channel that every instance listens to.

Load balancers offer sticky sessions, which pin each user to one instance. I would treat that as a stopgap. It hides the problem until an instance is replaced, and then every user pinned to it loses their session at once.

The test is simple: could you switch off any one instance, right now, without anyone noticing? If the answer is no, you are not ready to scale out, and you are not ready for a zero-downtime deploy either.

Databases are the hard part

The application tier is easy to scale out precisely because you moved its state somewhere else. That somewhere is the database, and a database cannot be made stateless. I would take the options in this order, and only move to the next when the current one is exhausted.

1. Fix the queries and indexes. Covered below. It is the cheapest move and often the largest gain.
2. Scale the database up. For a database, vertical scaling is the correct first response, not a lazy one. A single well-tuned relational database such as PostgreSQL on a large machine handles far more than most teams expect.
3. Pool connections. A connection pooler in front of the database lets many application instances share a small number of real connections.
4. Cache reads. Results that are read often and change rarely do not need to reach the database every time.
5. Add read replicas. A replica is a copy of the database that follows the primary. Read queries go to replicas, and writes still go to the one primary. This scales reads and does nothing for writes. Replicas can also run slightly behind, so a user who saves a change and reloads may briefly see the old value unless their next read goes to the primary.
6. Partition or shard. Sharding splits the data across several independent databases by a key, such as the customer. It is the only option on this list that scales writes, and it is the most expensive. Every query has to know which shard to ask. Queries and transactions that span shards become difficult or impossible. Moving data between shards later is a project of its own, and the choice of key is very hard to reverse.

Managed and distributed databases take on some of this work for you, with their own constraints on queries, transactions and cost. My opinion is that sharding is adopted too early far more often than too late. If you have not done the first five steps properly, you are not at step six.

Autoscaling: letting capacity follow demand

Autoscaling is horizontal scaling run by a rule. You choose a metric, such as CPU use, requests per instance or the length of a job queue, and set a minimum and a maximum number of instances. The platform adds instances when the metric rises and removes them when it falls. Cloud providers offer this for groups of virtual machines, Kubernetes has a Horizontal Pod Autoscaler for containers, and serverless platforms scale per request without you managing instances at all.

It suits workloads that are spiky or hard to predict. A steady workload does not need it. Before relying on it, check four things, then test it under load:

• Start-up time. If a new instance takes several minutes to be ready, autoscaling will always arrive after the spike. Fast start-up matters more than a clever rule.
• The right metric. CPU is a poor signal for an application that spends its time waiting on a database. Queue length or request latency often reflects real load better.
• Downstream limits. Autoscaling the application tier pushes more load and more connections onto the database. Make sure the thing behind it can take what you are about to send.
• A maximum. Set an upper limit so that a traffic spike, a retry storm or a bug cannot scale your bill without bound. I cover reading those costs in what your cloud bill is actually telling you.

When the honest answer is to fix the query or add a cache first

A slow application is not necessarily short of hardware. In my experience, when a system slows down under load, the cause is usually specific and small, and more servers would not have helped. The common culprits:

• A query on a large table with no suitable index.
• The N+1 pattern, where a page runs one query for a list and then one more query for every row in it.
• A query with no limit, fetching every row when the page shows twenty.
• A slow third-party call made while the user waits, when it could run in a background job.
• The same expensive result computed on every request when it changes once an hour.

Each of these is a code or database fix. Scaling out makes several of them worse, because more application servers send more of the same bad queries to the same database. A cache in front of expensive reads and a content delivery network in front of static files can remove a large share of the load. Caching has its own cost: you have to decide how stale an answer may be, and clear the cache when the data changes.

The way to know which case you are in is to measure. Tracing and the database's slow query log show where requests spend their time, and a load test shows what breaks first. That is the purpose of performance testing: finding the real limit before your users do.

A decision guide

Work down the list and stop at the first one that applies.

1. You have not measured where the time goes. Measure first. Every other decision depends on it.
2. One query or one endpoint is the problem. Fix it, index it or cache it. Do not scale anything.
3. The whole machine is saturated and a bigger size exists. Scale up. It is the fastest fix and it buys time to do the rest properly.
4. You need to survive a machine failure or deploy without downtime. Run at least two instances. This is a reliability reason and it applies whatever your traffic is. It means doing the statelessness work now.
5. Traffic is spiky or growing past one machine. Scale the stateless tiers out and add autoscaling with a sensible maximum.
6. The database is the limit for reads. Cache, then add read replicas.
7. The database is the limit for writes, after everything above. Now consider partitioning or sharding, with care.

One point on architecture. Scaling out does not require microservices. A monolith scales horizontally perfectly well if it is stateless: you run more copies of the whole thing. Splitting into services lets you scale one part independently, which only matters when that part has a very different load from the rest. The microservices and monolith comparison goes through when that trade is worth making.

If you would like a second opinion on where your own limit is and what to change first, that is the kind of work our cloud engineering team does.

Frequently asked questions

Is horizontal v vertical scaling an either/or choice?

No. Most systems use both. The usual pattern is to scale the database vertically and the application servers horizontally.

Scale up vs scale out: which is cheaper?

Scaling up is cheaper in engineering time and usually cheaper at small scale. Scaling out can be cheaper in infrastructure once traffic varies through the day, because capacity can shrink when demand does. Count the engineering and operational effort as part of the cost, since that is where scaling out is expensive.

Can a database be scaled horizontally?

Reads can, with read replicas. Writes can, with sharding or a distributed database, at a considerable cost in complexity. Scale the database up, tune it and cache in front of it before either.

Is autoscaling vertical or horizontal?

Almost always horizontal: the platform adds and removes instances. Automatic vertical resizing exists in some platforms but is less common, because changing the size of a running machine usually means restarting it.