There can be three reasons to scaling-out your installation:
- High throughput (EPS)
- Indexing & Search performance
- High availability
Generally speaking, it is always simpler to add more CPU power to the AiO (all-in-one) installation. A single core should be able to handle roughly 2K EPS, so a 32 core server should be expected to handle about 60K EPS.
If you require higher EPS and you can add CPU - we recommend doing that.
If you are limited due to available hardware sizing, or if the machine already has more than 48 CPU cores, it's time to start scaling out.
Indexing & Search Performance
We recommend not working with more than 64GB RAM for a single node (this is due to Sophie running on top of the JVM, see here).
From our experience, if a single day spans more than 30GB of data, it's time to scale out.
Of course, a single node cannot support high-availability. If this is a concern, you should scale-out..