Designing a lightweight and scalable frontend.

Some parts of most applications located in the internet are accessible for everyone. This may be an API for user clients, a simple website or a login mask for a member area. These parts are, from a scalability point of view, most critical.
One problem when planing an application that should be able to match any imaginable amount of requests is the imbalanced load between client and server. For a http request a client simply needs to establish a tcp connection and send a http request though it.
The server on the other side also has to establish the communication and then generate the content. If we think of of typical website the server(s) may have to parse some thousand lines of code, execute it, make some database lookups, put some content together and deliver it.

An example would be the new TYPO3 Neos CMS. After a fresh install, setting it on production level I run a ab-benchmark against the site.

ab -k -c4 -n1000 -g neos "http://neos.demo/"

Server Software: Apache/2.4.7
Server Hostname: neos.demo
Server Port: 80

Document Path: /
Document Length: 6594 bytes

Concurrency Level: 4
Time taken for tests: 12.393 seconds
Complete requests: 1000
Failed requests: 0
Keep-Alive requests: 992
Total transferred: 6882652 bytes
HTML transferred: 6594000 bytes
Requests per second: 79.69 [#/sec] (mean)
Time per request: 49.871 [ms] (mean)
Time per request: 12.393 [ms] (mean, across all concurrent requests)
Transfer rate: 542.36 [Kbytes/sec] received

Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.0 0 0
Processing: 37 50 8.5 47 94
Waiting: 37 49 8.5 47 94
Total: 37 50 8.5 47 94

Percentage of the requests served within a certain time (ms)
50% 47
66% 51
75% 54
80% 56
90% 62
95% 66
98% 74
99% 78
100% 94 (longest request)

A mean request time of 50 ms and 79.73 requests done by second are quite impressive for such a large framework.

During the benchmark, the host running the benchmark on was idling. On the webserver that has to deliver the content you can generate any load you want just by playing with the -c setting of the benchmark.

That is not surprising. When you take a look on the trace xdebug can create for one request, you see that the server has to call 26889 functions just to deliver the example page. On 80 req/s that are 80*26889=2151120 functions that have to be executed every second. An OP-Code cache helps, but still the OP-Code has to be executed. So the application itself amplifies this most simple form of attack.

So I created some other example. I took the task of displaying this website you currently look at, put this in a lightweight application. To help comparing the results, I also used PHP.
To store the contents in, I chose a Redis database, as it is persistent and very fast.
The only function of this minimal CMS is to do some simple routing, to fetch content from the redis database and render some information into placeholders. All basic function the TYPO3 neos website example does and for example my old page on did.

And now I get with
ab -k -c4 -n1000 -g lygie ""

Server Software: Apache/2.4.7
Server Hostname:
Server Port: 80

Document Path: /
Document Length: 6054 bytes

Concurrency Level: 4
Time taken for tests: 1.624 seconds
Complete requests: 10000
Failed requests: 0
Total transferred: 64360000 bytes
HTML transferred: 60540000 bytes
Requests per second: 6239.96 [#/sec] (mean)
Time per request: 0.640 [ms] (mean)
Time per request: 0.162 [ms] (mean, across all concurrent requests)
Transfer rate: 38697.44 [Kbytes/sec] received

Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.0 0 2
Processing: 0 1 0.2 1 4
Waiting: 0 1 0.2 1 4
Total: 0 1 0.2 1 4

Percentage of the requests served within a certain time (ms)
50% 1
66% 1
75% 1
80% 1
90% 1
95% 1
98% 1
99% 1
100% 3 (longest request)

6239 requests per second and a mean request time of 0.64ms (on localhost). So the page can handle more than 70 times the traffic and serve a single request in only 1.28% of the time. (sending out on a bandwidth of 304 Mbit/s)

There are some functions missing (list view of news, voting, commenting), but I am very sure you can build this without dropping under 3000 req/s.

Of cause the application is not state as the art if you want testability, loose coupling, modularisation. You can have all this for any part of the application that is not worldwide accessible or not vital. For example you can write the admin backend storing the data in Redis with Symfony, Rails or what aver framework or language you like. So you can use fancy development tools where it is possible and a minimalistic approach where it is necessary.

With a application being able to send out thousands of requests per second on s single machine, it is possible to use common DDOS protection like:

/sbin/iptables -A INPUT -p tcp --syn --dport 80 -m connlimit \

--connlimit-above 200 -j REJECT --reject-with tcp-reset

A connection limit of 200 should not bother most users, even if they are behind a large proxy. If so I can use DNS round robin to splitting the load between them so that a single IP can do 2000 concurrent requests on 10 servers. Many IPs can still do 10*3000= 30000 requests per second on 10 servers.

Through a local WLAN connection with 54 Mbit/s you even cannot reach this limit above using the ab benchmark, as the network speed is the limitation.

For a common CMS website you would need to lower the connection limit to something around 10 or 20 requests if you want to lower the load. This will cause trouble for normal visitors.

You can look at the code I used:
Pager: 4 3 2 1