Benchmarking WordPress on Xenial Xerus with PHP7 and HTTP/2
Yesterday I set-up a test environment for WordPress in an Ubuntu Server 16.04 virtual machine. Today, I mirrored that environment in another virtual machine, but with Ubuntu Server 14.04, and proceeded to run some benchmarks.
I ran all the benchmarks from a third virtual machine, running Ubuntu Desktop 14.04, and I created entries in that machine’s hosts file to switch where the requests would be routed.
For the first benchmark, I used Apache HTTP server benchmarking tool. The benchmark made 1000 requests to each server, spread across 100 clients. The requests were all made over HTTP/1.
Ubuntu 14.04 Trusty Tahr ab test
corenominal@loki-ubu-vm:~$ ab -c 100 -n 1000 https://corenominal.org/
This is ApacheBench, Version 2.3 <$Revision: 1528965 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking corenominal.org (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Completed 1000 requests
Finished 1000 requests
Server Software: nginx/1.4.6
Server Hostname: corenominal.org
Server Port: 443
SSL/TLS Protocol: TLSv1.2,ECDHE-RSA-AES256-GCM-SHA384,2048,256
Document Path: /
Document Length: 27442 bytes
Concurrency Level: 100
Time taken for tests: 14.167 seconds
Complete requests: 1000
Failed requests: 0
Total transferred: 27692000 bytes
HTML transferred: 27442000 bytes
Requests per second: 70.58 [#/sec] (mean)
Time per request: 1416.739 [ms] (mean)
Time per request: 14.167 [ms] (mean, across all concurrent requests)
Transfer rate: 1908.82 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 3 18 41.0 5 192
Processing: 291 1343 180.2 1387 1604
Waiting: 276 1333 183.8 1378 1604
Total: 409 1361 163.2 1393 1788
Percentage of the requests served within a certain time (ms)
50% 1393
66% 1412
75% 1425
80% 1436
90% 1464
95% 1485
98% 1544
99% 1684
100% 1788 (longest request)Ubuntu 16.04 Xenial Xerus ab test
corenominal@loki-ubu-vm:~$ ab -c 100 -n 1000 https://corenominal.org/
This is ApacheBench, Version 2.3 <$Revision: 1528965 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking corenominal.org (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Completed 1000 requests
Finished 1000 requests
Server Software: nginx/1.9.12
Server Hostname: corenominal.org
Server Port: 443
SSL/TLS Protocol: TLSv1.2,ECDHE-RSA-AES256-GCM-SHA384,2048,256
Document Path: /
Document Length: 27442 bytes
Concurrency Level: 100
Time taken for tests: 8.951 seconds
Complete requests: 1000
Failed requests: 0
Total transferred: 27647000 bytes
HTML transferred: 27442000 bytes
Requests per second: 111.72 [#/sec] (mean)
Time per request: 895.119 [ms] (mean)
Time per request: 8.951 [ms] (mean, across all concurrent requests)
Transfer rate: 3016.25 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 3 16 36.1 4 135
Processing: 43 841 141.5 872 991
Waiting: 34 823 140.2 855 964
Total: 158 857 116.8 878 1085
Percentage of the requests served within a certain time (ms)
50% 878
66% 887
75% 894
80% 901
90% 932
95% 948
98% 965
99% 1001
100% 1085 (longest request)No caching was enabled for the above and all requests were handled via PHP. As you can see, Xenial served the requests in 8.951 seconds, compared to Trusty’s 14.167 seconds. Xenial was definitely quicker.
For the second benchmark, I switched to using h2load, a HTTP/2 benchmarking tool. Again, I set it up to run 1000 requests with 100 concurrent clients. The results were interesting.
Ubuntu 14.04 Trusty Tahr h2load
corenominal@loki-ubu-vm:~$ h2load -n1000 -c100 https://corenominal.org
starting benchmark...
spawning thread #0: 100 total client(s). 1000 total requests
TLS Protocol: TLSv1.2
Cipher: ECDHE-RSA-AES256-GCM-SHA384
Application protocol: http/1.1
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done
finished in 11.93s, 83.83 req/s, 2.22MB/s
requests: 1000 total, 1000 started, 1000 done, 1000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 1000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 26.47MB (27761000) total, 230.47KB (236000) headers (space savings 0.00%), 26.17MB (27442000) data
min max mean sd +/- sd
time for request: 46.58ms 11.70s 5.99s 3.33s 58.80%
time for connect: 32.07ms 226.03ms 177.22ms 60.05ms 82.00%
time to 1st byte: 136.05ms 1.68s 923.10ms 432.88ms 59.00%
req/s : 0.84 1.03 0.88 0.04 83.00%Ubuntu 16.04 Xenial Xerus h2load
corenominal@loki-ubu-vm:~$ h2load -n1000 -c100 https://corenominal.org
starting benchmark...
spawning thread #0: 100 total client(s). 1000 total requests
TLS Protocol: TLSv1.2
Cipher: ECDHE-RSA-AES256-GCM-SHA384
Application protocol: h2
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done
finished in 1.42s, 704.94 req/s, 2.65MB/s
requests: 1000 total, 1000 started, 1000 done, 134 succeeded, 866 failed, 0 errored, 0 timeout
status codes: 134 2xx, 0 3xx, 0 4xx, 866 5xx
traffic: 3.76MB (3943504) total, 59.99KB (61429) headers (space savings 42.79%), 3.68MB (3854315) data
min max mean sd +/- sd
time for request: 36.14ms 1.29s 157.89ms 256.57ms 89.70%
time for connect: 113.85ms 163.81ms 128.71ms 10.05ms 61.00%
time to 1st byte: 153.04ms 1.27s 273.07ms 224.19ms 91.00%
req/s : 7.07 64.62 44.04 15.69 75.00%Trusty completed the requests in 11.93 seconds (I’m not sure why this result should be quicker than the previous ab test), but as expected, served the requests with HTTP/1. Xenial completed the requests in 1.42 seconds with HTTP/2, but experienced a bucket load of failed requests.
I don’t know what happened there, I checked the log files and I noticed a lot of “502 bad gateway” errors, so I figure I’ve still got some work to do with configuring nginx and PHP 7. I did have a play around with some settings, but failed to get it to work. In fact, the only way I managed to get it to work, without any failed requests, was by enabling WP Super Cache and bypassing PHP altogether.
Ubuntu 16.04 Xenial Xerus h2load & caching
corenominal@loki-ubu-vm:~$ h2load -n1000 -c100 https://corenominal.org/
starting benchmark...
spawning thread #0: 100 total client(s). 1000 total requests
TLS Protocol: TLSv1.2
Cipher: ECDHE-RSA-AES256-GCM-SHA384
Application protocol: h2
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done
finished in 553.59ms, 1806.38 req/s, 47.85MB/s
requests: 1000 total, 1000 started, 1000 done, 1000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 1000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 26.49MB (27773900) total, 107.42KB (110000) headers (space savings 38.89%), 26.33MB (27614000) data
min max mean sd +/- sd
time for request: 2.50ms 391.62ms 233.88ms 107.34ms 61.70%
time for connect: 82.37ms 186.38ms 144.62ms 17.21ms 89.00%
time to 1st byte: 84.63ms 308.66ms 225.79ms 51.67ms 68.00%
req/s : 18.15 102.75 24.93 16.67 95.00%The above results are quite impressive, but I think I need to do a bit more research on this before I consider enabling HTTP/2 on any production servers. If anyone can shed any light as to what might be occurring with the Xenial h2load benchmark, I’d appreciate it.