corenominal

Full stack web developer, interested in all the things, but especially the web, code, design, Linux, OS X, PHP, WordPress, JavaScript & robots.

Benchmarking WordPress on Xenial Xerus with PHP7 and HTTP/2

Yesterday I set-up a test environment for WordPress in an Ubuntu Server 16.04 virtual machine. Today, I mirrored that environment in another virtual machine, but with Ubuntu Server 14.04, and proceeded to run some benchmarks.

I ran all the benchmarks from a third virtual machine, running Ubuntu Desktop 14.04, and I created entries in that machine’s hosts file to switch where the requests would be routed.

For the first benchmark, I used Apache HTTP server benchmarking tool. The benchmark made 1000 requests to each server, spread across 100 clients. The requests were all made over HTTP/1.

Ubuntu 14.04 Trusty Tahr ab test

corenominal@loki-ubu-vm:~$ ab -c 100 -n 1000 https://corenominal.org/
This is ApacheBench, Version 2.3 <$Revision: 1528965 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking corenominal.org (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Completed 1000 requests
Finished 1000 requests


Server Software:        nginx/1.4.6
Server Hostname:        corenominal.org
Server Port:            443
SSL/TLS Protocol:       TLSv1.2,ECDHE-RSA-AES256-GCM-SHA384,2048,256

Document Path:          /
Document Length:        27442 bytes

Concurrency Level:      100
Time taken for tests:   14.167 seconds
Complete requests:      1000
Failed requests:        0
Total transferred:      27692000 bytes
HTML transferred:       27442000 bytes
Requests per second:    70.58 [#/sec] (mean)
Time per request:       1416.739 [ms] (mean)
Time per request:       14.167 [ms] (mean, across all concurrent requests)
Transfer rate:          1908.82 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        3   18  41.0      5     192
Processing:   291 1343 180.2   1387    1604
Waiting:      276 1333 183.8   1378    1604
Total:        409 1361 163.2   1393    1788

Percentage of the requests served within a certain time (ms)
  50%   1393
  66%   1412
  75%   1425
  80%   1436
  90%   1464
  95%   1485
  98%   1544
  99%   1684
 100%   1788 (longest request)

Ubuntu 16.04 Xenial Xerus ab test

corenominal@loki-ubu-vm:~$ ab -c 100 -n 1000 https://corenominal.org/
This is ApacheBench, Version 2.3 <$Revision: 1528965 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking corenominal.org (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Completed 1000 requests
Finished 1000 requests


Server Software:        nginx/1.9.12
Server Hostname:        corenominal.org
Server Port:            443
SSL/TLS Protocol:       TLSv1.2,ECDHE-RSA-AES256-GCM-SHA384,2048,256

Document Path:          /
Document Length:        27442 bytes

Concurrency Level:      100
Time taken for tests:   8.951 seconds
Complete requests:      1000
Failed requests:        0
Total transferred:      27647000 bytes
HTML transferred:       27442000 bytes
Requests per second:    111.72 [#/sec] (mean)
Time per request:       895.119 [ms] (mean)
Time per request:       8.951 [ms] (mean, across all concurrent requests)
Transfer rate:          3016.25 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        3   16  36.1      4     135
Processing:    43  841 141.5    872     991
Waiting:       34  823 140.2    855     964
Total:        158  857 116.8    878    1085

Percentage of the requests served within a certain time (ms)
  50%    878
  66%    887
  75%    894
  80%    901
  90%    932
  95%    948
  98%    965
  99%   1001
 100%   1085 (longest request)

No caching was enabled for the above and all requests were handled via PHP. As you can see, Xenial served the requests in 8.951 seconds, compared to Trusty’s 14.167 seconds. Xenial was definitely quicker.

For the second benchmark, I switched to using h2load, a HTTP/2 benchmarking tool. Again, I set it up to run 1000 requests with 100 concurrent clients. The results were interesting.

Ubuntu 14.04 Trusty Tahr h2load

corenominal@loki-ubu-vm:~$ h2load -n1000 -c100 https://corenominal.org
starting benchmark...
spawning thread #0: 100 total client(s). 1000 total requests
TLS Protocol: TLSv1.2
Cipher: ECDHE-RSA-AES256-GCM-SHA384
Application protocol: http/1.1
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 11.93s, 83.83 req/s, 2.22MB/s
requests: 1000 total, 1000 started, 1000 done, 1000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 1000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 26.47MB (27761000) total, 230.47KB (236000) headers (space savings 0.00%), 26.17MB (27442000) data
                     min         max         mean         sd        +/- sd
time for request:    46.58ms      11.70s       5.99s       3.33s    58.80%
time for connect:    32.07ms    226.03ms    177.22ms     60.05ms    82.00%
time to 1st byte:   136.05ms       1.68s    923.10ms    432.88ms    59.00%
req/s           :       0.84        1.03        0.88        0.04    83.00%

Ubuntu 16.04 Xenial Xerus h2load

corenominal@loki-ubu-vm:~$ h2load -n1000 -c100 https://corenominal.org
starting benchmark...
spawning thread #0: 100 total client(s). 1000 total requests
TLS Protocol: TLSv1.2
Cipher: ECDHE-RSA-AES256-GCM-SHA384
Application protocol: h2
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 1.42s, 704.94 req/s, 2.65MB/s
requests: 1000 total, 1000 started, 1000 done, 134 succeeded, 866 failed, 0 errored, 0 timeout
status codes: 134 2xx, 0 3xx, 0 4xx, 866 5xx
traffic: 3.76MB (3943504) total, 59.99KB (61429) headers (space savings 42.79%), 3.68MB (3854315) data
                     min         max         mean         sd        +/- sd
time for request:    36.14ms       1.29s    157.89ms    256.57ms    89.70%
time for connect:   113.85ms    163.81ms    128.71ms     10.05ms    61.00%
time to 1st byte:   153.04ms       1.27s    273.07ms    224.19ms    91.00%
req/s           :       7.07       64.62       44.04       15.69    75.00%

Trusty completed the requests in 11.93 seconds (I’m not sure why this result should be quicker than the previous ab test), but as expected, served the requests with HTTP/1. Xenial completed the requests in 1.42 seconds with HTTP/2, but experienced a bucket load of failed requests.

I don’t know what happened there, I checked the log files and I noticed a lot of “502 bad gateway” errors, so I figure I’ve still got some work to do with configuring nginx and PHP 7. I did have a play around with some settings, but failed to get it to work. In fact, the only way I managed to get it to work, without any failed requests, was by enabling WP Super Cache and bypassing PHP altogether.

Ubuntu 16.04 Xenial Xerus h2load & caching

corenominal@loki-ubu-vm:~$ h2load -n1000 -c100 https://corenominal.org/
starting benchmark...
spawning thread #0: 100 total client(s). 1000 total requests
TLS Protocol: TLSv1.2
Cipher: ECDHE-RSA-AES256-GCM-SHA384
Application protocol: h2
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 553.59ms, 1806.38 req/s, 47.85MB/s
requests: 1000 total, 1000 started, 1000 done, 1000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 1000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 26.49MB (27773900) total, 107.42KB (110000) headers (space savings 38.89%), 26.33MB (27614000) data
                     min         max         mean         sd        +/- sd
time for request:     2.50ms    391.62ms    233.88ms    107.34ms    61.70%
time for connect:    82.37ms    186.38ms    144.62ms     17.21ms    89.00%
time to 1st byte:    84.63ms    308.66ms    225.79ms     51.67ms    68.00%
req/s           :      18.15      102.75       24.93       16.67    95.00%

The above results are quite impressive, but I think I need to do a bit more research on this before I consider enabling HTTP/2 on any production servers. If anyone can shed any light as to what might be occurring with the Xenial h2load benchmark, I’d appreciate it.

Leave a comment

Your email address will not be published. Required fields are marked *