Benchmarks, pt. 1

Findings:

stat RAILS PHOENIX improvement
avg latency 975.57ms 244.41ms 3.99x
avg req/s 1.36r/s 3.827r/s 2.81x
max latency 3.97s 1.05s 3.78x
max req/s 5.66s 6.33s 1.12x
total reqs* 2401.66 6444.66 2.68x

The stats above were computed based on 3 runs.
(*) means that these were averaged (took each number per run and
averaged them – for max requests per second this isn’t very meaningful).

They were not run concurrently (phoenix and rails at the same time)
and I gave like 10 seconds of breathing time between each run.

Phoenix had 4x lower latency, ~3x more requests per second on average, the average maximum latency was ~4x lower, approximately the same in terms of maximum averaged requests per second (this is kind of a meaningless number), and in terms of average total number of requests attained over the course of the runs was 2.68x greater.

Phoenix – killin’ the game.

Below are the numbers. If you find any error in interpretation/understanding/arithmetic,
please reach out to me.

I’d greatly appreciate any input! 

Rails

➜  ~  wrk -t25 -c25 -d60s "https://trophus.herokuapp.com"
Running 1m test @ https://trophus.herokuapp.com
  25 threads and 25 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   386.79ms  180.51ms 848.84ms   76.00%
    Req/Sec     1.56      0.51     2.00     56.00%
  2262 requests in 1.00m, 29.99MB read
  Socket errors: connect 0, read 0, write 0, timeout 183
Requests/sec:     37.65
Transfer/sec:    511.14KB
➜  ~  wrk -t25 -c25 -d60s "https://trophus.herokuapp.com"
Running 1m test @ https://trophus.herokuapp.com
  25 threads and 25 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     1.23s   792.82ms   3.13s    42.59%
    Req/Sec     1.24      1.37     9.00     78.92%
  2360 requests in 1.00m, 31.36MB read
  Socket errors: connect 0, read 0, write 0, timeout 8
Requests/sec:     39.28
Transfer/sec:    534.39KB
➜  ~  wrk -t25 -c25 -d60s "https://trophus.herokuapp.com"
Running 1m test @ https://trophus.herokuapp.com
  25 threads and 25 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     1.31s     1.50s    7.92s    86.54%
    Req/Sec     1.28      1.13     6.00     59.77%
  2583 requests in 1.00m, 34.27MB read
  Socket errors: connect 0, read 0, write 0, timeout 20
Requests/sec:     42.99
Transfer/sec:    584.08KB

Phoenix:

➜  ~  wrk -t25 -c25 -d60s "https://www.trophus.com"
Running 1m test @ https://www.trophus.com
  25 threads and 25 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   291.59ms  176.61ms   2.17s    88.73%
    Req/Sec     3.22      1.12     5.00     68.33%
  5601 requests in 1.00m, 30.52MB read
Requests/sec:     93.24
Transfer/sec:    520.27KB
➜  ~  wrk -t25 -c25 -d60s "https://www.trophus.com"
Running 1m test @ https://www.trophus.com
  25 threads and 25 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   225.57ms   54.49ms 526.81ms   86.79%
    Req/Sec     4.06      0.80     6.00     59.06%
  6760 requests in 1.00m, 36.84MB read
Requests/sec:    112.55
Transfer/sec:    628.01KB
➜  ~  wrk -t25 -c25 -d60s "https://www.trophus.com"
Running 1m test @ https://www.trophus.com
  25 threads and 25 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   216.06ms   32.34ms 454.83ms   74.22%
    Req/Sec     4.20      1.18     8.00     64.00%
  6973 requests in 1.00m, 38.00MB read
Requests/sec:    116.06
Transfer/sec:    647.62KB