pixelserv-tls : HTTP/1.1 Persistent Connections

This article discusses features and performance of pixelserv-tls, version KL. As of this writing a pre-release test version is used for analysis.

The original pixelserv handles each request by fork()'ing a new process. It is an expensive operation on both CPU time and memory footprint. With pixelserv-tls version Ke, threads were introduced in place of processes. Threads are sometimes aka lightweight processes. All threads share the same address space. This reduces both memory footprint and CPU usage.

Regardless a process or a thread, pixelserv-tls kills the thread/process after handling one request, and shuts down the established connection to a client. This is a waste of work if subsequently the client has more requests to the same adverts host. Also this behaviour does not comply to the HTTP/1.1 specification for persistent connection until explicitly teared down.

pixelserv-tls version KL introduces support for HTTP/1.1 persistent connections. A connection from one client will be, as prior version of pixelserv-tls, handled by one thread. The new feature will keep this thread alive and the connection active as long as a client wants. This speeds up both HTTP and HTTPS requests. Establishment of a HTTPS connection requires exchanging eight messages between client and server before the actual request can be encrypted and sent. Connection reuse trims lot of overhead that translates into faster response time.

New counters that monitor thread performance

Four new counters are introduced in version KL, specifically to measure thread performance. They are:

  • kcc - current number of active threads serving clients
  • kmx - maximum number of active threads ever recorded
  • kvg - average number of requests serviced in a thread's lifespan
  • krq - maximum number of requests serviced by one thread ever recorded

These counters provide a glimpse into how efficient the threads perform and are generally sufficient for everyday use.

With DEBUG log ('-l 5'), more output can be captured for deeper analysis though the analytical process is not straight forward. For the first time and in determining a few default values, I went through the process and presented the result below.

Further analysis of thread performance

A test version, KL-test5, is used in the study. I collected about 14,000 service thread creations over a few days. I wanted to know the distribution of number of requests per thread, and the lifespan in seconds of each thread.

Figure 1

It's found that most threads (77%) only handle one request in its lifetime. Summation of 1, 2, 3 or 4 requests per thread, the percentage jumps to around 95% as shown in Figure 1. Up to 20 requests per thread, it covers almost 100% of the service threads.

The result is not surprising. A client usually browses with one or two tabs at the same time. Unless a website is abusively adverts intensive, a connection of a given adverts host tends to be unlikely reused. With more tabs open on the client, chance of reuse increases and a modern browser will keep it alive and reuse an existing connection.

Figure 2

With the same data of 14,000 thread creations, Figure 2 shows the distribution of lifespan in seconds of these threads. 30% of threads last less than one second. Threads lasting 19s or less account for 37%. Quite a lot threads last about 20s (about 30%). So threads lasting 20s or less jumps to 68%. Then threads lasting 30s or more start its constant rate of ascent until about 60s where almost 100% of threads are accounted.

I couldn't explain the 20s, 30s or 60s phenomenon. I could guess that the distribution highly depends on traffic as well as the browser used on client side. For example, Chrome tends to keep a connection alive for longer (about 5mins?) than Safari. This translates into higher likelihood of reuse and higher number of requests per thread.

Further analysis of HTTP POST processing

pixelserv-tls version KL also has the new feature to capture HTTP POST content. It's of interest to see how this performs.

The 14,000 thread creations handled lots of requests. Among these requests are a total of 3296 HTTP POST requests. For which, I list the statistics:

  • 1218 (37.0%) requests carrying complete POST content in the body of the request
  • 2078 (63.0%) requests requiring one or more subsequent messages to deliver its full content
  • 1287 (39.0%) requests delivering content within 1s with one or more messages
  • 17 (0.52%) requests delivering content within 2s with one or more messages
  • 738 (22.4%) requests "never delivering" any content before 10s timeout
  • 36 (1.09%) requests of unknown status

The 1.09% of unknown status are due to the analytical method I used. The number is small and let's ignore for now. The 22.4% timeout are confirmed to be a bug in KL-test5 and will be fixed in KL-test7 or the final release.

As majority (76.5%) of POST content starts delivering within 1s, the default timeout of 10s used in KL-test5 is excessive. We shall adopt 5s in the final release, even that is more than enough. Note that this timeout for POST handling is internal and will not be user adjustable in version KL.

Congratulations and thank you if you read up to here!

comments powered by Disqus