Connecting timeouts in the context of the TCP protocol are easiest to emulate. Just point the connecting target to a remote address that always drops any incoming (SYN) packets via a firewall rule or something similar. We provide such a "black-hole service" at the port 12345 of the agentzh.org host. You can make use of it if your test running environment allows public network access. Consider the following test case.

We have to configure the resolver directive here because we need to resolve the domain name agentzh.org at request time (in Lua). We check the NGINX error log via the error_log section for the error string returned by the cosocket object’s connect() method.

It is important to use a relatively small timeout threshold in the test cases so that we do not have to wait for too long to complete the test run. Tests are meant to be run very often. The more frequently we run the tests, the more value we may gain from automating the tests.

It is worth mentioning that the test scaffold’s HTTP client does have a timeout threshold as well, which is 3 seconds by default. If your test request takes more than 3 seconds, you get an error message in the test report:

This message is what we would get if we commented out the settimeout call and relies on the default 60 second timeout threshold in cosockets.

We could change this default timeout threshold used by the test scaffold client by setting a value to the timeout data section, as in

Now we have 10 seconds of timeout protection instead of 3.

Emulating reading timeouts is also easy. Just try reading from a wire where the other end never writes anything but still keeps the connection alive. Consider the following example:

Here we use the main_config data section to define a TCP server of our own, listening at the port of 5678 on the local host. This is a mocked-up server that can establish new TCP connections but never write out anything and just sleep for 10 second before closing the session. Note that we are using the ngx_stream_lua module in the stream {} configuration block. In our location = /t, which is the main target of this test case, connects to our mock server and tries to read a line from the wire. Apparently the 100ms timeout threshold on the client side is reached first and we can successfully exercise the error handling code for the reading timeout error.

Triggering sending timeouts is much harder than connecting and reading timeouts. This is due to the asynchronous nature of writing.

For performance reasons, there exists at least two layers of buffers for writes:

To make the situation even worse, there also at least exists a system-level receive buffer layer on the other end of the connection.

To make a send timeout error happen, the most naive way is to fill out all these buffers along the data sending chain while ensuring that the other end never actually reads anything on the application level. Thus, buffering makes a sending timeout particularly hard to reproduce and emulate in a typical testing and development environment with a small amount of (test) payload.

Fortunately there is a userland trick that can intercept the libc wrappers for the actual system calls for socket I/O and do funny things that could otherwise be very difficult to achieve. Our mockeagain library implements such a trick and supports emulating timeout errors at user-specified precise positions in the output data stream.

The following example triggers a sending timeout right after sending out the "hello, world" string as the response body.

Note the send_timeout directive that is used to configure the sending timeout for NGINX downstream writing operations. Here we use a small threshold, 100ms, to ensure our test case runs fast and never hits the default 3 seconds timeout threshold of the test scaffold client. The postpone_output 1 directive effectively turns off the "postpone output buffer" of NGINX, which may hold our output data before even reaching the libc system call wrappers. Finally, the ngx.flush() call in Lua ensures that no buffers along the NGINX output filter chain holds our data without sending downward.

Before running this test case, we have to set the following system environment variables (in the bash syntax):

Let’s go through them one by one:

Now you should get the test block above passed!

Ideally, we could set these environments directly inside the test file because this test case will never pass without these environments anyway. We could add the following Perl code snippet to the very beginning of the test file prologue (yes, even before the use statement):

The BEGIN {} block is required here because it runs before Perl loads any modules, especially Test::Nginx::Socket, in which we want these environments to take effect.

It is a bad idea, however, to hard-code the path of the mockeagain.so file in the test file itself since different test runners might put mockeagain in different places in the file system. Better let the test runner configure the LD_LIBRARY_PATH environment containing the actual library path from outside.

The raw_request data section can be used to specify whatever data for the test request. It is often used with the eval section filter so that we can easily encode special characters like \r. Let’s look at the following example.

So we easily construct a malformed request that does not have a Host header, which results in a 400 response from the NGINX server, as expected.

The request data section we have been using so far, on the other hand, always ensures that a well-formed HTTP request is sent to the test server.

Client aborts are a very intriguing phenomenon in the web world. Sometimes we want the server to continue processing even after the client aborts the connection; on other occasions we just want to abort the whole request handler immediately in such cases. Either way, we need robust way to emulate client aborts in our unit test cases.

We have already discussed the timeout data section that can be used to adjust the default timeout protection threshold used by the test scaffold client. We could also use it to abort the connection prematurely. A small timeout threshold is often desired for this purpose. To suppress the test scaffold from printing out an error on client timeout, we can specify the abort data section to signal the test scaffold. Let’s put these together in a simple test case.

In this example, we make the test scaffold client abort the connection after 0.2 seconds via the timeout section. Also we prevent the test scaffold from printing out the client timeout error by specifying the abort section. Finally, in the Lua application code, we checks for client abort events by turning on the lua_check_client_abort directive and aborts the server processing by calling ngx.exit(444) in our Lua callback function registered by the ngx.on_abort API.

Unlike most well-formed HTTP clients in the market, the HTTP client used by Test::Nginx::Socket never actively closes the connection unless a timeout error happens (exceeding the timeout threshold as specified by the --- timeout section). This can ensure the NGINX server always actually closes the connection when the request specifies the "Connection: close" request header.

When the server does not close the connection, there is a "connection leak" bug on the server side. For example, NGINX uses reference counting (in r→main→count) in its HTTP subsystem to determine whether a request can be closed and freed. When there is an error in this reference counting, NGINX may never close the request, leading to resource leaks. In such cases, the corresponding test cases always fail with a client-side timeout error, for instance,

Obviously Test::Nginx::Socket is a malicious HTTP client by default in this aspect. This is also why our test scaffold avoids using a well-formed HTTP client library itself. Most test suite is focusing on extreme and erroneous cases anyway and well-formed HTTP clients help hiding problems instead of exposing them.