Our application worked great when testing with the load of one user. The web server responded in a timely manner so we decide to promote to production. Within a short period of time, the web server begins timing out, rejecting requests, and responding very slow back to the clients. We add a quad-core processor and more memory to the server, but the time outs and rejected requests remain largely unresolved. Why didn’t the expensive hardware alleviate the issue?Our production example failed because the web server couldn’t complete the high volume of requests. A simple exploration of how web servers handle requests will reveal that EACH web request has its own dedicated thread. If ten requests come in, ten threads are started. During the processing of each request, the thread attached to each request enters a blocking state.
The image above is Microsoft’s illustration of the request operation pipeline for ASP.NET. A thread is taken from the thread pool starting with the runtime step and is not released until the handler at the bottom is finished processing. Back to our numbers, given ten requests, ten threads are in a blocking state until completion. No problem! We have a quad core server and each core is more than capable of handling every request on its own. What happens when we have 10,000 requests simultaneously? Is our thread pool large enough to accommodate this load? I doubt it. If we convert to an asynchronous model, we can send the request threads back to the thread pool for other requests while the handlers are performing slow operations.
The image above illustrates how different threads are started when utilizing MVC’s AsyncController. ASP.NET web forms (IHttpAsyncHandler) and MVC (AsyncController) are both equipped to implement asynchronous server calls. The fact that the request thread is released early and placed back into the thread pool facilitates the success of a high volume web server.
It is important to note that not every web call should be asynchronous. In fact, a majority of your standard web calls should remain synchronous. Your asynchronous calls should be focused around the specific calls that are network or I/O reliant or serve as bottlenecks. You should continue to utilize the synchronous model when dealing with the short-running, simple, or CPU-reliant calls. Introducing the asynchronous model to CPU-reliant calls may result in slower performance due to the added overhead.
In closing, it is very easy to see how asynchronous server web calls can benefit high volume web applications. The release of the initial request thread is crucial to permit the web server to answer the next caller in the queue. Explore and experiment! A wise developer once said, “Don’t stink, think async!”
Image Credit: Diana Robinson
