ASP.Net Threading
Inside the ASP.Net Worker Process there are two thread pools. The worker thread pool handles all incoming requests and the I/O Thread pool handles the I/O (accessing the file system, web services and databases, etc.). Each App Domain has its own thread pool and the number of operations that can be queued to the thread pool is limited only by available memory; however, the thread pool limits the number of threads that can be active in the process simultaneously.
Source: Microsoft Tech Ed 2007 DVD: Web 405 "Building Highly Scalable ASP.NET Web Sites by Exploiting Asynchronous Programming Models" by Jeff Prosise.
So how many threads are there in these thread pools? I had always assumed that the number of threads varies from machine to machine – that ASP.NET and IIS were carefully and cleverly balancing the number of available threads against available hardware, but that is simply not the case. The fact is that ASP.Net installs with a fixed, default number of threads to play with: the 1.x Framework defaults to just 20 worker threads (per CPU) and 20 I/O threads (per CPU). The 2.0 Framework defaults to 100 threads in each pool, per CPU. Now this can be increased by adding some new settings to the machine.config file. The default worker thread limit was raised to 250 per CPU and 1000 I/O threads per CPU with the .NET 2.0 SP1 and later Frameworks. 32 bit windows can handle about 1400 concurrent threads, 64 bit windows can handle more, though I don’t have the figures.
In a normal (synchronous) Page Request a single worker thread handles the entire request from the moment it is received until the completed page is returned to the browser. When the I/O operation begins, a thread is pulled from the I/O thread pool, but the worker thread is idle until that I/O thread returns. So, if your page load event fires off one or more I/O operations, then that main worker thread could be idle for 1 or more seconds and in that time it could have serviced hundreds of additional incoming page requests.
Source: Microsoft Tech Ed 2007 DVD: Web 405 "Building Highly Scalable ASP.NET Web Sites by Exploiting Asynchronous Programming Models" by Jeff Prosise.
So long as the number of concurrent requests does not exceed the number of threads available in the pool, all is well. But when you are building enterprise level applications the thread pool can become depleted under heavy load, and remember by default heavy load is more than just 200 simultaneous requests assuming a dual CPU Server.
When this happens, new requests are entered into the request queue (and the users making the requests watch that little hour glass spin and consider trying another site). ASP.NET will allow the request queue to grow only so big before it starts to reject requests at which point it starts returning Error 503, Service Unavailable.
If you are not aware of this “Glass Ceiling of Scalability”, this is a perplexing error – one that never happened in testing and may not be reproducible in your test environment, as it only happens under extreme load.
Asynchronous Programming models in ASP.NET
To solve this problem ASP.Net provides four asynchronous Programming models. Asyncronous Pages, Asyncronous HttpHandlers, Asyncronous HttpModules and Asyncronous Web Services. The only one that is well documented and reasonably well known is the asynchronous Web Services model. Since there is quite a lot of documentation on that, and since in future web services should be implemented using the Windows Communication Foundation, we shall concentrate only on the other three.
Let’s begin with the first asynchronous programming model, Asynchronous Pages.
Asynchronous Pages
Source: Microsoft Tech Ed 2007 DVD: Web 405 "Building Highly Scalable ASP.NET Web Sites by Exploiting Asynchronous Programming Models" by Jeff Prosise.
To make a page Asynchronous, we insert what we refer to as an “Async Point” into that page’s lifecycle, which you can see in green on the right. We need to write and register with ASP.NET a pair of Begin and End Events. At the appropriate point in the page’s lifecycle, ASP.NET will call our begin method. In the begin method we will launch an asynchronous I/O operation, for example an asynchronous database query, and we will immediately return from the begin method. As soon as we return, ASP.Net will drop the thread that was assigned to that request back into the thread pool where it may service hundreds or even thousands of additional page requests while we wait for our I/O operation to complete.
As you’ll see when we get to the sample code, we return from our begin method an IAsyncResult Interface, through which we can signal ASP.NET when the async operation that we launched has completed. It is when we do that, that ASP.NET reaches back into the thread pool, pulls out a second worker thread and calls our end method, and then allows the processing of that request to resume as normal.
So, from ASP.NET’s standpoint it is just a normal request, but it is processed by 2 different threads; and that will bring up a few issues that we’ll need to discuss in a few moments.
Now, none of this was impossible with the 1.1 framework, but it was a lot of extra work, and you lost some of the features of ASP.NET in the process. The beauty of the 2.0 and later frameworks is that this functionality is built right into the Http pipeline, and so for the most part everything works in the asynchronous page just as it did in the synchronous one.
In order to create an Asynchronous page you need to include the Async=”True” attribute in the page directive of your .aspx file. That directive tells the ASP.NET engine to implement an additional Interface on the derived page class which lets ASP.NET know at runtime that this is an asynchronous page.
What happens if you forget to set that attribute? Well the good news is that the code will still run just fine, but it will run synchronously, meaning that you did all that extra coding for nothing. I should also point out that to make an Asynchronous data call, you also need to add “async=true;” or “Asynchronous Processing=true;” to your connection string – If you forget that and make your data call asynchronously, you will get a SQL Exception.
The second thing we need to do in order to create an asynchronous page is to register Begin and End Events. There are 2 ways to register these events. The first way is to use a new method introduced in ASP.NET 2.0 called AddOnPreRenderCompleteAsync:
using System;
using System.Net;
using System.Web;
using System.Web.UI;
using System.Web.UI.WebControls;
public partial class temp : System.Web.UI.Page
{
private static readonly Uri c_UrlImage1 = new Uri(@"http://www.asyncassassin.com/asyncassassin/image.axd?picture=2008%2f12%2fSlide3.JPG");
private HttpWebRequest request;
void Page_Load(object sender, EventArgs e)
{
request = (HttpWebRequest)WebRequest.Create(c_UrlImage1);
AddOnPreRenderCompleteAsync(
BeginAsyncOperation,
EndAsyncOperation
);
}
IAsyncResult BeginAsyncOperation(object sender, EventArgs e,
AsyncCallback cb, object state)
{
// Begin async operation and return IAsyncResult
return request.BeginGetResponse(cb, state);
}
void EndAsyncOperation(IAsyncResult ar)
{
// Get results of async operation
HttpWebResponse response = (HttpWebResponse)request.EndGetResponse(ar);
Label1.Text = String.Format("Image at {0} is {1:N0} bytes", response.ResponseUri, response.ContentLength);
}
}
The second way is to use RegisterAsyncTask:
using System;
using System.Net;
using System.Web;
using System.Web.UI;
using System.Web.UI.WebControls;
public partial class temp : System.Web.UI.Page
{
private static readonly Uri c_UrlImage1 = new Uri(@"http://www.asyncassassin.com/asyncassassin/image.axd?picture=2008%2f12%2fSlide3.JPG");
private HttpWebRequest request;
void Page_Load(object sender, EventArgs e)
{
request = (HttpWebRequest)WebRequest.Create(c_UrlImage1);
PageAsyncTask task = new PageAsyncTask(
BeginAsyncOperation,
EndAsyncOperation,
TimeoutAsyncOperation,
null
);
RegisterAsyncTask(task);
}
IAsyncResult BeginAsyncOperation(object sender, EventArgs e,
AsyncCallback cb, object state)
{
// Begin async operation and return IAsyncResult
return request.BeginGetResponse(cb, state);
}
void EndAsyncOperation(IAsyncResult ar)
{
// Get results of async operation
HttpWebResponse response = (HttpWebResponse)request.EndGetResponse(ar);
Label1.Text = String.Format("Image at {0} is {1:N0} bytes", response.ResponseUri, response.ContentLength);
}
void TimeoutAsyncOperation(IAsyncResult ar)
{
// Called if async operation times out (@ Page AsyncTimeout)
Label1.Text = "Data temporarily unavailable";
}
}
These methods can be called anywhere in the page’s lifecycle before the PreRender event, and are typically called from the Page_Load event or from the click event of a button during a postback. By the way, you can register these methods from within a UserControl, as long as that control is running on a page that has set the async = true attribute. Again, if it runs on a page without that attribute, the code will still run just fine, but it will run synchronously.
As you can see from just these simple examples, building asynchronous pages is more difficult than building synchronous ones. I’m not going to lie to you. And real world use of these techniques is even more complicated – there is no Business Logic or data layer in the examples above. I don’t want you to leave here believing that you need to make every page asynchronous. You don’t. What I recommend, is doing surgical strikes. Identify that handful of pages in your application that perform the lengthiest I/O and consider converting those into asynchronous pages. The cool thing about this, is that it can improve not only scalability, but also performance, because when you are not holding onto the threads, new requests get into the pipeline faster, they spend less time waiting in that application request queue out there. So, users are happier because pages that they would have had to wait on before – even the ones you have not converted to asynchronous pages, but which might have been delayed while threads were idle, will now load faster. What’s more, as you’ll see in a moment, using RegisterAsyncTask will allow you to perform I/O operations in parallel, which may also improve performance. Having said that, making pages asynchronous is not really about improving performance, it is about improving scalability – making sure that we use the threads in the thread pool as efficiently as we possibly can.
Now I’m sure you are wondering why there are two ways, what the differences are between them, and when you should choose one over the other. Well, there are 3 important differences between AddOnPreRenderCompleteAsync and RegisterAsyncTask.
- As we have seen, RegisterAsyncTask allows you to specify a timeout method. It is important to note that the timeout value you specify in the Page Directive of your .aspx page <%@ Page Async="true" AsyncTimeout="5" ... %> is the timeout for ALL asynchronous tasks the page is performing, not 5 secs per async task - all async tasks must be competed within 5 seconds, so be sure to allow enough time here.
- If you use AddOnPreRenderCompleteAsync, you may find that some things that worked before, no longer work. For example, if you are using User.Identity.Name in your code to get the authenticated username inorder to personalize a page. If this method is called by the first thread it will work fine. But if you call it on the second thread – in your End method or any of the events that fire after the end method User.Identity.Name will be null. This is because as a Request travels through the ASP.NET Http Pipeline, it is accompanied by an object of type HttpContext that basically encapsulates all of the information that ASP.NET knows about that request. When you use AddOnPreRenderCompleteAsync ASP.NET does not take the extra time to map everything in that context object from thread one to thread two. That’s why User.Identity.Name does not work in thread two. In fact, you will often find that HttpContext.Current is null in thread two. However, if you use RegisterAsyncTask, ASP.Net DOES map everything in that context from thread one to thread two. It does take a few microseconds longer to do this, but it will make your life considerably easier.
- The third difference is probably the most important of all. AddOnPreRenderCompleteAsync is a quick and easy way of making a page asynchronous and works well if you have a simple page that needs to perform only 1 asynchronous I/O operation. In real life, a page often needs to perform multiple database queries, or grab data from a webservice and pass it to a database, or something like that. The cool thing about RegisterAsyncTask is that it allows you to quickly and easily queue up multiple Async I/O operations. The last argument is a Boolean value that allows you to specify whether each task can run in parallel. Some times you need to wait for one data call to complete in order to send that data somewhere else, but other times you may need to get data from multiple, unrelated sources and this allows you to fetch them all at the same time, instead of one after the other.
Source: Microsoft Tech Ed 2007 DVD: Web 405 "Building Highly Scalable ASP.NET Web Sites by Exploiting Asynchronous Programming Models" by Jeff Prosise.
N-Tier Applications
OK. So I expect some of you are thinking “But what if I have a data access layer in my application? My pages can’t go directly to the database, they have to go through that data access layer, or they have to go through my BLL, which calls the Data Access Layer.”
Well, ideally, you should simply add the asynchronous methods to your DAL. If you wrote the DAL yourself or have access to its source code, you should add the Begin and End methods to it Adding the asynchronous methods to your DAL is the best, most scalable solution and doesn’t change the example code much at all: Instead of calling begin and end methods defined inside the page class, you simply call MyDAL.Begin… or MyBll.Begin… when you call RegisterAsyncTask or AddOnPreRenderAsync.
Unfortunately, neither Llblgen nor the Enterprise library (nor LINQ for that matter) supports asynchronous data calls natively. However, I believe that you can modify the generated code in llblgen to enable asynchronous data calls. You could also crack open the source code of the Enterprise library and add the asynchronous methods yourself, but before you try check to see if it has already been done.
Asynchronous HTTP Handlers
The 2nd Asynchronous Programming model in ASP.NET is for HttpHandlers and has been around since .Net 1.x, but was not documented any better in version 2 than it was in version 1. Http Handlers are one of the two fundamental building blocks of ASP.NET, an http handler is an object that is built to handle http requests and convert them into http responses. For the most part, each handler corresponds to a file type. For example, there is a built in handler in ASP.NET that handles .aspx files. It is that handler that knows how to instantiate a control tree and send that tree to a rendering engine. The ASMX Handler knows how to decode SOAP and allows us to build web services.
Basically an HTTP Handler is just a class that implements the IHttpHandler interface, which consists of an IsResuable Boolean function and a ProcessRequest method which is the heart of an httphandler as its job is to turn a request into a response. The ProcessRequest method is passed an HttpContext Object containing all the data asp.net has collected about the request, as well as exposing the Session, Server, Request and Response objects that you are used to working with in page requests.
using System.Web;
public class HelloHandler : IHttpHandler
{
public void ProcessRequest(HttpContext context)
{
string name = context.Request["Name"];
context.Response.Write("Hello, " + name);
}
public bool IsReusable
{
get { return true; }
}
}
There are 2 ways to build them. One way is to add the class to your project and register is in the web.config. If you want to register it for any file extension not currently handled by asp.net, you would need to add that extension to IIS. The easier way, is to deploy them as .ASHX files. The ASHX extension has already been registered in IIS, it is auto compiled, no changes are required in the web.config and performance is the same. Ok. So you know what they are and how to build one, when is an appropriate time to use one?
Handlers are commonly used to generate custom XML and RSS feeds, to unzip and render files stored as BLOB fields in the database including image files or logos, HTTP Handlers can also be used as the target of AJAX calls.
A common mistake that programmers new to .net, especially those like myself who came from classic ASP or PHP, is to use the Page_Load event of a page to create a new http response. For example, before I learned about httphandlers, I would use the page load event to create an xml document or a dynamic PDF file and output it to the response stream with a response.End() to prevent the page continuing after I output my file. The problem with that approach is that you are executing a ton of code in asp.net that doesn’t need to execute. When ASP.NET sees that request come in, it thinks it is going to need to build and render a control tree. By pointing the link at the handler instead, you will gain a 10-20% performance increase every time that request is fetched, just because of the overhead you have reduced. Put simply, Http Handlers minimize the amount of code that executes in ASP.NET.
To implement an Asynchronous handler you use the interface IHttpAsyncHandler, which adds BeginProcessRequest and EndProcessRequest methods. The threading works the same way as with an async page. After the begin method is called, the thread returns to the thread pool and handles other incoming requests until the I/O thread completes its work, at which point it grabs a new thread from the thread pool and completes the request.
Page.RegisterAsyncTask cannot be used here, so if you need to run multiple async tasks you will need to implement your own IAsyncResult Interface and pass in your own callbacks to prevent the EndProcessRequest method being called before you have completed all your async operations.
Asynchronous HTTP Modules
HTTP Modules are another fundamental building block of ASP.NET. They don’t handle requests, instead they sit in the HTTP Pipeline where they have the power to review every request coming in and every response going out. Not only can they view them, but they can modify them as well. Many of the features of ASP.NET are implemented using httpmodules: authentication, Session State and Caching for example, and by creating your own HTTP Modules you can extend ASP.NET in a lot of interesting ways. You could use an HTTP Module for example to add google analytics code to all pages, or a custom footer. Logging is another common use of HTTP Modules.
E-Commerce web sites can take advantage of HTTP Modules by overriding the default behavior of the Session Cookie. By default, ASP.NET Session Cookies are only temporary, so if you use them to store shopping cart information, after 20 minutes of inactivity, or a browser shut down they are gone. You may have noticed that Amazon.com retains shopping cart information much longer: You could shut down your laptop, fly to Japan and when you restart and return to Amazon your items will still be there. If you wanted to do this in ASP.NET you could waste a lot of time writing your own Session State Cookie Class, or you could write about 10 lines of code in the form of an HTTP Module that would intercept the cookie created by the Session Object before it gets to the browser, and modify it to make it a persistent cookie. So, there are lots and lots of practical uses for HTTP Modules.
An Http Module is nothing more than a class that implements the IHttpModule Interface, which involves an Init method for registering any and all events that you are interested in intercepting, and a dispose method for cleaning up any resources you may have used.
using System;
using System.Web;
public class BigBrotherModule : IHttpModule
{
public void Init(HttpApplication application)
{
application.EndRequest +=
new EventHandler(OnEndRequest);
}
void OnEndRequest(Object sender, EventArgs e)
{
HttpApplication application = (HttpApplication)sender;
application.Context.Response.Write
("Bill Gates is watching you");
}
public void Dispose() { }
}
The events you can intercept in an HTTP Module:
Source: Microsoft Tech Ed 2007 DVD: Web 405 "Building Highly Scalable ASP.NET Web Sites by Exploiting Asynchronous Programming Models" by Jeff Prosise.
Notice the HTTP Handler at the end there that converts the request into a response. These events will always fire in this order, in every request. The Authenticate Request event is the one fired by ASP.NET when a requested page requires authentication. It checks to see if you have an authentication cookie and if you do not, redirects the request to the login page. In the simple example, I was using that End Request event, which is the last one before the response is sent to the browser.
So, that is what HTTP Modules are for, and how they work. Why do we need an Asynchronous version? Well if you really want to see how scalable your application is, add an HTTP Module that makes a synchronous call to a webservice or a database. Since the event you register will be fired on every request, you will tie up an additional thread from the asp.net thread pool on every single request that is just waiting for these I/O processes to complete. So, if you write a synchronous HTTP Module that inserts a record into a database for every single request, and that insert takes 1 second, EVERY single request handled by your application will be delayed by 1 second. So if you need to do any type of I/O from within an HTTP Module, I recommend you make the calls asynchronously and if you are retrieving data, cache it!
To Register Async Event Handlers in an http module - In the Init Method, simply register your begin and end methods using AddOnPreRequestHandlerExecuteAsync:
using System.Web;
public void Init (HttpApplication application)
{
AddOnPreRequestHandlerExecuteAsync (
new BeginEventHandler (BeginPreRequestHandlerExecute),
new EndEventHandler (EndPreRequestHandlerExecute)
);
}
IAsyncResult BeginPreRequestHandlerExecute (Object source,
EventArgs e, AsyncCallback cb, Object state)
{
// TODO: Begin async operation and return IAsyncResult
}
void EndPreRequestHandlerExecute (IAsyncResult ar)
{
// TODO: Get results of async operation
}
Error Handling while multithreading
Errors can happen at any point during the execution of a command. When ASP.NET can detect errors before initiating the actual async operation, it will throw an exception from the begin method; this is very similar to the synchronous case in which you get the exceptions from a call to ExecuteReader or similar methods directly. This includes invalid parameters, bad state of related objects (no connection set for a SqlCommand, for example), or some connectivity issues (the server or the network is down, for example).
Now, once we send the operation to the server and return, ASP.NET doesn’t have any way to let you know if something goes wrong at the exact moment it happens. It cannot just throw an exception as there is no user code above it in the stack when doing intermediate processing, so you wouldn't be able to catch an exception if it threw one. What happens instead is that ASP.Net stores the error information, and signals that the operation is complete. Later on, when your code calls the end method, ASP.Net detects that there was an error during processing and the exception is thrown.
The bottom line is that you need to be prepared to handle errors in both the begin and the end methods, so it is wise to wrap both events in a try – Catch block.
Conclusion
Now you have seen three of the asynchronous programming models ASP.NET has to offer, hopefully I have impressed upon you how important it is to at least consider using them when creating pages that do I/O if you expect those pages to be heavily trafficked. Remember you can also create asynchronous web services. I didn’t cover those here because there is pretty good documentation for that already.
The good thing about Asynchronous Programming models is that it enables us to build scalable and responsive applications that use minimal resources (threads/context switches).
What is the down side? Well it forces you to split the code into many callback methods, making it hard to read, confusing to debug and difficult for programmers unfamiliar with asynchronous programming to maintain.
With this in mind, whenever I add an asynchronous method to an object in the my projects, I also add a traditional Synchronous version. For example, if I had created a BeginUpdatexxx() Method in the BLL, there would also be a traditional Updatexxx() Method, so that if anyone else finds themselves having to use that object, they won’t be left scratching their heads, wondering “how on earth do I use that?”
Asynchronous command execution is a powerful extension to.NET. It enables new high-scalability scenarios at the cost of some extra complexity.
For more information on multi-threading in ASP.NET I highly recommend you read "Custom Threading in ASP.Net".