Each time a request for a .NET resource (.aspx, page, .ascx control, etc.) comes in a thread is grabbed from the available worker thread pool in the asp.net worker process (aspnet_wp.exe on IIS 5.x, w3wp.exe on IIS 6/7) and is assigned to a request. That thread is not released back into the thread pool until the final page has been rendered to the client and the request is complete.

Inside the ASP.Net Worker Process there are two thread pools. The worker thread pool handles all incoming requests and the I/O Thread pool handles the I/O (accessing the file system, web services and databases, etc.). But how many threads are there in these thread pools? I had assumed that the number of threads would vary from machine to machine – that ASP.NET and IIS would carefully balance the number of available threads against available hardware, but that is simply not the case. ASP.Net installs with a fixed, default number of threads to play with: The CLR for the 1.x Framework sets these defaults at just 20 worker threads and 20 I/O thread per CPU. Now this can be increased by modifying the machine.config, but if you are not aware of this, then 20 threads is all you’re playing with. If you have multiple sites sharing the same worker process, then they are all sharing this same thread pool.

So long as the number of concurrent requests does not exceed the number of threads available in the pool, all is well. But when you are building enterprise level applications the thread pool can become depleted under heavy load, and remember by default heavy load is more than just 20 simultaneous requests. When this happens, new requests are entered into the request queue (and the users making the requests watch an hour glass spin). ASP.NET will allow the request queue to grow only so big before it starts to reject requests at which point it starts returning Error 503, Service Unavailable.

If you are not aware of this “Glass Ceiling of Scalability”, this is a perplexing error – one that never happened in testing and may not reproducible in your test environment, as it only happens under extreme load.

So the first thing you can do to improve scalability is to raise the values. The defaults for the ASP.NET 2.0 are 100 threads in each pool per CPU and the defaults for the ASP.NET 3.x CLR is 250 per CPU for worker threads and 1000 per CPU for I/O threads, however you can tune it further using the guidelines below. 32 bit windows can handle about 1400 concurrent threads, 64 bit windows can handle more, though I don’t have the figures.

You can tune ASP.NET thread pool using maxWorkerThreads, minWorkerThreads, maxIoThreads, minFreeThreads, minLocalRequestFreeThreads and maxconnection attributes in your Machine.config file. Here are the default settings.

<system.net>

  <connectionManagement>

     <add address="*" maxconnection="2" />

  </connectionManagement>

</system.net>

<system.web>

  <httpRuntime minFreeThreads="8" minLocalRequestFreeThreads="4"  />

  <processModel maxWorkerThreads="100" maxIoThreads="100"  />

</system.web>

Here is the formula to reduce contention. Apply the recommended changes that are described below, across the settings and not in isolation.

• Configuration setting Default value (.NET Framework 1.1) Recommended value
• maxconnection 2 12 * #CPUs
• maxIoThreads 20 100
• maxWorkerThreads 20 100
• minFreeThreads 8 88 * #CPUs
• minLocalRequestFreeThreads 4 76 * #CPUs
• Set maxconnection to 12 * # of CPUs. This setting controls the maximum number of outgoing HTTP connections that you can initiate from a client. In this case, ASP.NET is the client. Set maxconnection to 12 * # of CPUs.
• Set maxIoThreads to 100. This setting controls the maximum number of I/O threads in the .NET thread pool. This number is automatically multiplied by the number of available CPUs. Set maxloThreads to 100.
• Set maxWorkerThreads to 100. This setting controls the maximum number of worker threads in the thread pool. This number is then automatically multiplied by the number of available CPUs. * Set maxWorkerThreads to 100.
• Set minFreeThreads to 88 * # of CPUs. This setting is used by the worker process to queue all the incoming requests if the number of available threads in the thread pool falls below the value for this setting. This setting effectively limits the number of requests that can run concurrently to maxWorkerThreads – minFreeThreads. Set minFreeThreads to 88 * # of CPUs. This limits the number of concurrent requests to 12 (assuming maxWorkerThreads is 100).
• Set minLocalRequestFreeThreads to 76 * # of CPUs. This setting is used by the worker process to queue requests from localhost (where a Web application sends requests to a local Web service) if the number of available threads in the thread pool falls below this number. This setting is similar to minFreeThreads but it only applies to localhost requests from the local computer. Set minLocalRequestFreeThreads to 76 * # of CPUs.

Note The recommendations that are provided in this section are not rules. They are a starting point. Test to determine the appropriate settings for your scenario. If you move your application to a new computer, ensure that you recalculate and reconfigure the settings based on the number of CPUs in the new computer.

By raising these values, you raise the “glass ceiling of scalability”, and in many cases that may be all you need to do, but what happens when you start getting more than 250 simultaneous requests? To make optimum use of the thread pool all IO requests that you know could take a second or more to process should be made asynchronously. More information on Asynchronous programming in ASP.NET is coming soon.

I recommend testing your new machine.config locally or virtually somewhere first because if you make a mistake - for example you paste this in to the wrong area, or there is already a <system.web> element and you paste in a second one - ALL websites on the box will stop functioning as ASP.Net will not be able to parse the machine.config!!!

A Data Access Layer that does not support Asychronous I/O is not scalable. Period. However, since Microsoft kindly provided the source code along with the Enterprise Library, it is possible to make the Enterprise Library's Data Application Block scalable by adding the Asynchronous methods (BeginExecuteNonQuery, BeginExecuteReader, etc.) to the SQL Database Object. I took the code from the 3.x library and modified the file SqlDatabase.cs found here c:\EntLib3Src\App Blocks\Src\Data\Sql (assuming you installed the source files to the root of your C drive). The following download includes the modified binaries and my SqlDatabase.cs code file:

AsyncEL.zip (348.39 kb)

I don't think I implemented every overload, and again this is for the 3.0 library, but you should be able to modify the 4.x libraries and add more overloads by following the patterns I'm using. To use the binaries included in the zip file above, simply drop them into the bin folder of your web site or web application project, add references if necessary and don't forget to add the async=true attribute to both your <% page %>  tag and your connection string! Here is a simple example of how you might use it, to get you started:

 

 

Download the Full Enterprise Library (including the complete source code) from http://www.codeplex.com/entlib.

 

If you have added a Web Reference for a Web Service recently you may have noticed that the wsdl tool no longer creates Begin and End methods for you, instead it implements what is known as the Event based Async Pattern. Which is fine if you are building a Windows or Console application, but if you are trying to call a web service asynchronously using the AddOnPreRenderAsync or RegisterAsyncTask methods in ASP.NET you really need a Begin method that returns an IAsyncResult Interface. There are several ways you can work around this.

You could, assuming you still have it, create a project in Visual Studio 2003, add the web reference there and copy it to your 2005 project, or you can run the wsdl tool manually and use the /parameters option to specify a configuration file with the "oldAsync" flag. For example, for a reference to http://my.server.com/service, which creates a proxy class with both begin/end (a.k.a. "old style") and event-based (a.k.a. "new style") methods, you'd use the command

  wsdl.exe /parameters:MyParameters.xml http://my.server.com/service

Where MyParameters.xml is as below:

MyParameters.xml:

<wsdlParameters xmlns="http://microsoft.com/webReference/">
  <nologo>true</nologo>
  <parsableerrors>true</parsableerrors>
  <sharetypes>true</sharetypes>
  <webReferenceOptions>
    <verbose>false</verbose>
    <codeGenerationOptions>properties oldAsync newAsync</codeGenerationOptions>
    <style>client</style>
  </webReferenceOptions>
</wsdlParameters>

However, my preferred method is to set a project level property WebReference_EnableLegacyEventingModel to true.

To do that, you need unload the project from Visual Studio, and edit the project file directly in a text editor. (I recommend creating a backup first)

In the first section of PropertyGroup, you will see many properties like:

    <ProjectGuid>{F4DC6946-F07E-4812-818A-F35C5E34E2FA}</ProjectGuid>
    <OutputType>Exe</OutputType>
...

Don't change any of those, but do add a new property into that section:

    <WebReference_EnableLegacyEventingModel>true</WebReference_EnableLegacyEventingModel>

Save the file, and reload the project into Visual Studio.

After that, you have to regenate all proxy code (by updating the reference, or running the custom tool on the .map file)

This is my preferred method because unlike the others I described, this one lets you refresh or update the web reference as normal, which if your project is shared with other developers is a big plus!