I am writing a client simulation program in which all simulated client runs some predefined routine against server - which is a web server running in azure with four instances.

All simulated client run the same routine after getting connected to server.

At any time I would like to simulate 300 to 800 clients using my program.

My question is: Should I create N instances of client class and run them in N different threads? OR

Should I use Task Library to do the things?

• c#
• multithreading
• task-parallel-library
• simulation

You certainly should not create 800 threads.

Let's take a step back here. You have a device called a "server" which takes in "requests" from "clients" and sends out "responses" back to those clients. Let's suppose that the requests are pieces of paper delivered by the post office, and the responses are boxes containing books, also delivered by the post office.

You wish to simulate 800 clients in order to test the server.

Let's suppose that a thread is a person and a processor is a chair. A person can only do work while sitting in the chair.

Creating 800 threads is the equivalent of going out and hiring 800 people, and paying each of them to send a letter to the server. But you only have four chairs, so those 800 people have to all take turns using the chairs.

That would be a ludicrous solution in real life. Threads, like people, are insanely expensive. You should be minimizing the number of threads you create.

So, should you instead create 800 tasks via the task factory and let the TPL parallelize them for you?

No, you should not do that either. The TPL has a pool of people (threads) to draw from, and it tries to arrange things so that there are no more people on the payroll than there are chairs for them to sit in. But your task is not "chair bound" -- the people are going to sit in the chair, send the request to the server, and then get out of the chair while they wait for the response to come back. While they are waiting, the TPL now has to hire more people to service the additional tasks.

Hitting a web server is I/O bound; you should only create thread-pooled tasks for tasks that are CPU bound.

The right solution is to hire two people.

One person -- the "I/O completion thread" -- does nothing but drop requests in the mailbox and check for incoming packages. The other person -- the "simulation" person -- works out what the right "schedule" is for simulating 800 clients. The simulation person works out the schedule, and then goes to sleep. She wakes up when it is time to send another request to the server. When she wakes up, she tells the I/O completion thread to drop this letter in the mailbox, and wake her up when the response comes in. She then goes back to sleep until either it is time to send another request, or a response comes in that needs to be verified.

What you should do is either (1) get the beta version of C# 5 and use async/await to create tasks that send requests to the server, and then yield control back to the message loop until either it is time to send another request or a response comes in. Or, if you don't want to use C# 5, you should create a Task Completion Source, and set up tasks that have the right continuations.

In short: the right way to handle many parallel I/O tasks is to create a very small number of threads, each of which does a very small amount of work at a time. Let the I/O completion thread handle the details of the I/O. You do not need to hire 800 people in order to simulate sending 800 letters. Hire two people, one to watch the mailbox and one to write the letters.

The answer in this case is not so simple. It really depends on how you want your clients to be simulated:

1. If you want to have 800 clients connected, but not necessarily at the same time, it's a good idea to use Tasks. They are lightweight and make efficient use of the underlying ThreadPool.

2. If you really want the clients to be absolutely all in parallel, I'm afraid there is no way to actually avoid threads. There is no magical way to get 800 lightweight simultaneous executing tasks. The Task abstraction is lightweight precisely because it uses the thread pool. This means that many tasks are mapped to a small number of actual threads. But, of course, this implies that they do not truly run in parallel, but are instead scheduled to run whenever possible. The ThreadPool has a maximum number of threads of 250 (AFAIK), so no more than 250 "clients" will actually execute at one time if you use Tasks. The solution is set max threads to 800, but at this point it's the same as using classic threads.

I would use the task library and let the task library handle all the threading for you. You don't want to spin up 800 threads. Its a bad idea to have that many simultaneous threads going at a time, here is another stack overflow question that talks about that: Maximum number of threads in a .NET app?

For this application domains are your best bet.

An application domain is the runtime unit of isolation in which a .NET application executes. It provides a managed memory boundary, a container for application configuration settings as well as providing a communication interface for distributed applications.

Each .NET application usually hosts just one application domain which is automatically created by the CLR when the given process/program starts. It is sometime useful (in a case such as yours) to create additional application domains with in a single process/program. Using multiple application domains avoids communication complications and arise using several individual processes and provides isolation of your tasks.

For what you want you have two options.

1. Start X threads on separate thread in the same domain.

This will mean that you will have to be very weary of being thread-safe, which will be very difficult with such a task as simulating multiple logins, simulate clients etc.

1. Start X threads in the same process each in its own application domain.

This will keep each of the spun threads isolated and also easy to access by the hosting application/program. By having all of you X simulation in X separate application domains, each domain would be isolated and unable to interfere with another client simulation via static class members etc.

The following is aided by an extract from Joseph Albahari's book C# 4.0 In a Nutshell which I strongly recommend getting:

An example of 40 concurrent client simulations might be of use to you:

class program
{
    static void main()
    {
        // Create 40 domains and 40 threads.
        AppDomain[] domains = new AppDomain[40];
        Thread[] thread = new Thread[40];

        for (int i = 0; i < 40; i++)
        {
            domains[i] = AppDomain.CreateDomain("Client Simulation " + i);
            thread[i] = new Thread(SimulateClientInOtherDomain);
        }

        // Start all threads, passing to each thread its app domain.
        for (int j = 0; j < 40; j++)
            threads[j].Start(domains[j]);

        // Wait for the threads to finish.
        for (int k = 0; k < 40; k++)
            threads[k].Join();

        // Unload the application domains.
        for (int l = 0; l < 40; l++)
            AppDomain.Unload(domains[l]);
    }

    // Thread start with input of with domain to run on/in.
    static void SimulateClientInOtherDomain(object domain)
    {
        ((AppDomain)domain).DoCallBack(Simulate);
    }

    static void Simulate()
    {
       Client simClient1 = new Client("Bill", "Gates", ...);
       simClient1.Simulate();
    }
}

I hope this helps.