pnp.gif

How To: Create a Load Test Plug-In using Visual Studio Team System

J. D. Meier, Prashant Bansode, Carlos Farre, Scott Barber

Applies to

  • Microsoft® Visual Studio® Team System
  • Performance Testing.

Summary

This How To explains how to write and use an extensible Load Test plug-in that allows code to be executed at different times while the load test is running. To create a Load Test plug-in, you will need to create a class that inherits the ILoadTestPlugin interface. This How To explains the steps involved in creating a Load Test plug-in class and integrating it with your load test.

Contents

  • Objectives
  • Overview
  • Summary of Steps
  • Step 1 – Create Load Test Containing One or More Web Tests in a Test Mix
  • Step 2 – Create a Load Test Plug-In Class
  • Step 3 – Add the New Load Test Plug-In Class to the Test Project
  • Step 4 – Add the Load Test Plug-In to the Load Test
  • Step 5 – Add Context Parameters
  • Step 6 – Run the Load Test

Objectives

  • Learn how to write and use Load Test plug-in.

Overview

You can create a Load Test plug-in to run code at different times while a load test is running; for example:
  • After a threshold has been violated
  • Before a load test starts and after a load test finishes
  • Before a test iteration starts and after it finishes
  • When a load test aborts
  • When the load in the test has changed.

The events that the implementation of ILoadTestPlugin can handle are LoadTestStarting, LoadTestFinished, LoadTestWarmupComplete, TestStarting, TestFinished, TestSelected, ThresholdExceeded, HeartBeat, and LoadTestAborted.
An implementation of a Load Test plug-in can be used to extend or modify the behavior of a load test as it runs. The following list contains some possible uses of a Load Test plug-in:
  • Start an external process such as a monitoring tool just before the load test starts, and stop it when the load test completes.
  • Provide programmatic control over the user load as the load test runs; overriding the user load specified in the load test editor.
  • Provide programmatic control over the selection of an individual test to run within the load test, overriding the default test selection algorithm.
  • Abort the load test if the value of a performance counter causes the failure of a threshold rule defined in the load test.
  • Maintain a shared variable across all test iterations in a load test.

Summary of Steps

  • Step 1. Create Load Test Containing One or More Web Tests in a Test Mix
  • Step 2. Create a Load Test Plug-In Class
  • Step 3. Add the New Load Test Plug-In Class to the Test Project
  • Step 4. Add the Load Test Plug-In to the Load Test
  • Step 5. Add Context Parameters
  • Step 6. Run the Load Test

Step 1. Create a Load Test Containing One or More Web Tests in a Test Mix

First, create a test project that will contain your load test and Web tests. Next, create individual Web tests that reflect key business user scenarios. Finally, create a load test to which you will add your Web tests. While creating the load test, you can set many run-time properties to generate the desired load simulation; for example, you can specify the load pattern, browser, and network types and add the performance counters to be monitored.
For more information on creating load tests, see “How To: Create a Load Test Using Visual Studio .NET 2005" at << to link to how to >>

Step 2. Create a Load Test Plug-In

In this step, you create a plug-in class for your load test project. The Load Test plug-in is a point of extensibility that allows code to be executed at different times while the load test is running. To create a plug-in, you can expand upon the built-in functionality of the load test. To do this, you must first create a class that inherits the ILoadTestPlugin interface. This class must implement the Initialize method of this interface.
The process of writing a Load Test plug-in can be summarized as follows:
  • Create a plug-in class.
  • Implement the Initialize method.
  • Implement the relevant event handler.

Create a Load Test Plug-in Class

To create a load test plug in class, carry out the following steps:
  • Add a new class to your test. To do so, right-click your project, click Add, click Class, and then name your class MyLoadTestPlugin
  • Add the LoadTesting namespace to your project. To do so, add a reference to Microsoft.VisualStudio.QualityTools.LoadTestFramework by right-clicking your test project and then clicking Add Reference. Next, add the following using statement to your class:
using Microsoft.VisualStudio.TestTools.LoadTesting;
  • Inherit your class from ILoadTestPlugin as follows:
public class MyLoadTestPlugin: ILoadTestPlugin
  • Add member variables to your class as follows:
private LoadTest mLoadTest;


Your class should now look like this:

 using System;
 using System.Collections.Generic;
 using System.IO;
 using System.Text;
 using Microsoft.VisualStudio.TestTools.LoadTesting;

 namespace CustomPlugin
 {
 
    public class CustomLoadTestPlugin : ILoadTestPlugin
    {
        #region ILoadTestPlugin Members
        private LoadTest mLoadTest;
        private string someOthermemberVariable;       
    }
 }


More Information
The purpose of the member variables is as follows:
  • mLoadTest is initialized with the LoadTest object. The LoadTest object holds the context of a load test’s settings, the available run-time variables, and the event handlers to be executed in different stages of load test execution. Below you will find a description of the context variables used from the LoadTest object and event handler used to control the number of test iterations.
  • If required, other member variables can be declared that can be populated by using the parameters passed through the parameter context.

Implement the Initialize Method

Implement the Initialize method of the ILoadTestPlugin interface, which takes the LoadTest object as a parameter. In the Initialize method, assign local variable mLoadTest and any other declared member variables whose value is available through parameter context. Also subscribe to any of the events that are relevant to the purpose of your Load Test plug-in, for example TestFinishedEvent, raised when iteration of one test is finished.

 public void Initialize(LoadTest mloadTest)
 {
   //load Test object
   this.mLoadTest = myloadTest;            
   if (this.mLoadTest.Context.ContainsKey("SomeOtherParameter"))
   {
      try
      {
         //Retrieve the value
	 someOthermemberVariable = (String)this.mLoadTest.Context[
"SomeOtherParameter"]);
      }
      catch (FormatException)
      {
	   throw new ApplicationException(
                " SomeOtherParameter not in string format");
      }
   }
                
   //Subscribe to the relevant event, in this case we are subscribing to
   // TestFinished event 
   this.mLoadTest.TestFinished += new 
                EventHandler<TestFinishedEventArgs>(LoadTestFinished);
 
   ….
 }


More Information
Before the test starts, the Initialize method is called once. The mLoadTest variable is assigned with LoadTest passed as a parameter; member variables are assigned from the context parameters read from the LoadTest object.
Also, the Load Test plug-in subscribes to the one of the following event handlers, depending on which one is relevant to the intended purpose of the custom code:
  • LoadTestStarting – Occurs just before the load test begins to run on the agent. If an ILoadTestPlugin implementation has code that should be run before the load test starts on any agent, that code should be placed in the Load Test plug-in Initialize method instead of in the LoadTestStarting event handler, because the former is guaranteed to run before the load test is started on any agent, whereas the latter is not.
  • LoadTestFinished – Occurs when the load test has stopped running on the agent.
  • LoadTestWarmupComplete – Occurs when the warm-up period has completed.
  • TestStarting – Occurs when an iteration of one of the tests contained within the load test is started. TestStartingEventArgs contains a reference to the test that is starting, and the LoadTestScenario in which it is starting.
  • TestFinished – Occurs when an iteration of one of the tests contained within the load test finishes. TestFinishedEventArgs contains the TestIterationResult object for the test that is finishing.
  • TestSelected – Occurs every time an iteration of one of the tests contained within the LoadTest is selected. The event handler may change the test to be started by setting the TestName property of the TestStartingEventArgs class.
  • ThresholdExceeded – Occurs when a threshold rule is exceeded. ThresholdExceededEventArgs identifies the performance counter that exceeded the threshold, the value of the performance counter sample, and whether the value exceeded the critical threshold level or only the warning threshold level. If there is more than one agent computer running the load test, the ThresholdExceeded event will be fired on all the agents.
  • Heartbeat – Occurs once every second. HeartBeatEventArgs indicates the elapsed time into the load test warm-up period or measurement period. ILoadTestPlugin could use the Heartbeat event as a time to change the CurrentLoad property of a LoadTestScenario in the load test.
  • LoadTestAborted – Occurs when the load test is aborted.

Implement the Relevant Event Handler.

In our case, we have subscribed to the TestFinished event; you will implement the relevant event handler as follows:

 void LoadTestFinished(object sender, TestFinishedEventArgs e)
 {
      // custom code to be executed when the test is finished.
 }


More Information
Every time a test is completed; the TestFinished event is raised, which will be handled by this callback function. Any custom code in the event handler will be executed each time the test execution has finished.

Example Load Testing Plug-In Class

The following is the complete code sample for the plug-in class, which you can use as a reference to build your own custom Load Test plug-in:

 using System;
 using System.Collections.Generic;
 using System.IO;
 using System.Text;
 using Microsoft.VisualStudio.TestTools.LoadTesting;
 
 namespace CustomPlugin
 {
    public class CustomLoadTestPlugin : ILoadTestPlugin
    {
        #region ILoadTestPlugin Members
        private LoadTest mLoadTest;
        private string SomeOtherParameter;
 
        public void Initialize(LoadTest loadTest)
        {
            mLoadTest = loadTest;
 
            if (mLoadTest.Context.ContainsKey("SomeOtherParameter"))
            {
                try
                {
                   SomeOtherParameter =
                   (string)mLoadTest.Context["SomeOtherParameter"];
                }
                catch (FormatException)
                {
                    throw new ApplicationException("MaxTestIterations 
                                                    not in string format");
                }
 
                mLoadTest.TestFinished += new 
                EventHandler<TestFinishedEventArgs>(mLoadTestTestFinished);
 
                ….
            }
        }


        void mLoadTestTestFinished(object sender, TestFinishedEventArgs e)
        {
            // Custom code to be executed when test execution is finished
        }
        #endregion
    }
 }

Step 3. Add the New Load Test Plug-In Class to the Test Project.

After the Load Test Plug-in class has been developed, add the class project containing the plug-in into the test project. This is required for configuring the load test properties for the Load Test plug-in.
  • Right Click on the test solution.
  • Select Add and then click Existing Project....
  • Browse to project containing the Load Test Plug in.
  • Select the Load Test Plug in and it will be added to the Test Project.

Step 4. Add the Load Test Plug-In to the Load Test.

In this step, you add the Load Test plug-in to the load test so that it can be executed. For a load test to execute the Load Test plug in, the load test needs to be configured after the Load Test plug-in has been added to the test project. The Load Test plug-in needs to be added in the properties of the root node of the load test. You do this by performing the following detailed steps:
  • After the load test has been created, click the root node of your load test.
  • In the Properties pane of the load test, click the ellipsis (* … *).
LoadTestPlug-In 1.GIF
  • The Set Load Test Plug-in dialog box appears, showing available Load Test plug-ins. Select MyLoadTestPlugin.
LoadTestPlug-In 2.GIF
  • Click the Ok button.

Step 5 Add Context Parameters.

The context parameters are data object variables that you pass to the Load Test plug-in. These paramters can be assigned to local variables during the initialize phase before the load test executes, or during the execution of the callback function. Context parameters are used to allow the passing of variables during test execution to make it possible to change load test settings such as current load, based on conditions such as the Web test name and the number of test executions.
Add context parameters by performing the following steps:
  • In the load test, right-click Run Settings.
  • Click the Add Context Parameter option.
LoadTestPlug-In 3.GIF
  • In the Properties pane, you will see Name and Parameter1 fields with an empty Value. Create the required parameters one at a time; for example, add a parameter named SomeOtherParameter and set its value to the context parameter you want to pass.
LoadTestPlug-In 4.GIF
  • Add any other additional parameters.

Step 6 – Run the Load Test

In this step, you run the load test (created with one or more Web tests in the mix setting) which now includes the Load Test plug-in that was added in the previous steps.
  • In the Load Test Editor, right-click the load test and select the Run Test option. The test runs until the condition is met and an abort message is displayed at the end.

Resources

Last edited Mar 15, 2007 at 10:43 PM by prashantbansode, version 15

Comments

No comments yet.