Web Performance Test Using Visual Studio - Part II
Performance of an application is very important for a multi-user application. Performance is not only the speed of execution; it includes the load and concurrency aspects. Visual Studio is one of the tools used for Performance Test. Visual Studio Test edition or Visual Studio 2010 Ultimate provides the support for test automation.
Introduction
Performance of an application is very important for a multi-user application. Performance is not only the speed of execution; it includes the load and concurrency aspects. Performance Test is an answer for the following questions
- How can we ensure that our new application will support the expected user load?
- How to avoid the issues which will arise only in real load conditions?
- How to find the response time?
- How to plan the capacity of the servers?
Visual Studio is one of the tools used for Performance Test. Visual Studio Test edition or Visual Studio 2010 Ultimate provides the support for test automation.
This will be the second part of a series of articles.
Part I: Web Test and Customization
Part II: Load Test, RIG and Load test terminologies.
Part III: Performance Counters, Load Test Result Analysis.
Load Test
Load test is used for performing load test on an application. It simulates multiple users as virtual users and executes the test scripts to simulate the real user load on different servers like Application Server, Database server and Web server. Load test can be used with any of the test scripts.
Different Terminologies
Think Time
Think time is the time taken between two requests. This can be the time taken by the user to fill a form, view a page or reading some text etc. Think times are used for simulating the real user scenario; how the system will work with a real user.
Constant Load
Constant load means same number of users hit the site from the starting of the test till the end. It is like 25 users are using the system for a period of 1hr. All 25 users are hitting the system continuously. This type of testing is mainly used for stress testing.
Step Load
In step load, users will join in a step manner. This is the same as different users hitting the system in various times and the number of users in a system is not constant. Following parameter needs to be specified in step load pattern
Start user count :- How many user should hit the system at the time of test start
Step duration :- After how many seconds the next users should join the system
Step user count :- How many user needs to join the system after step duration
Maximum user count :- what is the maximum user count.
Test Mix
Test Mix specifies how different scenarios are executed or used in the system. Different test mixes are formed by studying the system’s usage.
For example, if we are doing a load testing on online marketing site; around 60% people will search for different products, 30% will buy few products and 10% bookmark the products. From this usage information, we can form a test mix as 60% simulated users will execute the script for search, 30% will execute the script for buying a product and remaining 10% will execute the script for bookmarking a product.
Load test Creation
Right click on the project and select Add -> New Test -> Load test template. It will open the New Load Test Wizard
Next, specify the scenario name like booking the ticket, searching a book, etc. Also specify the Think Time profile. We can either use the think times recorded as part of the web test recording or can use a normal think time distribution. We can also avoid the think times recorded in the web test using the third option.
Next, specify the Load Pattern. Specify whether we are going to use a constant load or a step load pattern. In constant load specify the user count. In case of step load specify the start user count, step duration, step user count and maximum user count.
Next, specify the test mix model. The different options are
[Following definitions are from Visual Studio]
Based on the total number of tests
This model of test mix determines which test is run when a virtual user starts test iteration. At the end of the load test, the number of times that a particular test was run matches the assigned test distribution. Follow this model when you are basing the test mix on transaction percentages in an US log or in production data.
Based on the number of virtual users
This model of test mix determines the percentage of virtual users who will run a particular test. At any point in the load test, the number of users who are running a particular test matches the assigned distribution. Follow this model when you are basing the test mix on the percentage of users running a particular testBased on user pace
Over the course of the load test, each test is run the specified number of times per user per hour. Follow this model when you want virtual users to run tests at a certain pace throughout the load test.Based on sequential test order
Each virtual user runs the tests in the order that the tests are defined in the scenario. The virtual user continues cycling through the tests in this order until the load test is complete.
Next, specify the Test Mix. In Test Mix, we are adding different web test scripts to the load test. If we have only one script, all the users are going to perform the same task. If we have 2 or more scripts, we can specify, how many user needs to perform a specific task.
For example, we have two scripts, one for searching a book and another for buying a book. In my load test, I can specify that 70% of users to do search and remaining 30% do the other operation.
Test mix is also for simulating the real user experience. Some of the features are used by many users and others may not be used that much frequently. Depends on the feature usage, we can test the system, which will give a real performance result.
In the following example, we selected two web test scripts and specified the test mix as 65% of the users will execute Webtest1 and 35% will execute Webtest2.
Next, specify the network Mix. Here we can simulate the network like LAN, WAN or internet and compare the performance of the system in different networks.
Next, specify the Browser mix. This will be useful to find how the system is performing in different browsers.
Next, add the systems used as web server, application server and database server for collecting the performance counters. Add the required performance counters for each system.
Next, specify the duration of the test or how many iterations are required. If a warm-up period is set, the load test will automatically increase the load gradually during the warm-up period.
Once, you finish the creation of the load test, it will create a .loadtest file like below. We can edit and add all the settings directly from the below screen itself.
Select the constant Load Pattern and select the corresponding properties. From the property grid, we can change the load pattern as Constant, Step or Goal based.
Goal based load test is used to find the maximum user load which satisfies the condition. Here, we are setting the goal like average response time should be 8min; once the system reaches the goal, it stops execution. Same way we can use goal based testing to find the maximum load which support a processor utilization of 70%.
From the Run Settings properties, we can alter the run duration, web test connection pool size, web test connection model, warm-up duration, etc.
We can specify the threshold value for counters. Once it reaches the warning level, it will start displaying warning. When it reaches the Critical threshold value, the requests will start failing.
You can add custom counters and counter sets to the Counter Sets collection.
From the Scenario properties, we can alter the Think time profile and associated think time.
Execute Load Test
Execute the load test by selecting Run test.
Once the test started, you can observe different performance counters and test status in the following screen. You can drag the counters from left side and drop it in the graph area, which will display the graph corresponding to the counter. Requests summary and tests summary can be observed under Overview section.
We can observe the requests, errors, pages, transactions and the details like how many failed, response time, content length from tables option.
You can change the graph display options to only one graph, two vertical graphs, four vertical panels, etc.
Result
Once the load test execution completes, it display the summary report as follows.
We can export the report to Excel using Create Excel Report option. It creates an excel report with multiple sheets representing average response time, runs, page time, etc.
We can create two types of reports.
Trend – create an excel report with the trends in the selected run result.
Comparison - create an excel report by comparing two or more run results.
Detail option will allow us to study the result in granular format. We can select the time period from the bottom area and study the trend in that particular period.
Rig
Sometimes, in performance test, we need to test the system with very high load. Each machine is having limitation in generating the load. From one system we can go a maximum of 500 to 800 user load. If you need more load, we need to add multiple systems to perform the load test. If we are running it from multiple systems, then the analysis of test results and creation of a combined report will be difficult.
Rig is the solution for running the load test from multiple client systems. We need to install the Load controller on one system and Load agent on other systems. VSTS Rig means a group of systems, which consist of one controller and one or more agents. Controller will distribute the work among the agents and will collect the data from all agents and create a single report.
Load Controller and Load Agent installation and configuration, please refer http://blogs.msdn.com/b/edglas/archive/2007/02/17/load-agent-and-load-controller-installation-and-configuration-guide.aspx
Conclusion
Load test can be used for performing different performance tests on a system. We can test the system by setting different test mixes, user loads, stress conditions, network mixes, browsers and load patterns. Load test will generate a detailed report, which can be queried from the LoadTest or LoadTest2010 database under SQLExpress. Also, from the result we can create an Excel Trend report or Comparison report. Summary report can be saved as HTML file.
