Image CAPTCHA
Enter the characters shown in the image.

You are here

JMeter response assertions: How to know what to validate in an HTTP response request

JMeter response assertions: How to know what to validate in an HTTP Response Request

 

For those of us who work with JMeter and performance tests, we know the importance of adding validations to all HTTP Requests.
Validations are not just a way to confirm that the server is responding as expected, but they also serve to verify that the script we are developing is acting as it should, meaning, it is simulating the user behavior as it was designed.

For example, if running our script on the system implies that a record is created in the database, we have to ensure that after running the script, the record has indeed been created and that the script doesn’t say everything went smoothly if it hadn’t actually created it. That would be an example of a bad choice of an assertion or validation of our requests.

We want to avoid false negatives and false positives. Here’s a review of what those mean:

  • False Positive: The test says there is an error when there really isn’t one and then we lose time looking for the cause of the error until we realize that the error was just in the test, or in the environment, or the data, etc.
  • False Negative: There are errors but they are not found. Since they were not found, we believe that these features have been fully covered and they are bug free although they are not.

If there is an error, it’s important that the script fails. Conversely, if it goes well, the script has to confirm that it went well and we have to be able to trust it.

 

How do we select good assertions?

The reality is that I have not found a guide on how to choose these, and it always stumps me when I am teaching JMeter to someone and they ask me how do I know what to validate in the response of an HTTP request?

As I mentioned in the post, 8 Ways to Improve JMeter Scripts Just validating a response code is not enough, because we can lose other errors that we need to capture in order to act accordingly. 

Therefore, today I am sharing with you the methods and criteria for selecting which JMeter response assertions to make when validating the responses of the server. 

 

 

1 - Validate the login

Typically, after authenticating a system, a 'Welcome @username' message appears and our name appears somewhere on the screen, indicating that the login was successful. In these cases, we have an excellent element to validate.

Let’s see this with an example:

  1. log in with the user’s credentials 
  2. successfully enter the site and it shows the user name on the screen
  3. search for how the element appears in the response HTML
  4. validate it in JMeter

Recall that in JMeter, we can specify a variable in the validations, then, in cases where the user name is parameterized (read from a csv file or defined as a user variable), one can verify the text ${UserName} and JMeter will replace the variable value.

 

 

2. Validate window content

Many other times requests return a page in which you can choose text/titles that should come in the HTML response. For example, if we access the JMeter site, (http://jmeter.apache.org/) we can see the following:

 

We know that if this page loads correctly, the title “What can I do with it?”  should appear in the response to the request. If it doesn’t appear, then you know you have an error, so it’s a good assertion to add as a validation that we are getting the expected page.

The challenge in this type of validation is to verify content that we know that, if there is an error, then that text does not come in the response. We can rely on developers to provide us with this kind of information.

 

 

3. Consider the next steps in the script

If we know that the next action to take is click a button X, or access the link Z, or select in the combo Option B, a very good strategy is to validate that the element on which we operate in the next step, appears in the response of the request. For example, if after loading the JMeter site, I want to select the option “Download Releases” I can validate this text and I assure the flow continues.

For this, I capture the response from the server verifying that it includes the option I want to check: 

And then I create the validation in JMeter:

 

 

4. Validate values that need to be extracted

There are times when we obtain variables from the response of an HTTP request through the use of regular expressions (Add -> Post Processors -> Regular Expression Extractor) with the intention to use that variable in a subsequent request.

We can then validate that the variable appears in the server response. For this we can make use of validations that use regular expressions.

For example:

Thus, if the regular expression doesn’t match in the response, we will get a validation error.

 

 

5. Take special care with special characters

Be careful when writing the assertion in JMeter because the text is not always given in the same format that is displayed. For example, to validate the title Apache JMeter ™, I cannot write it verbatim because the server response comes as Apache JMeter & trade; and not the other way as seen on the screen. If you do not use the text as it appears in the HTML, the validation (and regular expressions) will not find it.

 

 

6. Use a proper amount of assertions

JMeter allows us to add as many assertions as the we want, so that each of the assertions that were named above may be accompanied by other assertions such as the titles of the HTML responses (<title> … </ title> ), in order to add strength to the validation. When adding more assertions, we have to be careful that the script is not too loaded, which is why it’s not advisable to add more than two assertions per request, so we must ensure that the ones we choose are determining factors in defining the correctness of the response.

 

I hope that this list will be useful for you when you add assertions. Can you share any other ideas you may have?

 

 

About The Author: 

Leticia Almeida - Leticia is passionate about improving the performance of IT systems. A certified BlazeMeter performance tester, she has many years of experience in performance testing for Abstracta, where she has taken on several roles, analyzing the performance of applications built upon multiple technologies. She has lead various test teams and managed performance projects for clients from all corners of the world such as the USA, Latin America, and Japan.


 

Media: 

.

Performance Analysis

List mode
Table mode
Icons mode

Displaying 1 - 52 of 52

Title Testing Objectives Since Price Quotes Ping
eggPlant Functional Application Life Cycle Tracking, Cross Platform Testing, Desktop testing, Image Based Testing, Embedded Software, Image Comparison, Java testing, Mobile Testing, Network testing, Performance Analysis, Web testing, Cross-Browser testing 2,008 Active
Parasoft SOAtest API testing, Banking Process validation, DataBase testing, Interoperability testing, Stress Testing, Message testing, Performance Analysis, Protocol Testing, Security Testing, SOA testing, Web Monitoring, Web testing, Cross-Browser testing 2,002 Call manufacturer for more details Active
NeoLoad Stress Testing, Mobile Testing, Performance Analysis, Web testing 2,005 Please Call Manufacturer for more details, April 2012: Total Unit price start at $1300, January 2012: Unit price start at $1860 Active
StormRunner Load Mobile Testing, Performance Analysis, Web testing 2,013 Not Published Active
Fiddler Load Capabilities, Performance Analysis, Security Testing, Web testing, Cross-Browser testing 2,003 Free use Active
Silk Performer Load Capabilities, Mobile Testing, Performance Analysis, SAP Testing, Web testing 1,994 Active
JMeter API testing, Load Capabilities, Stress Testing, Network testing, Performance Analysis, Server testing, SOA testing, Web testing 1,999 Free Use Active
WebLOAD Load Capabilities, Stress Testing, Mobile Testing, Performance Analysis, Web testing 1,993 Call Manufacturer for details Active
AppLoader Desktop testing, Image Based Testing, EMail Testing, Image Comparison, Java testing, Load Capabilities, Stress Testing, Performance Analysis, SAP Testing, Web testing 2,000 Please Call Manufacturer for more details... Active
QF-Test Desktop testing, Image Comparison, Java testing, Load Capabilities, Performance Analysis, SAP Testing, Test Cases Statistics, Test Design, Web testing, Cross-Browser testing 2,001 Developer license (Test creation and execution): One GUI-technology included. 1,995 €, GUI-technology extensions: Purchase now or later on demand. 500 €, Maintenance (obligatory for initial license purchase): All Upgrades + support for 1 year with one GUI-technology. 480 €, Runtime license (Test execution): Substantial volume discounts for load testing. 995 € Active
SandStorm Stress Testing, Mobile Testing, Performance Analysis, Protocol Testing 2,011 Active
LoadUI NG Pro Network testing, Performance Analysis, Web testing 2,008 Active
StresStimulus SAP Testing, Mobile Testing, Network testing, Performance Analysis, Web testing, Cross-Browser testing, Test Cases Statistics, Server testing, DataBase testing, Load Capabilities, Stress Testing, Java testing 2,006 September 2015: Free version is available, September 2015: 250 VUs, Pro Edition, Permanent license - $870, September 2015: 10,000 VUs, Ent. Edition, weekly pass - $2,130 Active
Test Studio Desktop testing, Java testing, Mobile Testing, Performance Analysis, Test Cases Statistics, Test Design, Web testing, Cross-Browser testing 2,002 Test Studio Web & Desktop $2,499 perpetual license, include: Automated testing for Web and Desktop Visual Studio plugin, Test Studio Load $999 perpetual license, include: Load testing web UI Load testing web services, Test Studio Ultimate $3,499 perpetual license include: Test Studio Web & Desktop, Mobile, Load, Performance + API testing Active
HammerDB DataBase testing, Stress Testing, Performance Analysis 2,003 Free use Active
Load Tester DataBase testing, Data Migration Testing, Java testing, Stress Testing, Mobile Testing, Network testing, Performance Analysis, Cross-Browser testing 1,999 1 Month Lease $750 Active
WAPT Load Capabilities, Stress Testing, Performance Analysis, Web testing 2,010 2016: WAPT, $700 (server + 2 agents), 2016: WAPT PRO, $1200 (server + 2 agents) Active
SiteScope Performance Analysis, Web testing 2,008 Call Manufacturer Active
BlazeMeter Desktop testing, Stress Testing, Performance Analysis, Web testing, Cross-Browser testing 2,011 October 2014 :Basic - $249/month/1000 users, October 2014 :Pro - $649/month/5000 users, October 2014 :Pro Plus - $1349/month/10,000users Active
RTI Performance Analysis 1,999 Active
TestMaker Desktop testing, Performance Analysis, SOA testing, Web testing 2,000 Active
Loadster Load Capabilities, Stress Testing, Network testing, Performance Analysis, SOA testing, Web testing 2,004 2015: Loadster Cloud-1000 units : $79, 2015: Loadster Cloud-10000 units: $499 , 2015: Loadster Engine 3.x : $699 Active
CLIF Performance Analysis, Stress Testing 2,007 Free Use Active
mAutomate Mobile Testing, Performance Analysis 2,011 Active
eggPlant Performance Load Capabilities, Stress Testing, Performance Analysis, SOA testing, Web testing 2,008 Active
Topaz Data Flow Analysis, Performance Analysis 2,015 Active
Runscope API testing, Performance Analysis, SOA testing, Web Services, Web Monitoring, Web testing 2,013 Active
Tsung Protocol Testing, Network testing, Performance Analysis, SOA testing, Web Services, Web testing, Web Monitoring, Server testing, Load Capabilities, Stress Testing, Memory Analysis, Runtime Error Detection, Memory Leak Detection 2,001 Free Use 2015
AQtime Pro Code Analysis, Code Coverage testing, Performance Analysis 2,011 2016: Node-Locked (Named User) Licenses = $599, 2016: CONCURRENT USER LICENSE = $1899 Active
OpenSTA Load Capabilities, Stress Testing, Performance Analysis, Test Cases Statistics, Web testing 2,003 Free use 2007
StressTester Mobile Testing, Network testing, Performance Analysis, SOA testing, Web testing, Server testing, Message testing, DataBase testing, Database Upgrade Testing, Load Capabilities, Stress Testing 2,003 Nov 2015 : Controller including 25 Virtual Users - $2,049, Nov 2015 : Controller including 100 Virtual Users - $4,950, Nov 2015 : Controller including 500 Virtual Users - $15,939 Active
TestQuest 10 Mobile Testing, Performance Analysis, Load Capabilities, Stress Testing 1,999 Active
Apica ZebraTester Load Capabilities, Stress Testing, Performance Analysis, Web testing 2,015 Active
MStone Protocol Testing, Performance Analysis, Message testing, Load Capabilities, Stress Testing, EMail Testing 2,005 Free Use 2013
PureLoad Stress Testing, Performance Analysis, Web testing 1,998 PureLoad Web - $5000/500 virtual clients, PureLoad Enterprise - $12000/500 virtual clients, PureLoad Enterprise, Developers Edition - $1250/10 virtual clients Active
mPulse Metrics Analysis, Performance Analysis, SOA testing 2,006 July 2015: Lite - Free/ 1 user, July 2015: Pro - $99/month Active
OctoPerf Load Capabilities, Stress Testing, Performance Analysis 2,015 2015 - Basic 39€ (3 purchased minimum) , 2015 - Advanced 499€, 2015 - Enterprise 999€ Active
AppPerfect LoadTest Java testing, Stress Testing, Performance Analysis, Web testing 2,003 Feb 2015 : Desktop licence : $299/ 25 VU, Feb 2015 : Floating Licence : $399/ 25 VU, Feb 2015: Additional Virtual Users : $1/ VU Active
Monkop Security Testing, Mobile Testing, Network testing, Performance Analysis, Metrics Analysis, Memory Analysis 2,014 Nov 2015: Solo - Free, Nov 2015: Basic - $90 / month / 5 Multi-Device executions per month , Nov 2015: Pro - $349 / month / 30 Multi-Device executions per month Active
TestWise Code Analysis, Code Coverage testing, Code Review, Static Code Analysis, Performance Analysis 2,002 Call Manufacturer for details 2011
FlowLevel Java testing, Stress Testing, Performance Analysis, Protocol Testing, Server testing 2,004 Active
LoadTracer Load Capabilities, Stress Testing, Performance Analysis, Web testing 2,001 Call Manufacturer for details Active
Loadzen Stress Testing, Network testing, Performance Analysis, Server testing, Web testing 2,012 Free use: upto 25 virtual users Active
AgileLoad Load Capabilities, Memory Analysis, Mobile Testing, Performance Analysis, Web testing 2,006 May 2015: 50 Virtual users - $24 / day, May 2015: Free License - upto 10 virtual users Active
Commercial Tools
SmarteLoad Load Capabilities, Stress Testing, Performance Analysis, Web testing 2,007 Call Manufacturer for details Active
jClarity Illuminate Performance Analysis 2,014 April 2015: 14-day free trial, April 2015: Basic - $24 Monthly / Per Server*, April 2015: Pro - $119 Monthly / Per Server*, April 2015: Enterprise - Contact Manufacturer for details Active
OpenWebLoad Performance Analysis, Web testing 2,001 Free use 2001
AppAssist Mobile Testing, Performance Analysis 2,014 Active
PurpleStrike RT Load Capabilities, Performance Analysis 2,014 Active
JAR Load Load Capabilities, Back-end Stress, Stress Testing, Performance Analysis, Server testing, Uptime Testing, Web testing 2,010 Active
LoadFocus API testing, Load Capabilities, Performance Analysis, Website Speed Testing, Web Monitoring 2,013 Not Published Active

MANUFACTURERS Wallboard

View MANUFACTURERS Wallboard

Total Manufacturers in our archive: 280
Testing tool manufacturers world-wide list
10Levels ABID CONSULTING AccelQ Accord Software ActiMind AdaCore
AdaLog AgileLoad AgileWay Agitar Algorismi ALL4TEC
Andreas Kleffel Android Apache Apica Apollo Systems AppAssist.mobi
Applitools AppPerfect Appsee ApTest Assertible Assure
Atlassian AutoIt Consulti .. Automation Anyw .. Automation Cons .. Axosoft Aztaz Software
Backtrace I/O Badboy BlazeMeter Borvid BrowserStack BSQUARE
BStriker Intern .. CA Technologies Canonical Canoo Engineeri .. Catch Software CelestialTeapot
Chris Mallett Cleanscape ClicTest CloudQA Codeborne CodeCentrix
CodePlex projec .. Codoid Cogitek Compuware Configure IT Conflair
ConSol Core Services Coronys Ltd Countersoft CresTech Softwa .. CrossBrowserTes ..
Crosscheck Netw .. Crowdsourced Te .. Cucumber Ltd Cyara Cygnet Infotech DareBoost
Databene Datamatics Glob .. DevExpress DTM soft Dynatrace LLC EasyQA
Eclipse EkaTechserv Elvior Emmanuel Jorge Empirix EPAM Systems
Equafy Esterel Technol .. eXept Software .. Experitest Finaris Froglogic
FrontEndART Ltd GeneXus GitHub project gnoso Google Code Pro .. GrammaTech
Gurock Software HelpSystems HENIX Hewlett Packard .. Hexawise High-Tech Bridg ..
Hiptest Hitex IBM Rational imbus Shanghai Impetus Inflectra
informUp InTENSO - IT Ex .. Ipswitch Jamo Solutions Janova JAR Technologie ..
JBoss Developer jClarity Jellly.io JetBrains Jively jQuery foundati ..
JS Foundation Jspresso Kanoah KMS Technology Kualitee LDRA Limited
Litmus LoadFocus Loadster Perfor .. MarathonITE Marketcircle Marketcircle
Maveryx Meliora Ltd Micro Focus Sof .. Microsoft Mobile Labs Mobile1st
Mockaroo, LLC Monkop Mozila MSys Technologi .. Navicat NeoTys
Neowise Softwar .. NetCart NORIZZK.COM Novosync Mobili .. NRG Global NTT Resonant
OC Systems Odin Technology OpCord Oracle Orcanos Original Softwa ..
OW2 PANAYA Parasoft PassMark Patterson Consu .. Perfecto Mobile
Pivotal, Inc. Plutora Postman (API To .. PractiTest PrimaTest Process One
Programming Res .. Psoda PureLoad PushToTest Python Q-Assurance
QA Systems QACube QASymphony QAWorks QMetry Quali
Qualitia Softwa .. Quality First S .. Quotium RadView Softwar .. Ranorex RedLine13
Reflective Solu .. ReQtest RevDeBug Robotium Tech Rogue Wave Soft .. Rommana Softwar ..
RTTS ruby-doc.org Runscope Sandklef GNU La .. Sauce Labs Seapine Softwar ..
SeleniumHQ Sencha Sensiple Siemens PLM Sof .. SmartBear Softw .. SmarteSoft
SOASTA SoftLogica Softomotive Softsmith Solution-Soft SonarSource
Sourceforge Spirent Technol .. SQS Software Qu .. Square Stimulus Techno .. Swifting AB
Synopsys T-komp T-Plan TechExcel TechTalk Telerik By Prog ..
Tellurium Test Collab Test Goat Test Recon TestCaseLab testCloud.de Gm ..
TestCraft Techn .. Testenium Testim.io TestingBot TestLodge Testmunk
Testomato TestOptimal TestPlant TestPro Testsigma Techn .. Testuff
The Core Bankin .. The MathWorks The Open Group thePHP.cc Thoughtbot Thoughtworks
Tigris.org Time Simulator Top-Q Trace Technolog .. TrendIC TRICENTIS
Tritusa Pty Ltd TWD Solutions P .. TypeMock Tyto Software Ubertesters UniTESK
Universal Test .. Usetrace Ltd Utrecht Univers .. Validata Group Vanamco AG Vector Software
Veracode Verifaya Corpor .. Verit VersionOne Viewlike.us Vornex Inc.
Watir.com WcfStorm Soluti .. We Are Mammoth Web Performance .. Wintask Wireshark Found ..
Worksoft Xceptance XK72 Xpand IT XQual ZAPTEST
Zeenyx Software .. Zephyr Zeta Software zutubi pty

JMeter on YouTube

Theme by Danetsoft and Danang Probo Sayekti