That's a Bug
João Farias

João Farias

Making educated guesses based on unreliable data provided by people who don't know what they want.

Assertions - JMeter Basics Part 2 of 4

Time to add more specific validations to the responses JMeter gets

JMeter Assertions

![JMeter]({{ “https://jmeter.apache.org/images/jmeter.png” | absolute_url }})

Following our JMeter series we are able to create different scenarios for testing multiple endpoints, using Thread Groups and Listeners, and avoid duplication of configuration using variables.

These tools allow us to evaluate the responses’ contents and the time they took to reach us back.

![Summary]({{ “assets/jmeter/post3/summary.png” | absolute_url }})

![View]({{ “assets/jmeter/post3/view.png” | absolute_url }})

However, unless the server give us some response different than 200 OK, we will never see a failure.

In situations of heavy load, all sort of problems can happen on the server:

  • Response’s content may be corrupted;
  • Specific response can take a long time to return, which is not acceptable for some products.

Additionally, we may want to investigate how the server reacts in situations different from 200 OK:

  • If a resource does not exist, does the system return a 404 response fast enough?
  • If a server error occurs (500 code), does the system continues to work properly?

To answer such questions, it is necessary to run additional validations to each request.

In JMeter, these validations are called Assertions.

Let’s take a look at some of the simplest, but useful JMeter Assertions.

Types of Assertions

JMeter offers a vast range of Assertions:

TypeUsage
Response AssertionApply a string pattern to verify against the server response
Duration AssertionCheck the response was received within a given elapsed time
Size AssertionCheck the size of the server response contains the wanted number of bytes
XML AssertionCheck the response is a valid XML document
Beanshell AssertionExecute your own logic using Beanshell scripting
MD5Hex AssertionAllows to check the MD5 hash of the response data (great for static files)
HTML AssertionCheck html response syntax using JTidy
XPath AssertionTests if a document is well-formed, with possible DTD validation, or putting the document through JTidy and testing with XPath
XML Shema AssertionValidate an XML response against an XML schema
JSR223 AssertionRun your own code logic using a JSR223 Script
Compare AssertionCompares results between themselves
SMIME AssertionEvaluate the sample results from the Mail Reader Sampler
JSON AssertionExecute JsonPath expressions and validate Json documents

For this post, we will focus on Response Assertion, Duration Assertion, and JSON Assertion.

Response Assertion

The Response Assertion performs content validations on the response, both on the data and meta-data.

Its goal is to detect bad responses generated during heavy server load and stress.

In the example below, we have defined a response validation for all Thread Groups (note the definition on the same level as the Thread Groups).

This Assertion will act (Apply to) to the first response (Main sample) and re-directs (sub-sample), if any.

The validation itself checks if the Response Code Equals to 200

![Response Assertion]({{ “assets/jmeter/post3/response1.png” | absolute_url }})

We can create as many All-Thread-Groups level Assertions as we want.

Below, we are checking if the Response Message Equals to OK.

![Response Assertion]({{ “assets/jmeter/post3/response2.png” | absolute_url }})

When we run again, all responses continue to be green, because they fullfil both criteria.

![Response Assertion Results]({{ “assets/jmeter/post3/response3.png” | absolute_url }})

Let’s change the first Assertion to an invalid value, 500 (Server Error code).

![Response Assertion 2]({{ “assets/jmeter/post3/response4.png” | absolute_url }})

Now we see the exact Assertion that failed and the diff between what was expected (comparison) and what we got (received).

![Response Assertion Results 2]({{ “assets/jmeter/post3/response5.png” | absolute_url }})

Duration Assertion

Duration Assertions investigate the timing the request.

Since this assertion is request specific, it only makes sense to have it under a Thread Group:

![Duration Assertion]({{ “assets/jmeter/post3/duration1.png” | absolute_url }})

![Duration Assertion]({{ “assets/jmeter/post3/duration2.png” | absolute_url }})

Duration Assertion allows to define the acceptable time threshold for the request, in our case, we defined 500ms for the Litecoin Orderbook endpoint and 1 second for the Litecoin Trades endpoint.

In case any request fails to meet this limit, JMeter will show a friendly error message:

![Duration Assertion Results]({{ “assets/jmeter/post3/duration3.png” | absolute_url }})

JSON Assertion

JSON is an extension of the Response Assertion. It aims to facilitate the navigation in JSON responses, in order to do simply integrity verifications of the data.

In the Assert JSON Path exists, we set a JSON path and JMeter will throw an error if this does not exist.

In our case, we want to check that the asks key is composed of a 2-dimensional array: That’s why we entered on the first element of the first element of the array mapped to asks.

Additionally, we added a regular expression to match the element found on the path. In our case, I want to be sure we are talking about a decimal value (at least on numeric value, optionally followed by a dot ., followed by zero or more numeric values).

![JSON Assertion]({{ “assets/jmeter/post3/json_1.png” | absolute_url }})

To demonstrate a failure, I’ve marked the “Invert assertion”, we raises a failure if the regex matches.

![JSON Assertion]({{ “assets/jmeter/post3/json_2.png” | absolute_url }})

With these three assertions, you can investigate further the responses you get with JMeter.

You can see all posts of this JMeter series here.

And if you want to try out the resulting suite, you can download it here.

How would you implement the questions we asked at the beginning of this post?

  • If a resource does not exist, does the system return a 404 response fast enough?
  • If a server error occurs (500 code), does the system continues to work properly?

Comments

Comments are powered by Giscus and use GitHub Discussions. Sign in with GitHub to leave a comment.