I've tested some web applications whose unexpected errors or exceptions were handled by the application code and encoded in the response body as JSON.
A hypothetical response is shown below:
{ data: null, message: "TimeoutException: Transaction rolled back after 3000 seconds.", success: false }
The problem with this approach is that most of the times the HTTP response code is 200 (OK) which jmeter interprets as a successful sample.
I'm not a big fan of this solution I'd rather have it returning 4XX or 5XX response codes, in which case jmeter would fail the sample.
In order to workaround this, I normally use a BSF Assertion containing javascript code similar to:
try { eval('var response = ' + prev.getResponseDataAsString()); if (!response.success) { prev.setSuccessful(false); prev.setResponseMessage(response.message); } } catch(e) { prev.setSuccessful(false); prev.setResponseMessage("Invalid response. Expected a valid JSON."); }
First I try to evaluate the response string into a valid javascript object. If that succeeds I check the success flag and handle it properly, otherwise, in the catch block, I force the sample to fail due to a bad response format.
Note that in both failure scenarios I invoke the setSuccessful and the setResponseMessage methods to signal a sample failure and have better detailed error messages respectively.
And that's it!
Happy load testing!