Beanshell vs. JSR223 vs. Java For JMeter: Complete Showdown

Beanshell. JSR223. Java request sampler. Which is best for JMeter scripting? And to extend baseline JMeter functionality? In this blog, you'll compare performance and analyze which one is the best. 

First, you might be familiar with Beanshell and Java... but what about JSR223?

What Is JSR223?

JSR223 is a scripting API that you can use for JVM languages to debug JMeter test scripts.

JSR223 vs. Beanshell vs. Java Request Sampler

In another post about JMeter memory profiling, we recommended using JSR223 + Groovy for scripting. It is the best option as Groovy scripts can be compiled into native Java code. Therefore, Groovy script execution performance can be almost as fast as Java code. 

So if you are about to use scripts for something once, quick 'n dirty (e.g. reading configuration files at the beginning of the test) you’re welcome to use Beanshell/Javascript/whatever you’re comfortable with.

But, if you are about to do some extensive load testing through scripting (i.e. constructing massive HTTP requests from calculated data) you need to consider either Groovy, custom Java requests, or developing your own JMeter plugin.

For comparison purposes, we are going to use the same simple code which performs AES encryption of the given text followed by RSA encryption of the AES secret key. Both are quite resource-intensive procedures, hence it will be quite easy to determine the most performing option. 

For comparing the performance of the different scripting engines, we are going to:

1. Run the test with 1 thread (virtual user) for 1 minute.
2. Only one machine (JMeter Console) will be used for the test.
3. All the engines will be executing the same code doing cryptographic operations.

The idea is that JMeter should execute the encryption and decryption as fast as it can so the most performing engine will be able to do more cycles.

The assessed scripting engines are:

• Beanshell (as is)
• JSR223 (Groovy as language, compilation caching enabled)
• Java (as the JMeter Java Request Sampler)

These engines also include associated the CPU/RAM cost on the load generator side (BlazeMeter Console). Tests will be executed using JMeter 5.4.3.

Beanshell Sampler

There are two Beanshell Samplers, one that does encryption and another that decrypts the encrypted text. 

Beanshell - Encrypt Sampler Code:

import javax.crypto.Cipher;
import javax.crypto.KeyGenerator;
import javax.crypto.SecretKey;
import java.security.KeyFactory;
import java.security.PublicKey;
import java.security.spec.X509EncodedKeySpec;
import java.util.Base64;

KeyGenerator keyGen = KeyGenerator.getInstance("AES");
keyGen.init(256);
SecretKey secretKey = keyGen.generateKey();

Cipher aesCipher = Cipher.getInstance("AES");
aesCipher.init(Cipher.ENCRYPT_MODE, secretKey);
byte[] encryptedData = aesCipher.doFinal(Parameters.getBytes());

X509EncodedKeySpec x509EncodedKeySpec = new X509EncodedKeySpec(Base64.getDecoder().decode(vars.get("publicKey")));
KeyFactory keyFactory = KeyFactory.getInstance("RSA");
PublicKey publicKey = keyFactory.generatePublic(x509EncodedKeySpec);

Cipher cipher = Cipher.getInstance("RSA/ECB/PKCS1Padding");
cipher.init(Cipher.PUBLIC_KEY, publicKey);
byte[] encryptedKey = cipher.doFinal(secretKey.getEncoded());

String key = Base64.getEncoder().encodeToString(encryptedKey);
String data = Base64.getEncoder().encodeToString(encryptedData);
vars.put("encryptedKey", key);
vars.put("encryptedData", data);

SampleResult.setResponseData("Encrypted key: " + key + "\nEncrypted data: " + data)

Beanshell - Decrypt Sampler Code:

import javax.crypto.Cipher;
import javax.crypto.SecretKey;
import javax.crypto.spec.SecretKeySpec;
import java.security.KeyFactory;
import java.security.PrivateKey;
import java.security.spec.PKCS8EncodedKeySpec;
import java.util.Base64;

byte[] encryptedKey = Base64.getDecoder().decode(vars.get("encryptedKey"));
byte[] encryptedData = Base64.getDecoder().decode(vars.get("encryptedData"));

PKCS8EncodedKeySpec pkcs8EncodedKeySpec = new PKCS8EncodedKeySpec(Base64.getDecoder().decode(vars.get("privateKey")));
KeyFactory keyFactory = KeyFactory.getInstance("RSA");
PrivateKey privateKey = keyFactory.generatePrivate(pkcs8EncodedKeySpec);

Cipher cipher = Cipher.getInstance("RSA/ECB/PKCS1Padding");
cipher.init(Cipher.PRIVATE_KEY, privateKey);
byte[] decryptedKey = cipher.doFinal(encryptedKey);

SecretKey originalKey = new SecretKeySpec(decryptedKey, 0, decryptedKey.length, "AES");
Cipher aesCipher = Cipher.getInstance("AES");
aesCipher.init(Cipher.DECRYPT_MODE, originalKey);
byte[] decryptedData = aesCipher.doFinal(encryptedData);
vars.put("decryptedData", new String(decryptedData));

SampleResult.setResponseData("Decrypted data: " + vars.get("decryptedData"));

How Beanshell Performed

Beanshell Load Test Results 

Beanshell Engine Health

JSR233 Sampler

Configuration

As mentioned, it is recommended to use JSR233 + Groovy for scripting. Since the JMeter 3.1 Groovy engine is already shipped with JMeter, you do not need to follow any extra steps.

It is important to note the following:

• If possible, use .groovy files instead of keeping the Groovy code inside the sampler.
• Do not refer to any variables as ${VAR} inside the Groovy script. Use either vars.get(“VAR”) or the Parameters stanza of JSR233 Sampler. The same applies to JMeter Functions.
• Do not untick the “Cache compiled script” box.

Apart from these points, there are no differences with the Beanshell samplers. The code is the same. 

How JSR223 Performed

JSR223 Load Test Results:

JSR223 Engine Health:

Java Request

Configuration

Java Request is your own implementation of the JavaSamplerClient, and all the methods should have the appropriate code. The absolute minimum is override of runTest() method. However, if you intend to parameterize your Java Request, you need to provide appropriate logic to read inputs and conditional interpretation of requests flow to determine whether Sampler passed or not.

The compiled class needs to be placed in /lib/ext folder of your JMeter installation. Preferably, place it in .jar form so JMeter can automatically pick it up. Otherwise, you’ll need to amend the JMeter classpath.

When using BlazeMeter just upload the .jar file altogether with your script and other extensions (if any) and the BlazeMeter engine will pick it up.

Example implementations for encryption and decryption are available in Github. If you want to run the test on your own hardware - just compile the project using Maven by running mvn package command and copy the java-sampler-rsa-encryption-1.0-SNAPSHOT.jar to JMeter’s “lib/ext” folder. You should see two new classes for the Java Request sampler:

Also in the “jmeter” folder there are 3 .jmx scripts just in case. 

How the Java Request Sampler Performed  

Java Load Test Results:

Java Engine Health:

Bottom Line: JSR223, Beanshell, or Java Request Sampler?

Here is BlazeMeter’s Comparison Report for JSR223, Beanshell, and Java in a single chart displaying the Hits per second metric. 

Here are the test metrics in a consolidated table:

Assuming all the above, what do these results mean?

• Beanshell is OK to be used for once-only activities. These include reading configuration files somewhere in a single-threaded setUp Thread Group or in situations when no alternatives exist.
• JSR233 is quite a reasonable option for scripting, but only with the “caching” feature enabled.
• And the winner is... Java Request, which provides blazing performance and cutting-edge productivity!

BlazeMeter supports running performance tests with all of these options for JMeter scripting. Start running these tests with BlazeMeter today.

START TESTING NOW

Related Resources: 

• How to Use BeanShell: JMeter's Favorite Built-in Component