Zowe Mainframe Batch Testing With BlazeMeter

Zowe mainframe has been an important development. Why?

Mainframe testing has historically been difficult to automate. With access to the mainframe typically being locked down from outside communication and with so much focus put on security, it’s understandable that much of mainframe testing remains manual.

Some advancements, like the JMeter RTE Plugin, have been introduced. These plugins let users automate their GUI mainframe tests using JMeter and BlazeMeter.

In this article, I’m going to focus on another aspect: mainframe batch testing, and how it can be accomplished by integrating open source Zowe with JMeter, and then run in BlazeMeter using Taurus.

What Is the Zowe Mainframe?

The Zowe mainframe is an open source framework for z/OS (the operating system for IBM z/Architecture mainframes). Zowe is part of the Open Mainframe Project. 

Zowe lets you communicate with the mainframe via CLI commands, similar to how you might work with Google Cloud Platform using the gcoud CLI, or via SDKs available for Javascript and Python.

There’s a wide range of mainframe functionality that’s exposed using Zowe. In this article we’re only going to be scratching the surface of what’s possible to demonstrate when running mainframe batch test cases, i.e testing operations done in batches instead of one at a time. We'll be using JMeter and BlazeMeter to simplify automating those tests as part of a CI/CD pipeline. 

It should be noted that in order to follow along with the rest of this article you’ll need to have access to a mainframe that’s running the Zowe API Mediation Layer to handle the commands we’ll be sending, you will also need the Zowe CLI tool installed and a Zowe zosmf-profile defined pointing to your mainframe system.

How to Test the Zowe Mainframe

There are three main phases for testing on the Zowe mainframe:

1. Set Up Your Script
2. Use the Zowe Command Listing
3. Automate Tests With BlazeMeter

1. Set Up Your Script

Let’s create a JMeter script. Our JMX is going to be very simple, it will only require a few components. When we’re done, you will have a JMeter script with a test structure that looks the following: 

We’ll work from the inside out. 

• Add a thread group to your test.
•  Under that add a JSR223 Sampler.
• Copy the following Groovy code into the JSR223 Sampler element:

//	Prepare to construct command from JMeter Variable

def sout = new StringBuilder(), serr = new StringBuilder()
def command = "no command given"

// 	Run command using the oppropriate OS format

if (System.properties['os.name'].contains("Windows")){
	command = ("cmd /c " + vars.get("command")).execute()
	log.info("Found Windows")
}
else {
	command = vars.get("command").execute()
	log.info("Found Not Windows")
}

//	Let command finish or timeout

command.consumeProcessOutput(sout, serr)
command.waitForOrKill(Long.valueOf(vars.get("zowe_timeout")))

//	Log the output and errors

log.info("\n\nout>\n$sout\n\nerr>\n$serr")

//	Set the sampler and response data with command and output

SampleResult.setSamplerData("Command sent to MainFrame: " + vars.get("command"))
SampleResult.setResponseData(sout.toString())

You can check out the comments in the code to see what each line is doing, but in essence, we’re taking a JMeter variable called “command” and running it on the operating system level.

We will be feeding this sampler commands via this variable. You’ll also notice the “zowe_timeout” variable used here, go to the Test Plan and set a user defined variable there called “zowe_timeout” and give it a value of ${__P(zowe_timeout, 60000)}. This will read from a JMeter property called zowe_timeout, but default to 60 seconds if not defined.

Technically, this is all you need. If you supply a Zowe command and your Zowe CLI installation has a zosmf-profile defined, this sampler will run and return the results which you can check by adding a View Results Tree to your test.

But let’s put in a little more elbow grease and make this test more flexible by reading our commands from a CSV file. 

2. Use the Zowe Command Listing

Add a CSV Dataset Config

On the same level as our JSR223 Sampler, let’s add a CSV Dataset Config and point it to a file called zowe-commands.csv. In our case we want the test to stop when there are no more entries in the CSV. So, we have to set a couple options here to make that happen. Specifically, note the settings for Recycle on EOF (End Of File) and Stop thread on EOF.

You may also have noticed that we’re getting more than our commands from the CSV - we also have a variable for “label” and one for “assertion”. I’m going to use “label” to set the name of our sampler (so that our reporting makes sense) and I’m going to use “assertion” to carry a string that I want to be present in the output.

Next, you'll:

• Name the JSR223 Sampler as ${label} (this is the syntax for JMeter variables).
• Add a Response Assertion to the sampler, and under “Patterns to Test” use ${assertion}. 

Create Zowe Commands CSV

Now, create the zowe-commands.csv file in the same directory as our JMeter script with the following format:

< zowe cli command >,< sampler label >,< assertion string >

< zowe cli command >,< sampler label >,< assertion string >

< zowe cli command >,< sampler label >,< assertion string >

…

Our test will populate the Zowe commands, and match the label and assertion that we want. A simple example line to list files based on your High Level Qualifier would look like this:

zowe zos-files list data-set "< HLQ >.*",List Files,< HLQ >.ISPF.ISPPROF

zowe zos-files list data-set "< HLQ >.*" is the command to list all datasets that start with the HLQ (replace with your own to try this). “List Files” will be used as the label or name of our sampler, and “< HLQ >.ISPF.ISPPROF” will have to show up in the output otherwise our sampler will fail. It all fits together nicely at this point and we’re almost done!

Use a While Controller

One more thing we want to be able to do is loop through our CSV with a single user. So take what we’ve done so far, the CSV Dataset Config and the JSR223 Sampler and wrap them in a While Controller (you’ll find that under Logic Controllers).

In the condition, just put in one of the variables from your CSV file, ${label} as an example. This way the elements that are under the While Controller will execute as long as the variable ${label} exists.

3. Automate Tests With BlazeMeter

So, now we’ve got a working script that is interacting with our Zowe mainframe, we want to get that test on BlazeMeter. In BlazeMeter, we can easily automate this test with an API call or use a tool like the Jenkins Plugin to automatically execute this test.

Deploy a BlazeMeter Private Location

First, we’ll want to make sure we have deployed a BlazeMeter Private Location that has access to our mainframe. Make sure that the Private Location is set up to use API Functional and Performance functionalities since we can run this test as both of those.

Configure the Test

In the Zowe mainframe test configuration, we’ll want to bring our JMeter script and the CSV file, but we’ll also need to include a YAML config from open source Taurus that will reference the JMX. The YAML will also let us install the Zowe CLI and create a zowe zosmf-profile as before the test starts.

Create the following Taurus YAML config in a text editor:

execution:
- executor: jmeter
  scenario: zowe-jmeter
  locations:
    harbor-: 1    # Fill in Private Location ID that can reach the MainFrame

scenarios:
  zowe-jmeter:
    script: MainframeBatchTest.jmx

services:
- module: shellexec
  prepare:
  - npm install -g @zowe/cli
  - zowe profiles create zosmf-profile default --host  --port  --user  --password  --reject-unauthorized false
  
#   ^^ for profile:
#   fill in details for host, port, username, password

With the profile command properly configured with the user, password, host, and port information in the above YAML, BlazeMeter will run this test, installing Zowe CLI, creating the required profile, and looping through all of your Zowe CLI commands in JMeter. You will get reporting based on the labels and assertions you stated in your Zowe Commands CSV file.

If our Zowe commands are using the --vasc (View All Spool Content) option, each of our commands will come back with the spool content from the mainframe like the following:

I’m running this test as a Functional Test on BlazeMeter in order to see the data for individual labels, but the same test configuration would work as a Performance Test as well.

Bottom Line

With our Zowe maintenance test saved on BlazeMeter, it inherits all the benefits pertaining to automation. As an example, we can kick this test off with a call to the /tests/{testID}/start BlazeMeter API endpoint, or we could point the Jenkins BlazeMeter Plugin to the test ID to easily automate running this test. 

If we wanted to take this a step further, before we make the call to start this test, we can use the /tests/{testID}/files BlazeMeter API endpoint to replace the zowe-commands.csv with a new set of commands, labels, and assertions effectively letting you pass in your commands at run time. The way we’ve written this test really gives us a lot of flexibility when we’re thinking about automating our test cases.

That's it! You are now ready to start your Zowe mainframe batch testing with BlazeMeter!

START TESTING NOW

Related Resources: 

•  Related Resource: Learn more about How to Test Mainframe LPARs