WebSocket Testing With Apache JMeter

JMeter WebSocket Load Testing Sampler

At the moment the only easy way to implement WebSocket testing with JMeter is to use JMeter WebSocket Sampler by Maciej Zaleski. It's the only RFC6455-compliant  extension which supports reusing one TCP session and it is easy to install and use.

Installation

To install JMeter WebSocket Sampler go to the JMeter Plugins Manager. This is also the best way to keep the Sampler up to date.

If you need to install manually, do the following:

• Download the latest version of the extension from the Releases page. Drop the .jar to /lib/ext folder of your JMeter installation
• Check Project Wiki for any dependencies, to be completely sure all the jars from target/site/plugins directory should go either to /lib or to /lib/ext folder of your JMeter installation. Currently (for release 1.0.1) the required libraries live in Jetty Bundles 9.1.1.v20140108.
• If everything went well you should see something called “WebSocket Sampler” within the JMeter Samplers list.

Notes:

1. JMeter doesn't pick up extensions dynamically. A restart is required.
2. If you don't see the WebSocket Sampler double check that WebSocket-versionx.x.x..jar lives in /lib/ext folder and that Jetty jars are present in /lib or lib/ext folder.
3. If using Blazemeter – all the files should be provided along with the script.
4. If it still doesn't work – check your jmeter.log file, it usually contains all the answers and will be the first thing you'll be requested for when you ask for help.

Usage

This section explains the WebSocket Sampler GUI elements.

For version 1.0.1 the following components are available for the WebSocket Sampler:

• Server Name or IP – WebSocket endpoint (the host, where server-side WebSocket component lives)
• Port Number – the port that theWebSocker server listens to. Usually HTTP port 80
• Timeout: Connection – maximum time in milliseconds for setting up a connection. Sampler fails if exceeded. Response – same for response message
• Implementation – the only available is RFC6455(v13) – the latest version of the WebSocket protocol standard
• Protocol – WebSocket protocol to use: A ws prefix identifies the WebSocket connection. A wss prefix identifies the WebSocket Secure connection
• Streaming Connection – indicates whether the TCP session will remain. If checked – the connection will persist, if left unchecked – the connection will be closed after the first response
• Request Data – defines outgoing messages
• Response Pattern – Sampler will wait for a response to contain the pattern defined (or till response timeout occurs)
• Close Connection Pattern – basically the same as “Response Pattern” but the connection will be closed instead
• Message Backlog – identifies maximum length of response messages to keep

Other fields are self-explanatory.

Demo

For this demonstration we'll be using Echo Test WebSocket Demo which uses the ws://echo.websocket.org endpoint which basically responds to client with the same message it received.

Let’s assume the following Test Plan structure:

• Test Plan
• Thread Group ( 1 thread, 1 second ramp-up, 1 loop)
• Loop Controller (2 loops)
• WebSocket Sampler
• Host – echo.websocket.org
• Streaming connection – on
• Request message - “Ground control to Major Tom”
• WebSocket Sampler
• Host – echo.websocket.org
• Streaming connection – off
• Request message - “Take your protein pills and put your helmet on”

We're going to send 2 messages one by one through one session via Loop Controller followed by 1 message via another session.

The results show that the message “Ground control to Major Tom” went through the same WebSocket Session (red rectangles) and the message “Take your protein pills and put your helmet on” went through another session which was closed right after response.

The first session disconnected on the test run end (or if “Close Connection Patten” is provided and matches).

That’s how the WebSocket Sampler from Maciej Zaleski can be used for JMeter WebSocket testing, supported by Blazemeter as an extension. Any enhancements?  send pull request to GitHub.

Any questions? – feel free to shout.

Not enough? Write your own WebSocket Client Extension

Need a missing feature right away? Your WebSocket server-side tier uses a specific protocol or authentication? Want your own implementation or to override connect, message and disconnect event? We won’t give you a fish. We’ll teach you how to fish.

The JMeter WebSocket extension discussed above uses Jetty client libraries. Another implementation of WebSocket is JSR 356 (Java API for WebSocket).  We'll tell you about how to create a WebSocket extension based on a JMeter Java Request Sampler.

Java Request Samplers are classes which implement the JavaSamplerClient interface. All classes which have appropriate JavaSamplerClient methods implementation and existing in JMeter classpath will be automatically picked up and displayed within the JMeter – Sampler – Java Request drop down.

In this example, once again, we will use the echo.websocket.org endpoint which will send a “Blazemeter rocks” message and wait for the answer.

Prerequisites

First of all we'll need Tyrus – an open source JSR-356 implementation. Tyrus requires Java 7 so make sure that you use Java SE 7 to build and execute the extension.

The most convenient way of getting dependencies is using the following Apache Maven configuration file:

<?xml version="1.0" encoding="UTF-8"?>

xmlns="http://maven.apache.org/POM/4.0.0"

         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">

    4.0.0 

    blazemeter-websocket

    blazemeter-websocket

    1.0-SNAPSHOT

            javax.websocket

            javax.websocket-client-api

            1.0

            org.glassfish.tyrus

            tyrus-client

            1.1

            org.glassfish.tyrus

            tyrus-container-grizzly

            1.1

            javax.json

            javax.json-api

            1.0

            org.glassfish

            javax.json

            1.0.1

Invoking the target “mvn dependency:copy-dependencies” will download all required jars into /target/dependency folder. However you may wish to continue with build, package, etc. maven plugins.

The source code of the extension is follows:

package com.blazemeter; 

import org.apache.jmeter.config.Arguments;

import org.apache.jmeter.protocol.java.sampler.AbstractJavaSamplerClient;

import org.apache.jmeter.protocol.java.sampler.JavaSamplerContext;

import org.apache.jmeter.samplers.SampleResult;

import org.apache.jorphan.logging.LoggingManager;

import org.apache.log.Logger;

import org.glassfish.tyrus.client.ClientManager;

import javax.websocket.*;

import java.io.IOException;

import java.net.URI;

import java.util.concurrent.CountDownLatch;

import java.util.concurrent.TimeUnit;

@ClientEndpoint

public class BlazemeterWebsocketRequest extends AbstractJavaSamplerClient {

    private static String ws_uri;

    private static String ws_message;

    private static String response_message;

    private static CountDownLatch latch;

    private static final Logger log = LoggingManager.getLoggerForClass();

    @Override

    public Arguments getDefaultParameters() {

        Arguments params = new Arguments();

        params.addArgument("URI", "ws://echo.websocket.org");

        params.addArgument("Message", "Blazemeter rocks!");

        return params;

    }

    @Override

    public void setupTest(JavaSamplerContext context) {

        ws_uri = context.getParameter("URI");

        ws_message = context.getParameter("Message");

    }

    @Override

    public SampleResult runTest(JavaSamplerContext javaSamplerContext) {

        SampleResult rv = new SampleResult();

        rv.sampleStart();

        latch = new CountDownLatch(1);

        ClientManager client = ClientManager.createClient();

        try {

            client.connectToServer(BlazemeterWebsocketRequest.class, new URI(ws_uri));

            latch.await(1L, TimeUnit.SECONDS);

        } catch (Throwable e) {

            throw new RuntimeException(e);

        }

        rv.setSuccessful(true);

        rv.setResponseMessage(response_message);

        rv.setResponseCode("200");

        if (response_message != null) {

            rv.setResponseData(response_message.getBytes());

        }

        rv.sampleEnd();

        return rv;

    }

    @OnOpen

    public void onOpen(Session session) {

        log.info("Connected ... " + session.getId());

        try {

            session.getBasicRemote().sendText(ws_message);

        } catch (IOException e) {

            throw new RuntimeException(e);

        }

    }

    @OnMessage

    public String onMessage(String message, Session session) {

        log.info("Received ... " + message + " on session " + session.getId());

        response_message = message;

        try {

            session.close(new CloseReason(CloseReason.CloseCodes.NORMAL_CLOSURE,""));

        } catch (IOException e) {

            e.printStackTrace();

        }

        return response_message;

    }

    @OnClose

    public void onClose(Session session, CloseReason closeReason) {

        log.info(String.format("Session %s close because of %s", session.getId(), closeReason));

    }

}

The Sampler GUI may look minimalistic…

… but it is a fully functional WebSocket client

This is pretty enough for a proof-of-concept assessment, and you also have full code control and can, for example, change the protocol by playing with annotations like  @ClientEndpoint(subprotocols= { “xsCrossfile”})

The same stands for custom headers, cookies, or whatever logic you need on connection open, close and message events.

Happy WebSocket testing! Don't hesitate to post your questions and comments.

P.S. :

 You may find references to the https://github.com/kawasima/jmeter-websocket/ plugin across the Web and some evidence of successful WebSocket testing with  the plugin.

We aren’t saying it is a bad plugin but we do recommend using it with caution as it wasn't updated for 2 years and it relies on the 8.1.9.v20130131 Jetty libraries version which not fully RFC6455 compliant, but rather a kind of experimental draft. It also doesn't seem to support reusing one TCP session (this is how browsers work and the majority of test cases will be connected with sending lots of messages through one session). 

To learn more advanced JMeter and explore hundreds of free testing-related resources, check out BlazeMeter University!

START TESTING NOW

Related Resources:

• JMeter WebSocket Samplers - A Practical Guide