Data Export From MongoDB

Performance testing requires solid data preparation. One can take a snapshot of system data, but there is no guarantee that this data will stay valid in the future

For example, when a performance test needs to check the response code of a rest api for a large amount of requests, it is a good practice to gather the data behind the API just before running the test.
MongoDB comes with a toolset to perform such things.
To export data from a mongodb, there is the mongoexport tool

In a multi environment situation (multiple stages), it is likely that the same performance test will be run against different environments.

The following bash script can take care of exporting data per environment

When this script is executed with the <environment> argument, the mongo data is exported for that environment

The result is a csv file that can be used in a JMeter project.

In this way, it is most likely that you request existing data through the API under load.

Installation on Debian Based Distros

Robotframework can be installed via pip as well as most of its libraries
Some libraries are not indexed in pip, but can be manually downloaded from GitHub
Such as https://github.com/iPlantCollaborativeOpenSource/Robotframework-MongoDB-Library
For external libraries, please follow the github project install instructions.

To install pip, securely download get-pip.py

Then install pip as root

Or you can use the apt indexed one

to list the installed packages

Packages should reside under :

Once you have pip installed, you can install the core robotframework as root

For the RIDE IDE :

For libraries the same basic installation procedures apply

For the DatabaseLibrary module pymysql is required

To upgrade packages a handy tool can be installed

With pip-review you can list upgradable candidates

you can upgrade these packages in an interactive way

When package upgrade would fail with pip, you can try the older easy_install tool.

Github Libraries of interest are :

MongoDB : https://github.com/iPlantCollaborativeOpenSource/Robotframework-MongoDB-Library

 

The Robot Framework

Paranoid_android_with_shadow-mirroredRobot Framework is a generic test automation framework for acceptance testing and acceptance test-driven development (ATDD). It has easy-to-use tabular test data syntax and it utilizes the keyword-driven testing approach. Its testing capabilities can be extended by test libraries implemented either with Python or Java, and users can create new higher-level keywords from existing ones using the same syntax that is used for creating test cases.

Robot Framework project is hosted on GitHub where you can find further documentation, source code, and issue tracker. Downloads are hosted at PyPI. The framework has a rich ecosystem around it consisting of various generic test libraries and tools that are developed as separate projects.

Robot Framework is operating system and application independent. The core framework is implemented using Python and runs also on Jython (JVM) and IronPython (.NET).

Performance Testing with detailed reporting in Jenkins

Installation

Jmeter can be downloaded here

Jmeter is a java application, so java is a prerequisite

Extract the tgz file in a directory of choice

Jmeter can be launched as follows:

<installDirectory>/apache-jmeter-2.13/bin/jmeter.sh

In our performance test scenarios we use some handy plugins which can be downloaded :

Move the zip files into the installed JMeter directory

Unzip them.

When the system asks to overwrite files, just accept.

Test Case Organisation

 

Generic components

File
Function
url-benchmark.jmx The JMeter project file
generateGraphs.sh Shell script to generate performance graphs
generateHTML.sh Shell script to generate a HTML report containing the performance graphs
dev Property file for the dev environment
test Property file for the test environment
stage Property file for the stage environment

The JMeter can take in parameters from the property files

Jmeter with property file

Jmeter From The Command Line

Jmeter can be executed from command line which makes it perfectly useful for Jenkins integration

Specific parameters that are not related to the environment where the test is running, can be specified by separate command line arguments

Following command gives an idea how this is done :

 

Argument
Function
-n non GUI mode
-t <JMeter project file> Specifies the JMeter project file
-l <report file> Specifies where to save the JMeter report file
-p <property file> Specifies the property file used by JMeter
-J<parameter> Specifies a parameter that is defined inside the JMeter project
JMeter system parameter to force the report file format. We need XML to generate graphs from

Command Line Parameters

Now let’s have a look on how command line parameters are taken in by JMeter :

globalVariables.png

Parameters are defined in a “User Defined Variables” node

Parameters that are taken in from outside have the following format

The corresponding internal variable for this example is RUN_TIME_IN_SECONDS

This can be used in any node of your project

image2015-3-25 10:1:11

Jenkins Integration : An Example

Job Parameterisation

Jenkins jobs can be parameterized. This is handy for environment selection

image2015-3-25 10:9:0

This variables can be selected when launching the job in Jenkins

image2015-3-25 10:10:53

Script Execution Details

Now let’s have a look to the shell script execution of the job

 

  1. First we remove all artifacts from a previous test
  2. exportResRegData.sh is not in scope of this document. It is a shell script that exports data from MongoDB. You can see that ${env} is taken from the parameters above

  3. The actual jmeter execution is explained above

  4. ./generateGraphs.sh is a shell script that takes care of graph generation. This will be explained in the following steps

  5. ./generateHTML.sh > report.html is a shell script that generates the final report file

Graph Generation

Online Documentation : http://jmeter-plugins.org/wiki/JMeterPluginsCMD/

generateGraph.sh

The script uses CMDRunner.jar as graph generation engine which comes from the extra plugins.

Argument
Function
–tool The tool used
–generate-png <file location> Generates a png file on the specified location
–input-jtl <file location> Specifies which jtl JMeter report file to process
–plugin-type Specifies which graph type needs to be generated

The rest is self-explanatory

Final Result

 

Triggering JMX tasks with robotframework

The robotframework does not have native support to communicate via JMX.

However there is a solution.

As Robot can run any command on a shell through the OperatingSystem Library, we can make use of a couple of jar files

Prerequisite is to have java installed on your system.

To execute JMX shell command the following jars need to be downloaded :

Both files need to be in your $PATH

Two Tcl scripts are needed too for correct operation : jmx-invoke.conf and jmx-attribute.conf

These can be found under the robotframework root directory

${ROBOTFRAMEWORK_HOME}/attributes/conf/

Run JMX tasks via robot

To run JMX tasks via robot, we make use JMXshell.jar and execute it via the OperatingSystem run command keyword.

 

${bindir} /usr/local/bin which is part of the linux $PATH
${artifacts} robotframework directory containing test artifacts
${bean} Bean name
${attribute} attribute of the JMX task
${host} Hostname of server to send message to
${port} Port to send message on
jmx-invoke.conf Tcl script that executes the actual invoke operation
jmx-attribute.conf Tcl script that executes the actual status retrieval operation
Argument
Function

Different Keywords For JMX Operations

Trigger JMX Operation

jmx-invoke.conf

Get JMX Job Status

jmx-attribute.conf

Poll A JMX operation status

Get A JMX operation status