In this tutorial, you will learn-
- What is Jenkins?
- Steps to install the BlazeMeter plugin for Jenkins
- Steps to configure the BlazeMeter plugin for Jenkins
- Steps to configure a job in BlazeMeter for Jenkins plugin
- Steps to Set Up and Run the Build Test
- Steps to Run a BlazeMeter Test in Jenkins Without a Plugin
- Performance Trend Report
- Performance Report
- Comprehensive Reporting Dashboard
Jenkins is a Continuous Integration (CI) tool, written in Java. CI systems orchestrate software build, testing, and deployment processes, to ensure that changes developers make don't have a negative impact on other software parts.
On Jenkins, builds can be started in various ways. They can be triggered by a commit in a version control system, scheduled via a cron-like mechanism, built when other builds have completed, or built by the request of a specific build URL.
Steps to Install Jenkins
- Download Jenkins from the link (only download the .war file).
- Run as Java -jar jenkins.war after sometime you will get a message saying "Jenkins is fully up and running".
- Open http://localhost:8080.
- The Jenkins dashboard should be visible to you.
How to Install the JMeter Plugin
The performance plugin provides Jmeter support on Jenkins.
To install the plugin:
- Download from the Performance Plugin page.
- Copy performance.hpi file to the plugins folder. If Jenkins is running from the .war file, put the plugin in the .jenkins/plugins path under the user home folder.
- Select the plugin up by restarting Jenkins.
- Look at Jenkins -> Your Project -> Configure -> "Add post-build action" dropdown. If you can see "Publish Performance Test Result Report" – everything is fine.
How to Execute a JMeter Test in Jenkins
Executing JMeter test in Jenkins is fairly easy. Don't forget the test needs to execute in non-GUI mode.
We recommend you store JMeter results in XML format for the first few times. Here's how:
- Add jmeter.save.saveservice.output_format=xml to the user.properties file. (this is available in /bin folder of JMeter).
- Pass the property by the -J command-line argument, like: C:\jmeter\bin\jmeter.bat -Jjmeter.save.saveservice.output_format=xml -n -t C:\jmeter\extras\Test.jmx -l Test.jtl
If you prefer, you can use CSV formatted files, but they require extra configuration.
The Windows batch command for the JMeter test configuration:
Was the execution successful? Examine the "Console Output" tab.
Find Project's "Workspace" in the .jtl results file and in the JMeter log file.
Steps to use the Performance Plugin
The Performance Plugin presents statistics and trends. The builds status can be assigned as failed, based on results.
- Open the project.
- Click "Configure".
- Click "Add post-build action".
- If your results file:
- Is in XML format, choose "JMeter".
- Is in CSV format, choose "JMeterCSV".
The plugin scans for results files in the "Report Files" path.
After the first build with the plugin, you will see empty charts "Performance Trend".
These charts change as more builds are created. It will display Median, average and values with 90 percentile along with the errors count. To get more details on each sampler that is executed, click the charts.
How to Pass Jenkins Parameters to a JMeter Test
With Jenkins you can determine build parameters. For instance: checking "This build is parametrized" on the "Configure" page. This allows you to set parameter names with the default values:
Why is this important? Because it enables you to configure JMeter test without changing the .JMX script or Ant or Maven configuration.
For example, determining the parameter with the name "foo" and the value "bar", enables referring to it as:
- %foo% - if run on a Windows machine
- $foo - if executed on Unix, Mac OSx or Linux machine
Check out this example of accessing a set JMeter thread from a JMeter Windows batch command text:
If you execute the build one more time, and check the "Console Output". You will observe that the %threads% variable has been modified with the related parameter value:
What is the BlazeMeter Jenkins Plugin?
BlazeMeter users can use the plugin to connect to Jenkins and perform automatic script executions. Jenkins enhances BlazeMeter by enabling you to automate the testing process and simplifying testing your builds.
- On the Jenkins dashboard, select 'Manage Jenkins', 'Manage Plugins' and 'Available' (from the top tabs).
- Click 'Ctrl + F' / 'CMD +F' in your browser. Mark the BlazeMeter checkbox.
- Click 'Download now and install after restart'.
- The download will start. Restart Jenkins.
- After restarting Jenkins, click 'Manage Jenkins' and then 'Configure System'.
- Enter https://a.blazemeter.com/ in the BlazeMeter Cloud section.
- Click 'Save'.
- Click 'Manage Jenkins' (again).
- Click 'Manage Credentials'.
- Click 'Add Credentials' and then choose the BlazeMeter API Key.
- Go to the Description field and add a nickname.
- Enter the key in the API Key field.
- Click the 'Test BlazeMeter User Key' button.
- Create a new job or choose an existing one.
- Click 'Configure'.
- Click 'Add Build Step' in the 'Build' section and choose 'BlazeMeter'.
You should now see all your tests in the drop down menu 'BlazeMeter Test'.
- Choose a test to run.
- Save your settings.
- Click the 'Build Now' button.
- To see the build progress, choose 'Console Output'. You will be able to see the test parameters and updates every few seconds.
After the test finishes running, you will see a summary of the test results and their impact on the build.
- To see a test report, choose 'BlazeMeter Report'. You can also edit and compare test results to previous runs.
- To access the logs and data generated during the test runs, go to the report's 'Logs' tab. They are stored in a ZIP file.
You don't have to install any software in order to kick off a BlazeMeter test from Jenkins (or even without Jenkins). BlazeMeter provides REST API so you can create, update, start, stop tests, get reports and much more, using simple REST calls which can be triggered by any program which is capable of sending a HTTP request, i.e. cURL or equivalent.
See our site for all available API endpoints.
You need to provide a mandatory parameter - API Key - along with any API request, check out 'Find My API Key' page to find it out. Remember to keep the key in a safe place as it provides unlimited access to your account including any sensitive data.
Jenkins provides Out of the Box intelligence reports and visualization reports:
The Performance Trend Report displays gradual change of the success and failure of test results over time. The report is generated by fetching the JUnit.xml report after a test ends. This capability, rendered by the common Performance Jenkins plugin, is inherent to Jenkins.
The Performance report displays a basic KPI/Transactions report that visualize a JTL that is an artifact of the test run. The JTL file's raw data from the end of tests is what generates this report.
The Comprehensive Reporting Dashboard gives an in-depth view into the performance KPIs. It enables users to find performance bottlenecks. The BlazeMeter link displays the variety of reports available, during or after the test has ended.
Congratulations! You are now ready to use Jenkins and continuously reduce the risk of performance degradations.
This article is brought to you by BlazeMeter. BlazeMeter is the leader in continuous performance testing for the continuous delivery era. They ensure the delivery of high-performance software by enabling DevOps teams to quickly and easily run open-source based performance tests against any mobile app, website or API at massive scale to validate performance at every stage of software delivery.
For more information, visit the BlazeMeter blog or contact the BlazeMeter Support Team.