Wednesday, January 22, 2014

Jenkins Series 1 - Create Trends for Existing Results - Using JUnit Publisher

After taking a long break on Non Information Server blogging thought of sharing my recent exposure to Dev Ops Model. 
Today I will share my experience how we can use Jenkins as a Dashboard to show trends of nightly build tests. You do not require to use Jenkins for executing these tests. Can simply use XML report files generated from your nightly build verification system  and produce these trends live instead of relying on Spread Sheets and their tabs.

Here are few steps need to use Jenkins to produce Trends on Dashboard when have test cases executed are based on JUnit framework or results produced are XML Reports adhering to JUnit Framework.

Step 1: Copy Report produced by your Build Verification Tests or Unit Tests into the Slave or machine where you want to run the process. You can even use Shared Folder for the same

Step 2: For Windows use Execute Windows Shell and for Unix use Shell

Step 3: With-in Windows or Unix see Image Below to copy the report file into workspace. Can use Custom workspace if do not want to copy these files. Can also use to write some script which can executed based on your build number can be supplied as Parameter. Below step can be used to copy any other files as well not required for reporting.
Copy XML Report Files to Workspace
Step 4: Use Publish JUnit Test Report to show these results on Jenkins dashboard for selected build. Jenkins plugin will parse the files and produce results on the dashboard.
Publish JUnit Test Result Reports to Jenkins Dashboard
Step 5: Save the Jenkins job

Step 6: Execute the Build with Build Now
Jenkins will process these XML files and produce a Trend Graph. Since it is only for 1 Build should get failure and success details. Repeat the process for different builds and can see trends like this below.
Chart showing Trends for different build on Dashboard


Now above trend you see shows red graph going upwards means have failure across builds reducing the confidence in the build. You can write the job once and can be triggered at different intervals or by external trigger. Can navigate the graph and see failures component wide. Will cover these details in Next Blogs on how to identify failures and see if any regressions or new failures and Fixed from previous build.


-Ritesh 
Disclaimer: The postings on this site are my own and don't necessarily represent IBM's positions, strategies or opinions

4 comments:

  1. Currently, Test result trend graph shows data in terms of testcase count vs. build number.

    Is there any way, Jenkins can generate test result history graph in terms of testcase count vs. build date

    ReplyDelete
  2. Pretty good post. I just came across your site and wanted to say that I’ve really enjoyed reading your posts. In any case I’ll be subscribing to your feed and I hope you will keep a good work!Cheer!

    sap online training
    software online training
    sap sd online training
    hadoop online training
    sap-crm-online-training

    ReplyDelete
  3. This is one awesome blog article. Much thanks again.
    I really enjoy the blog.Much thanks again. Really Great.


    oracle online training
    sap fico online training
    dotnet online training
    qa-qtp-software-testing-training-tutorial

    ReplyDelete
  4. hi Ritesh,

    can you please suggest how to generate a tests graph from .txt files or just from jenkins console log

    ReplyDelete