Unit test xml report not Junit compatible

Sysprogs forums Forums VisualGDB Unit test xml report not Junit compatible

This topic contains 11 replies, has 3 voices, and was last updated by  Roman.Thiel 7 months, 3 weeks ago.

Viewing 12 posts - 1 through 12 (of 12 total)
  • Author
    Posts
  • #26732

    Markg
    Participant

    Hello

    We are integrating our unit tests into our jenkins environment. For the test results we generate the xml report. Is it possible to have this Junit compatible? This way it is easy integratable with jenkins.

    https://www.ibm.com/support/knowledgecenter/en/SSQ2R2_14.2.0/com.ibm.rsar.analysis.codereview.cobol.doc/topics/cac_useresults_junit.html

    Regards,

    Mark

    #26737

    support
    Keymaster

    Hi,

    Although VisualGDB cannot directly generate a Junit-compatible output, it can be configured to generate the outputs in the Visual Studio/MSTest test report format.

    You can do this via the following command line

    As both JUnit and MSTest test report formats are very popular, there are various open-source translators between them, so we would advise simply running one of them after VisualGDB returns.

    #26763

    Markg
    Participant

    Do you have a suggestion on which tool to use? It does not seem to work with the general plugins of jenkins (xunit / MStest

    #26777

    support
    Keymaster

    Sorry, we have not explicitly tested VisualGDB with Jenkins yet. Producing the .trx file with the /vsoutput option and then using the regular MSTest plugin should normally work. If not, please share the details and we will try to help you configure it.

    #26815

    Markg
    Participant

    I’ve ran the unit test in jenkins, I made it fail intentionally. You can find theĀ  results file below. This results file Is given to MS test plugin and it does not detect test results.

     

    .trx file content:

    • This reply was modified 9 months, 1 week ago by  Markg.
    • This reply was modified 9 months, 1 week ago by  Markg.
    • This reply was modified 9 months, 1 week ago by  Markg.
    • This reply was modified 9 months, 1 week ago by  support. Reason: formatting
    #26822

    support
    Keymaster

    Thanks for checking this. It looks like some minor structural differences between the .trx file generated by VisualGDB and the files expected by the Jenkins tools are interfering with the parsing.

    Please try the following steps to diagnose it:

    1. Create a dummy C# unit test with the MSTest framework. Run it with MSTest.exe and obtain a .trx file.
    2. Update your Jenkins setup to use a fixed .trx file generated in step 1 instead of the one produced by VisualGDB.
    3. Check if the test results are imported correctly. If not, double-check the Jenkins setup.
    4. If the .trx file produced by MSTest works, but the one produced by VisualGDB doesn’t, try eliminating the differences between them. E.g. change the test names in one of the .trx files to match the other one and then try adding the elements present in another .trx file until Jenkins starts recognizing it.

    Once you determine what part of the .trx file is interfering with Jenkins, please let us know and we will update VisualGDB to generate it in a way expected by the Jenkins tools.

    #26834

    Markg
    Participant

    I’ve looked up a random trx file generated by mstest.

    That one works fine with the mstest plugin (on jenkins)

    if i remove the following line the file stops working. So guess it has something to do with that: (it is missing in the visual gdb generated file)

    <TestMethod codeBase=”c:/dev/pickles-results-harne…

    https://github.com/x97mdr/pickles/blob/master/src/Pickles/Pickles.Test/results-example-mstest.trx

    I’ve made it a bit smaller so it is easier to compare it to the one generated by visual gdb (the one posted above) :

    • This reply was modified 9 months, 1 week ago by  Markg.
    • This reply was modified 9 months, 1 week ago by  Markg.
    • This reply was modified 9 months, 1 week ago by  Markg.
    • This reply was modified 9 months, 1 week ago by  support. Reason: formatting
    #26838

    Markg
    Participant

    I have applied the fixes manually to the visual gdb created file and now it works! So if you do a diff on the two files posted below you should be able to generate files which can be used in jenkins.

    Original file:

    Fixed file:

    • This reply was modified 9 months, 1 week ago by  support. Reason: formatting
    #26843

    support
    Keymaster

    Thanks for finding this out. We have updated VisualGDB to create the TestMethod element as well (VisualGDB will only set the “name” attribute, as others are not relevant to the VisualGDB test logic). Please try this build: VisualGDB-5.5.2.3414.msi

    If it still doesn’t work, please try manually adding other attributes present in the regular .trx file (e.g. codeBase, adapterTypeName) and let us know which combination of them fixes the problem, so that we could update VisualGDB to set them to some dummy values.

    #26846

    Markg
    Participant

    Thanks for the update and quick fix! Just tested it and it works fine now. I will keep you up to date when we find any issues when we start using it for our production code.

    #26851

    support
    Keymaster

    No problem. Let us know if you run into any issues and we will be happy to resolve them.

    #27267

    Roman.Thiel
    Participant

    Thanks for the Update also from my side. I am now able to use “Publish MS Test result report” in Jenkins.

Viewing 12 posts - 1 through 12 (of 12 total)

You must be logged in to reply to this topic.