It has been our experience that, with agile practices on the rise and teams reporting on task progress through Jira, reporting on test progress and execution at a granular level has become less frequent. In fact, in some cases, some of our clients no longer use the reporting functionality of their test management tools. These past few weeks we have investigated this scenario, are test reports no longer important? Do QA’s not need these reports anymore because the necessary information is coming out of Jira? Are QEs just reporting from their automation frameworks and not end to end?
What?? Of course test reports are still important! And no, Jira does not provide all of the test information- jira is a project management tool, not a test management tool after all. But with test automation on the rise, for some reason some of our clients are forgoing end-to-end test reports and are just happy to understand whether everything passed, and if not, did we log a bug? Until there are incidents in production… At QoT we believe that reporting is still as important today as it was ten years ago before the rise of agile. So over the past couple of weeks we have assigned our staff the task of integrating their test automation frameworks into their test management tools, for automatic execution updates and so that we may report end-to-end on project execution and delivery. We want to, at the click of a button, be able to report to our clients on; automation progress, automation coverage, manual tests vs tests that are automated, end to end progress of the project, end to end status of the project, number of defects found, number of defects not closed etc etc.
We have successfully, and with very little blood, sweat and tears, managed to integrate Test Rail with Robot, Test Rail with Serenity, and Qase and Postman over these past few weeks. This integration has allowed us to use the features of these test management tools to automatically create reports that capture the project progress and execution from start to finish. Just like in the good ol’ days of QTP and QC. This setup has helped us to create greater visibility AND it has also made a lot of our lives easier. We are able to set-up and pull all of the metrics we alluded to above and at a fraction of the time.
Now, all we need to do is roll it out to all of our clients. So, first test management tools, next..the world! One test case at a time…
