Collier has an extremely useful chapter devoted to the subject of Testing. Early on he makes two important assertions:
i) “Testing must be integrated into the development process”
ii) “Essential to integrated testing is test automation”
The IDT certainly helps with the first of those assertions but, in it’s current state, does nothing to help with the second. My contention is that there may be an option within the wider SAP BusinessObjects deployment which might help with this but I’ll get to that later. For now, let’s focus on integrating testing with development. To be fair, I think Collier accepts that the goal of automated testing becomes difficult when you are using ‘black box’ proprietary solutions like the data access layers which SAP Universes effectively provide. His chapter focuses principally on the testing principles for data warehouse schemas, user interfaces and the various third party solutions available to help with automating those tests but acknowledges that not all the components of a BI environment can currently be automated. If we buy that then I believe there is value in looking at where the SAP IDT can at least progress things a bit.
Collier also makes reference in his chapter on Testing to Test Driven Development (TDD). It’s a compelling argument that – and I’m simplifying it here - puts agreeing tests at the start of all agile developments and applying them continuously throughout. This is the integration that the first of the assertions I’ve quoted above refers to and it’s where I could see the IDT adding particular value through the use of the Business Layer Queries.
Let’s use the productive and collaborative developers I referenced in my last post – Tom and Huck. When we left them last time, Tom was developing the Business Layer of the IDT ‘exploded’ Universe model whilst Huck worked on adding a new data source to the Data Foundation. As he developed the Business Layer, Tom was able to create queries that tested his amendments and made sure the results were in line with the user expectations originally defined in their User Stories (Collier is really good and challenging on this topic e.g. he advocates potentially creating these test queries even before development). How does he do that? By using the query panel built into the Business Layer interface.
Business Layer Queries are stored with the Business Layer as it is shared in the repository and with other developers. It’s sadly not possible to automate them in this interface but it does allow a single point of storage for the small unit tests that are necessary for all artefacts in your BI environment. They’re built using a a query interface very familiar to anybody that’s used BusinessObjects before.
Once the Queries are created, they can be renamed and have a description added to them that, I’d suggest, be used to record expected results to. Once the Business Layer is saved and exported to the Repository the queries are then available to other developers to run, validating that any changes they have made do not affect the expected results elsewhere. It’s quite a handy query interface actually as, not only does it return the results, it also returns basic data profiling information about distinct values and their spread in the result set.
With the new queries Tom has created now shared, the next time Huck downloads the latest version of the Business Layer to his sandpit environment he’s able to see if the expected results have been affected by any of the changes he has made to the Data Foundation. Hopefully not, but if there’s an issue, he’ll have to go an amend the Data Foundation to resolve it.
Now eventually, without automation, these tests are clearly going to become obstructive to the rapid delivery of working applications Agile Analytics is intended to deliver. I’d hope though that by at least centralising them they will help with the unit testing of individual iterations of Universes. When it comes to testing that the latest ‘for production’ iteration does not reverse the achievements of previous iterations an automated process will clearly be required. Use of the Metadata Management capacity of the SAP Information Steward should help guide development as to where possible impacts might be but does not provide the automated solution.
A potential option for that will be using the capability of the SAP BusinessObjects reporting environment to create a chained schedule of reports, each of which will only run upon successful completion of the previous. The first report to be run would be Scheduled to, upon the successful issuing of an Event (e.g. report returning data), trigger the next report which in turn would have to issue an Event and so on. If, at any point in this automated run of test reports there was a failure it would be simple enough to identify which Event had failed and look at what went wrong for the issuing report.
In summary then, what I’m suggesting is that the IDT be used to define and run (un-automated) the Queries required within an Agile Analytics iteration but, that the SAP BusinessObjects Schedule and Event features then be used to automate these tests as reports within the wider end-to-end project test plan. We integrate tests to the Universe development process but only automate them when an interation is complete.
Finally, whilst the IDT does not at present allow automation out of the box, I wonder if there is an option for somebody to develop an Eclipse (the platform the IDT appears to have been developed from) script that would run a whole suite of the queries automatically? Could be worth a look at a later date.
In my next and final post on Agile Analytics and the SAP Information Design Tool, I’ll be looking at how the IDT can support the concept of excellent evolutionary design.
No comments:
Post a Comment