Goals
- Automate testing of the business rule engine with standard & custom rules
- Develop a framework for test automation with configurations for test runs
- Provide a way to onboard & test new business rules without writing new test scripts
- Reduce the configuration efforts required in testing across multiple (product & client) environments
- Reduce the overall manual testing efforts
Results
- Reduced the manual testing efforts by 82%
- Reduced time for Onboarding & testing of new business rules by 65%
- Reduced the configuration efforts for multiple environments by 100%
- Collated test results across multiple environments into a single e-mailable report
- Test results were pushed to DB for more analysis using Grafana
Approach
- Identified the individual components of the rules engine (Event, Criteria, Actions) & wrote unit tests to accept test data & publish results
- Developed XML configuration for the rules allowing the tests to be configured dynamically
- Built / Altered the test suites reading the configuration from the database
- Used Spring boot, TestNG, Selenium & Jenkins as the tech stack for Test Automation
- Developed multiple execution reports for test result analysis
With the increase in the number of business rules defined at product & customer installations it was increasingly difficult for the product upgrades & customer wanted a reliable way of testing the business rules for every product release.
Also, the number of rules to be tested for every product release across multiple environments & configurations customer wanted to reduce the manual testing efforts in testing & configuration.
Solution
First step in automation was to understand & break down the rule engine into smaller independently testable components. Every rule had an Event, Criteria & one or more Actions. Automation of these components was put as an entry criteria for further tests. With this approach we were able to identify the issues early in the overall test execution & control the overall test execution. For example, in a rule trigged on an event satisfying a criteria a specific action being performed. if the event itself is not triggered any other rules associated were skipped for test execution.
We developed a test automation framework to read the rule configurations & test data from an XML file enabling the test engineers to make any modifications if the criteria or actions changed in the product. The framework provided a way to chain & test the independently tested units as a workflow / end to end test. Also, within the framework we used dependency injections of Spring to ensure the configurations can be controlled externally.
Once the core framework was setup, onboarding of new rules for testing was a simple process of defining an XML, test data & updating the test database with few configurations. The test results were logged into database for detailed analysis on every execution & different environments.
Automated testing of the business rule engine reduced the manual testing efforts by 82% for every release. Testing of new rules was possible with few simple configurations without need for custom scripts / test case development.
Using the test results from the executions, Grafana was used to develop multiple KPI reports for the management like number of rules automated, number of rules executed per cycle, number of rules passed across multiple environments, execution time per rule across environments & rules executed across product releases etc. Having the reports made it easy for the product team to focus more the changes in the specific release & in specific client environment.