Continuous Validation for ServiceNow - HOW IS IT DONE?
What are the key steps involved in continuously validating a ServiceNow App?
xLM Continuous Validation Framework for ServiceNow Apps
The above diagram depicts the key elements of a Continuous Validation Program for ServiceNow Apps (Note: ServiceNow IaaS/PaaS Qualification is not included here. The focus of this article is on App Validation).
One has to bear in mind that the underlying IaaS and PaaS infrastructure is constantly changing. In fact, the Cloud App itself is continuously changing. In the new cloud world, it does not make any value-sense to pin to an "ancient" version. Thus this Continuous Validation Framework is designed to mitigate these risks and ensure that your ServiceNow app is maintained in a validated state.
“Continuous validation is providing documented evidence to certify that a cloud app not only met the pre-established acceptance criteria, but “continuous” to meet thus mitigating the risk of unknown changes.”
Requirements Definition
This step provides the foundation for the continuous validation framework. We clearly specify Functional, Non-functional, Regulatory, Performance, Security, Logging, Disaster Recovery, Interface requirements, etc. working with the Customer/End User.
Risk Assessment
In our experience very rarely a logical, useful risk assessment is performed, let alone applying it to testing strategies. The output of our risk assessment is applied to our testing strategies to determine: What features to test? What should be the extent of negative testing? What type of testing strategies to utilize (for eg: datasets to use, N Pair Testing)?
We establish a risk-based approach to qualifying each requirement. Our risk-based approach is achieved by assigning a Risk Priority to each requirement. A typical Risk Assessment may be as follows:
High – A risk priority of High shall be assigned to a critical requirement which meets the following criteria:
- is not “out of the box” (OOTB) functionality AND
- is a legal/regulatory requirement.
All High priority requirements will be tested (both positive and negative testing).
Moderate – A risk priority of Medium shall be assigned to an important requirement which meets the following criteria:
- is achieved with “out of the box” features; AND
- is a legal/regulatory requirement;
All Moderate Priority requirements will be tested (positive testing) or verified (configuration verification).
Minimum – A risk priority of Low shall be assigned to a “nice to have” requirement which meets the following criteria:
- is achieved with “out of the box” software features.
Minimum Priority requirements will not be tested.
Specification Definition
We specify Configuration, Workflow, Interface, Security, Log Management, etc.. specifications to meet your requirements defined earlier.
Traceability Matrix
We establish traceability between requirements and testing to ensure coverage.
Test Automation Scripts
xLM leverages a Model Based Test Automation framework for developing various models to validate your ServiceNow apps. We use a data designer to generate test data including randomized data. We also use combinatorial testing strategies to reduce the number of iterations while increasing the test effectiveness.
Test Case Definitions - Sample
xLM Test Automation Model for ServiceNow Change App
Test Model Validation
The test automation model is validated to ensure that it is meets the specified objectives. This validation effort is based on a model design that generates good execution reports. All our execution reports provide enough evidence so that it is, to a certain extent are self-validating.
Test Execution
The model based test automation approach provides us with the flexibility to re-purpose the same model to conduct various types of tests (smoke, regression, greedy path, optimal path, load, performance, etc..). Also, such a framework lends itself more conducive to updates (remember your cloud app is constantly changing...so will your test automation framework!). Our framework is well suited for continuously running validation test scripts (say on a daily basis) - you can even randomize your smoke and regression tests continuously!
Test Summary Report Generated by the Automation Model
Test Protocol Execution - UI Testing
Test Coverage Graph
Validation Reporting
A robust ALM (Application Lifecycle Management) tool forms the heart of our Continuous Validation Platform. Our goal is to remove paper and manual generation of reports (in short: Microsoft Word will not play any role here!). Our ALM tool provides real-time dashboards, KPIs, summary reports, test deviation reports and more. Your real-time Validation Health Dashboard becomes a reality.
Test Execution Results History KPIs
Conclusion
xLM platform is designed to accommodate high velocity of changes with built-in support for robust testing strategies. It incorporates modern technology and frameworks to ensure data integrity requirements are met as expected by the Regulatory Agencies. xLM is the right track for ServiceNow validation!