By Nancy Friedrich, Aerospace & Defense Industry Solutions – Keysight Technologies.
Technology leadership increasingly stems from software development. Thanks to software automation, it is easier to hit software goals and release software in shorter cycles.
DevOps, a newer approach to speeding and simplifying software development, also includes information-technology (IT) operations. The goal of DevOps is to tighten software development so that software can be regularly distributed without compromising on high quality.
In aerospace defense, such demands for quality are stringent to the point of unwavering. The software cannot fail. Yet adopting a DevOps approach in a military or government organization often means building and testing software in highly secure environments with siloed teams and the need for top secret clearance.
These aspects introduce more challenges and thus higher risk. Start with these steps to ensure that your DevOps approach proves highly effective even against the complexities of the aerospace defense environment:
1. Leverage non-invasive automation
With non-invasive automation, the solution can interact with software and systems without jeopardizing any security requirements. This allows for robust automated end-to-end testing of highly secure systems. You use a combination of two techniques to achieve non-invasive automation:
- A two-system model allows the testing software to sit on a machine that is separate from the SUT.
- Testing is done via the user interface (UI) rather than through access to the source code or object layer, as they can be obscured or unavailable to standard automation tools.
When you test at the UI level, the automation software can interpret and intelligently interact with the UI of the system under test (SUT). This process works via a combination of optical character recognition (OCR) and computer vision. OCR allows the automation software to read the on-screen text. Computer vision scans the interface for visual elements, buttons, fields, values, colors, and more to make sense of what appears on screen. Automation tools that test at the UI level can also access any object properties available or exposed to hone their understanding of the UI.
2. Perform exploratory testing
Testing and evaluation processes often focus on linear, directed testing. Here, you take a series of steps to demonstrate that certain functions and workflows function properly. In practice, however, problems often arise in unexpected places. The testing surface (i.e., the totality of possible test cases) can increase exponentially when you account for application complexity and the systems in which they work. If you only test predetermined paths, issues outside of those paths will never be discovered. It often falls to manual testers to haphazardly click around an application, trying to find and document anomalous behavior and bugs.
3. Pick the right solution
A seemingly endless assortment of test automation options exist. For aerospace defense, you want to perform test automation in secure environments and deliver quality software in tight cycles. You need a solution that incorporates non-invasive testing, 2FA via smart cards or Common Access Cards (CACs), and is device, operating system, and language agnostic.
Also among these criteria is the need to deeply and broadly test both applications and the systems that employ them. Beyond covering your most essential user journeys, your testing solution should be able to conduct the following:
- all nodes — that is, any action, state, or variable value possible in the application
- all pairs, which covers pairs of actions taken in the application
- extended test runs of three or more nodes
- full exploratory tests that cover all possible user journeys through the application
No matter which path you take, your objective is to choose the test automation solution that allows you to streamline and dramatically shorten the evaluation phase of your development project. You want to modernize your workflows and maximize quality without introducing any weakness or risk — especially in the aerospace defense environment.
