DevOps - Automation and Continuous Testing is the Key

While leading the organization to adapt to continuous delivery and DevOps faced various challenges  where problem of many or 'automated' ---But'... or lengthy and sequential execution cycles with various reasons, not counting build size itself or network and geographical constraints. Test artifacts need to be copied across systems, components for different teams using varied test assets and if have to execute testes on frequently say hourly builds to identify failure immediately, quite challenging task. This holds the key to success though for both continuous integration and continuous delivery. Developer should get feedback on whether their code is "ok", using a quick personal build and test run immediately so developer can resolve it than working on something else (new function/logic), each line written impacts the system. On success, can run a complete set of tests to integrate the tested changes or integrate the changes and run the full set of tests based on time required. In any case quick turnaround is required and not possible to achieve it manually.

Now every one does automation, what is so special? Key is complete automation, with no 'But' or almost, system performing every task on its own and is various tools including of IBM hold the key here, providing seamless automation capabilities. It is for us to deliver the integrated Testing mechanism which not only perform tasks automatically but also reduce time required for testing even with increase in number of tests required to be executed.
 

 We tend to run everything but should be executed in the manner, run sanity test before full suite for individual components to ensure Build, Infrastructure, Configuration and other assets are working fine to minimal level of acceptance. Prioritize frequently failed test to be executed post Sanity to confirm current build is good for formal testing. It can lead to 'always working' test bucket with very little chance of failure, covering majority of bucket. With this we will have only a subset of test remain which either do not provide any major benefit or real time consuming and not required to run on every instance, reduce their execution frequency. DevOps also improve the process where we can  build and test the components separately and then perform integration testing enabling frequent testing of individual feature or component.


Concurrency or parallelism is another step where we can automate our Deployment of builds following DevOps model and automated provisioning/configuration enabling us to run tests across systems in parallel, providing results much faster. Following automated provisioning enable to use Template based approach, taking away need for install time and can snapshot areas where no new pieces need to be installed like databases / source systems.

 Stub(s) holds the key rather than for every small change we connect to external systems. This saves lots of time and tools like IBM Green Hat are great simulation tools. Consolidate Infrastructure using Cloud Based Deployment (Softlayer) to use local virtual systems where other test assets resided to remove network contentions and improve productivity.

 Will cover more of testing aspects and other DevOps related content including uDeploy, Chef, Softlayer in coming series.


-Ritesh 
Disclaimer: The postings on this site are my own and don't necessarily represent IBM's positions, strategies or opinions