While automation saves you a lot of time, it still takes time. You can't run all your tests all the time. It takes too long and would generate an unmanageable analysis and maintenance effort. In my group, we've taken both manual and automation testing to three levels: sanity, end-to-end, and full. In addition to our feature tests, on every code commit, we run a set of high level, cross-feature tests to make sure that a code change in one feature hasn't broken another one. Only then do we run a set of more extended tests specific to the feature for which the code was committed. Then, we run our suite of feature-level sanity tests on our continuous delivery environment every three hours to make sure all features are in good shape. We only do this on one browser though, because we've found that if a test fails, it doesn't usually depend on the browser. Finally, we run feature end-to-end testing on our nightly environment.
When we talk about continuous testing, and with it continuous delivery and DevOps, the term automation gets thrown around a lot. In a basic sense, we all understand what automation means — the use of some technology to complete a task. But when we talk about automation in terms of continuous testing, there are some nuances that we need to take into account.
Automated software testing can increase the depth and scope of tests to help improve software quality. Lengthy tests that are often avoided during manual testing can be run unattended. They can even be run on multiple computers with different configurations. Automated software testing can look inside an application and see memory contents, data tables, file contents, and internal program states to determine if the product is behaving as expected. Test automation can easily execute thousands of different complex test cases during every test run providing coverage that is impossible with manual tests.
TestPlant eggPlant is a niche tool that is designed to model the user’s POV and activity rather than simply scripting their actions. Testers can interact with the testing product as the end users would, making it easier for testers who may not have a development or programming background. TestPlant eggPlant can be used to create test cases and scenarios without any programming and can be integrated into lab management and CI solutions.