Testing skill and proficiency

(Regression) Testing

The purpose of regression testing is to ensure that changes due to software enhancements, patches, configuration changes, etc. have not introduced errors. One of the main reasons for regression testing is to determine whether a change in one part of the software has a negative impact on other parts of the software. Regression testing usually means re-running previously completed tests and checking whether program behavior has changed in undesirable ways and whether previously fixed errors have re-emerged.

Tests may be run at different levels:

• UI level testing - using tools such as Jasmine for Javascript UIs
• Service level testing - using tools such as CasperJS
• Unit level testing - using tools such as JUnit for Java code, PHPUnit for PHP code, ...

Regression testing can be performed to test a system efficiently by systematically selecting the appropriate minimum set of tests needed to adequately cover a particular change. The test pyramid (https://martinfowler.com/bliki/TestPyramid.html) suggests that one should have many more low-level unit tests than high level end-to-end tests running through a GUI.

Proficiency levels

Knowing that code can test other code and seeing it in practice, e.g. downloading tests that tests your own solution for a programming exercise. The terms unit test and test case, understanding the sequence of states it tests.

Be able to write your own unit tests (technically) and have a basic understanding of test design, i.e. defining test cases corresponding to a specification 

Designing complete test cases for the desired behaviour, using tests for specifying behaviour before the code to be tested is written. Understand the relation between design and test design, and how mocking is enabled and used. The idea of test coverage and running tests as part of CI.

Integration testing with testing of more architectural elements, like full stack with DB, domain objects, REST API and UI (whether desktop or web). What level of test coverages is required for various parts of a system.

System testing (testing a complete, deployed system), acceptance testing (testing according to customer's expectations).

Expert

Model-based testing and test completeness. Verification methods.

Tools

JUnit - test framework for Java, versions 3, 4 and 5. Platform for many other libraries and frameworks.

Hamcrest - library for more fluid ways of formulating tests.

Mockito - framework for mocking

CasperJS - integration testing of REST APIs

Selenium - web UI testing (drives or simulates a browser)

Arquillian - integration testing for Java

JavaFXTest - JavaFX UI testing