...
The purpose of regression testing is to ensure that changes due to software enhancements, patches, configuration changes, etc. have not introduced errors. One of the main reasons for regression testing is to determine whether a change in one part of the software has a negative impact on other parts of the software. Regression testing usually means re-running previously completed tests and checking whether program behavior has changed in undesirable ways and whether previously fixed errors have re-emerged.
Tests may be run at different levels:
- UI level testing - using tools such as Jasmine for Javascript UIs
- Service level testing - using tools such as CasperJS
- Unit level testing - using tools such as JUnit for Java code, PHPUnit for PHP code, ...
See Testing skill and proficiencyRegression testing can be performed to test a system efficiently by systematically selecting the appropriate minimum set of tests needed to adequately cover a particular change. The test pyramid (https://martinfowler.com/bliki/TestPyramid.html) suggests that one should have many more low-level unit tests than high level end-to-end tests running through a GUI.
Continuous integration (CI)
...
Systems for issue tracking may be both integrated and standalone. Examples are github and gitlab (integrated), Jira from Atlasssian (standalone, but integrated with other Atlassian products), bugzilla (standalone), ...
There is also a process of using strict issue tracking where code updates must be associated with an issue, code is updated if and only if there is a specified issue.
Code review
When providing code for an issue, you often want to review it before allowing the code to be committed into and merged with the code repository. This is particularly relevant when code is contributed from outsiders, like in open source projects, but also relevant in more closed settings, to ensure quality. Sometimes you also want to run CI on the code, before doing the review and stamp of approval.
...
If we want to introduce these activities in a curriculum in a study program, we should consider a staged approach, were activities and tools are introduced gradually. Based on proficiency levels (https://hr.od.nih.gov/workingatnih/competencies/proficiencyscale.htm: Fundamental Awareness (heard about), Novice (limited experience), Intermediate (practical application), Advanced (applied theory), Expert (recognized authority)). For a bachelor program, we may expect students to reach a Intermediate level in some activities, and Novice in others, while a master program may lift the bar depending on the specialisation.
| Activity/Proficiency | Fundamental Awareness | Novice | Intermediate | Advanced |
|---|---|---|---|---|
| Code management | Fetching code samples and other material from a code repository prepared by others, and being aware of the flow. | Basic commands for committing, pushing and pulling, and understanding what merge conflicts are and how to handle them. | Versions, tagging and branching and merging. Forking and updating. | Deployment and releases. |
| Code quality | Using tools for automatic formatting and simple refactoring. | Use tools for defining coding style and formatting code accordingly, have basic refactoring skills and run code quality checking tools locally or as part of CI. | Code analysis like null analysis and code metrics. Introduction of static and dynamic code analysis methods and tools | Verification of safety and mission critical systems Introduction of model checking and formal methods |
| (Regression) testing | Running pre-made tests on their code and understanding the fundamental idea of testing code behavior. | Writing and running their own unit tests (not necessarily TDD). | Writing integration tests according to the test pyramid; using mocking frameworks. | Regression testing Model-based testing |
| Continuous integration | Using tools for automising the process of building, testing, and deploying software. | Using a build system for assembling a runnable application. | Unit and integration testing, driven by build system. | CI automation using a server. |
| Issue tracking | Use issues tracking system to report bugs (to some system they are using). | Use issues to manage requirements that drive development. | Kansan-like management | |
| Code review | Going through code with others, to improve/ensure quality. | Pull requests, for contributing fixes to other system. | Using code review within own process, to close issues. | |
| Agile Methods | Knowing about Scrum etc. | Experience with some tools/meetings | E.g. Scrum Master |
Similar approaches found online:
Programmer Competency Matrix - https://sijinjoseph.com/programmer-competency-matrix/
Staged introduction (proficiency mapped to semester)
| Activity/Semester | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 |
|---|---|---|---|---|---|---|---|---|
| Code management | ||||||||
| Code quality |
| (Regression) testing | ||||||||
| Continuous integration | ||||||||
| Issue tracking | ||||||||
| Code review | ||||||||
| Agile Methods |
Bachelor in Programming (Games|Apps)
| Activity/Semester | 1 | 2 | 3 | 4 | 5 | 6 |
|---|---|---|---|---|---|---|
| Code management | ||||||
| Code quality | ||||||
| (Regression) testing | ||||||
| Continuous integration | ||||||
| Issue tracking | ||||||
| Code review |