Continuous Testing

Testing Philosophy

Cease dependence on mass inspection to achieve quality. Improve the process and build quality into the product in the first place. - Edward Deming

Continuous Testing

Testing is a cross-functional activity that should be done continuously from the beginning of development.

Continuous testing:

• Provides confidence that the code is working as desired.

• Results in fewer bugs, improves reputation and reduces support costs.

• Encourages good development practices.

Testing Strategy

Developers must define strategies for continuous testing, based on:

• Risks: Identify and prioritize the project risks.

• Mitigation of Risks: Decide actions to be taken.

Testing Types

While Continuous Integration course briefs upon basic tests such as unit and system test the next few slides will brief on other important tests.

Acceptance Testing

• It is the last stage of the application's testing cycle.

• This test is done using real-time scenarios to verify if the specifications are met.

Accepting testing can be either Operational Acceptance Testing (OAT) or end-user testing.

Types of AT

• Alpha & Beta testing: Improves usability of the end product.

  • Alpha: Testing is done in development environment.

  • Beta: Testing is done in customer environment.

• Contract Acceptance testing : Testing is done to ensure criteria and specifications agreed in the contract are met.

• Regulation Acceptance testing: Tests if the software complies with the regulations.

• Operational Acceptance testing: Tests the operational readiness of the application (such as security, backup, and recovery tests).

Functional Testing vs Acceptance Testing

Functional testing is a verification activity, while acceptance testing is a validation activity.

Capacity Testing

Scalability testing:

• Checks the response time of

  • Individual request.

  • Simultaneous user requests.

• More servers, services etc., added appropriately based on the test results.

Longevity testing:

• Measures the system performance.

• Assesses stability problems.

Throughput testing:

• Checks capability of the system to handle:

  - *Transactions*.
  
  - *Messages or page hits per second*.
  

Load testing:

• Checks capability of application to handle an increase in load.

Capacity Testing Practices

• Do testing in a production like an environment.

• Trigger only if the change passes the acceptance test.

• Carefully verify if failures are due to capacity issues (it may fail due to interface issues as well).

Manual Testing

Manual testing is done to ensure if the application is delivering the value as expected by the users.

Types of manual testing:

Showcases :

• Functionality demonstrated to users frequently during development.

• Specification issues are identified at an early stage.

• Exploratory testing:

  • Explores application features.

  • Leads to creation of a new set of test cases.

  • Captures new requirements for the application.

  Usability testing:

  • Determines if the application is useful, usable, accessible and desirable.

Smoke Testing

• Originated from hardware testing (a device when switched on is tested for smoke or fire).

• Also known as build verification testing.

• Not a comprehensive testing with limited test cases.

• Positive scenarios are tested with valid data.

• Ensures critical functionalities are working.

• Identifies issues introduced due to components integration.

Continuous Testing: Summary

• Automate the tests (like unit, component, acceptance).

• Trigger tests when a change is done to the code, configuration or the environment.

• Perform manual testing continuously throughout the project.

• Always improve the testing strategy.