In some contexts, investing heavily in testing is essential. For example, when processing financial transactions at a bank you’d want to have really solid integration tests.
These tests could be a mix of engineered data, re-playing of real-world data that has been manually audited, and maybe even a business rules engine built to double check and alert for any large day-over-day anomalies in processing (a banks’ total deposits increasing 50% day-over-day would be a red flag).
At the same said bank, there is a marketing effort to serve customized banners to the customer’s browser during online banking sessions. Ultimately, the business decides that this software is not critical to the businesses’ operations. The team could then decide to launch and iterate very quickly, only bothering to maintain a critical-path integration test.
Always consider: If the software you develop has bugs, or a customer receives a less than optimal experience, what is the impact on the businesses’:
- Operations
- Are internal operations degraded?
- Are sales being affected?
- Reputation
- How will customers’ trust for this company be affected?
- Will internal customers lose trust with our team?
- How will customers (internal and external) have their opinion of the company shifted?
Consider these factors when to determine what your minimum testing requirement is.
I generally error on the side of caution as I suspect many small defects or less than optimal experiences could cumulatively lead to a small, but perceptible harm to the companies’ reputation in the customer’s mind.
Ultimately, the more tests you have, the fewer defects you’ll have, the less manual testing you’ll have to do, the less documentation you’ll have to write, and you’ll end up with a more maintainable product (in most cases).
Under external pressures, like business deadlines, or given a desire to A/B test a new feature, we, as developers must determine what is the minimal amount of testing that could be applied is while still maintaining professional standards.