How QA Impacts Release Velocity in SaaS Teams

How Quality Assurance (QA) Affects the Velocity of SaaS Product Releases

In SaaS development teams, the velocity of product releases can determine success or failure.

The faster products are released, the faster customers can provide feedback, allowing for rapid iterations on those products and ultimately, greater competitive advantage. Conversely, the longer it takes to release products, the more backlogs build up, missed opportunities, and disappointed users result.

Typically, software development teams think of the velocity (i.e., how often a product is released) of releasing products as being a developer problem (i.e., have more developers, build better pipelines, write cleaner code).

In reality, QA is a key player in determining the velocity at which a SaaS product is released.

QA affects:

The rate at which SaaS-developed products are moved from “code complete” to live or in production.

The frequency of releases of SaaS-developed products is blocked and/or rolled back.

The level of confidence that teams feel when they ship changes to their SaaS-developed products.

The volume of rework that development teams must complete after a release of a SaaS-developed product.

In today’s SaaS environments, QA is not the final checkpoint before products are released but rather an enabler of product delivery (i.e., QA can accelerate or inhibit the speed at which products are delivered).

This blog post will explore the many ways QA affects release velocity in SaaS product development and how teams can improve release velocity without sacrificing product quality by taking advantage of the many ways in which high-performing organisations implement QA.

Release Velocity in SaaS Product Development

SaaS product development velocity is not solely about how often a product is released.

Release velocity is measured in:

• The number of days it takes from the time a feature is completed to when it is available to customers (in production).
• How often is a release to customers reliable?
• The speed of the feedback loop between a customer and a SaaS product development team.
• How quickly a SaaS product development team address defects in a product after it is released.

If release velocity is high, that means that a SaaS product development team can ship products confidently, learn continuously and quickly, and adapt quickly. When the release velocity is low, it generally means that there is systemic friction: in most cases, QA is at the centre of this friction.

Examples of how QA slows down release velocity

While we will discuss ways of speeding up release velocity, it is essential to understand the various ways that QA can unintentionally create a bottleneck.

1. Testing In Late Stages Of Development

When QA is performed after the development work has been “completed,” defects found in testing will be surfaced much later in the process.

This results in:

Engineers switching contexts to fix issues

Tickets are being reopened, and releases are being delayed

Regression testing cycles that are unplanned

Late-stage testing compresses timelines instead of providing buffer time.

2. The Overload of Manual Regression Testing

As SaaS products become larger, the size of regression testing suites also grows and expands. Without automation, regression testing will take days to perform as opposed to minutes.

As a result, releases are delayed while teams work to “batch” types of testing.

QA instead becomes a reactive type of role rather than a proactive role.

This reactive process,s due to the excessive amount of manual regression testing, is a frequent cause of low release velocity in SaaS products.

3. Poor Management of Test Data and Environments

Many SaaS teams waste time due to inconsistencies with test data, or unstable environments, or the unavailability of application dependencies required for testing.

The reliability of the QA process is directly linked to the reliability of the environments.

4. Lack of Visibility Into QA Metrics

If teams cannot view test execution times, defect leakage or trends in defect failures, the ability to optimise for improved release velocity will not be possible.

When QA lacks metric visibility, QA is essentially performing experimentation.

Ways That QA Can Speed Up Release Velocity for High-Velocity SaaS Teams

High-velocity SaaS teams view QA as an opportunity to accelerate release velocity rather than as a checkpoint to verify software quality. Below are some of the ways they accomplish this.

1. Shift-Left QA

When a team implements shift-left QA, this means that testing is performed at an earlier timeframe than it previously occurred.

This includes defining acceptance criteria prior to coding or writing automated and/or manual tests in parallel to feature development of APIs and application contracts at early phases of development.

The earlier a team can perform testing, the fewer surprises that can emerge during a given cycle. Fewer surprises equate to fewer delays for SaaS products. Correlating as follows, SaaS teams that Shift Quality Assurance (QA) Left have faster release speeds than teams that only Test at the end of the cycle.

Automated Testing is Used to Increase Velocity

One of the best levers for Improving Release Velocity in QA is Automated Testing. One of the largest impacts of Automation can be seen in the following areas:

Regression Testing.

API & Integration Testing.

Smoke and Sanity Testing.

Performance Baselines.

The majority of Automated Tests will run in parallel, on every code Change and without fail.

This enables a Faster Continuous Integration/Continuous Delivery (CI/CD) Process.

It Reduces Manual Effort.

It Provides Increased Confidence to Release.

Automated Regression Testing in CI/CD

No matter the size of a SaaS product, regression testing is inevitable. The question that must be answered is whether it will block release velocity or provide a basis for increased release velocity.

Automated Regression Testing:

Runs on every code commit or build.

Catches Issues before they are merged into the Main Branch.

Will Prevent Any Regression Issues from Reaching the Staging Environment.

Feedback Loops Increase Dramatically.

Most teams that use Automated Regression Testing are able to release once a day or multiple times per week.

QA Metrics that Impact Release Velocity Directly

“Measurement Leads to Improvement”

SaaS teams that perform at a high level can track QA Metrics and how they impact the Release Velocity of products. Examples of those QA Metrics would be:

Time to Execute Tests.

Defects that Escape the Testing Process.

Time to Detect a Defect.

Time Required to Fix a Defect.

Automation Coverage for Critical Paths.

By tracking these metrics, teams can identify where Quality Assurance (QA) is holding up the Release Velocity and where to implement Process or Automation changes to increase QA Release Velocity.

Real-Time QA Analytics and Dashboard

Recent QA Tools provide Real-Time Analytics Dashboards indicating the following:

Health of the CI/CD Pipeline.

Passed and Failed Test Trends Indicating Problems.

Tests That are Flaky or Inconsistent Patterns That Indicate a Problem.

Indications of Release Readiness.

Executives Have Real-Time Visibility and Make Decisions Faster.

Without Using Analytics, Executives and the QA Team Must Depend on Manual Status Updates

The Role of AI in Boosting QA Release Velocity

With the continued adoption of Artificial Intelligence in Quality Assurance (QA), QA will improve its speed of releases.

AI Tools Can:

Optimise the Testing & Reviewing Process with Intelligent Test Case Selection

Automatically Create Testing/Review Assignments of All Types

Improve QA Processes by Helping Identify Root Cause Analysis

By Allowing QA Teams to Focus on What’s Important (They Do Not Eliminate QA Teams)

The result of these AI Tools will be faster and better releases to the customer.

Cloud Testing & Parallelisation as a Strategy for Improving Release Velocity

Because of the need for rapid-testing-and-development in cloud-based (SAAS) environments, to make up for long test-execution times, cloud-based test platforms provide the following advantages:

Parallel Testing Execution / Parallel Testing of Abundant Environment Types

Cross-Environment and Cross-Browser Testing / Wide Coverage of Any Environment You Choose

Parallelisation of the QA Process will be the fastest way to increase release velocity while maintaining your previous testing coverage levels.

QA Release Velocity and Agile/DevOps Environments

For an Agile SaaS Environment, a SaaS team’s ability to achieve release velocity will be very dependent on how well QA is aligned with development.

High Release Velocity QA Teams:

Invite QA in During Sprint Planning

Include QA Tests in the Definition of Done

Integrate QA into CI/CD Testing Automation Pipelines

QA should not be viewed as a Separate Function from the Delivery of Software Products.

Managed QA Services’ Effect on Improvements to Release Velocity

Many SaaS Teams have added managed QA Services as a Strategy to Improve Their Release Velocity.

The Managed QA Service Providers Should:

Own The Overall Outcome (not ownership of the Test Execution Outputs)

Focus on Delivering a Scalable Automation Framework

Have Their QA Metrics Aligned to the Release Objectives of Their Clients

Having QA Services Focus on Increasing a Client’s Release Velocity Will Serve as a Force-Multiplier for a SaaS Product with High-Volume Growth.

Common Obstacles to a SaaS Release Velocity Increase

Even the Most Experienced Teams Will Experience Predictable Obstacles. The following are typical bottlenecks that SaaS teams encounter:

Excessive automation of features that are still in development

Lack of maintenance on tests

Reliance on unreliable UI tests

Disregarding testing at the API level

Treating QA as simply a pass-off to another team

The result of these bottlenecks is delayed releases to the customer.

Using Containerised Testing to Increase Release Velocity

By creating container-based test environments, SaaS teams are able to:

Develop consistent and repeatable test environments

Solve the “works on my machine” issue

Scale to meet the demands of large test runs

Reduce failures caused by reasons other than code-related issues, therefore increase predictability of the release.

Improving Release Velocity for Distributed SaaS Teams

The additional challenges of remote distributed SaaS teams include:

Time zone differences

Communication problems

Asynchronous passes off between departments

The use of cloud-native QA tools and automated pipelines to decrease dependence on manual coordination, it will enable an increase in velocity across all time zones.

How to Measure the Business Impact of QA in Relation to Release Velocity

Improvement of release velocity through QA is not simply a technology win.

Successful QA results in faster time-to-market, improved customer satisfaction, reduced costs of providing customer support and reduced levels of engineer burnout.

In the SaaS industry, speed of release has a compounding effect whereby each release builds on the prior release.

Conclusion: QA is a Release Velocity Enhancer

In the SaaS industry, Release Velocity does not depend solely upon the speed at which the code is being developed, but rather, how confident teams are in their ability to release verified code.

QA is integral to providing confidence to teams.

Teams that consider QA as a strategic element that is backed by automation, analytics and AI-driven practices experience faster releases with fewer surprises.

At Logiciel Solutions, we work with SaaS teams to build QA solutions that provide increased Release Velocity without an increase in risk. We focus on metrics, as opposed to just executing QA tests, when delivering QA solutions to SaaS teams.

If you feel like your Release Velocity is at its peak, more often than not, the issue is not “move faster”, but better systems to support your QA process.

Extended FAQs