How to Choose a CRO Agency (Red Flags)

Introduction

There has been a significant increase in searches related to Conversion Rate Optimization.

Search terms like:

• CRO Agency
• CRO Companies
• CRO Consultants
• All at peak volumes.
• However, there are still many businesses that:
• Hire a CRO Company
• Conduct several months of experiments
• Receive reports with great visuals, and
• Still wonder, “where’s my revenue?”
• The issue is not the CRO. It is your CRO agency selection.

This article discusses how to select a CRO agency and how to identify red flags and stories that separate CAPs based on vanity metrics from CAPs based on quantifiable revenue impact.

Why Using the Wrong CRO Agency is a Huge Cost

A bad CRO agency not only costs you money; it also costs you time, momentum, and trust in your CRO partner.

Some typical outputs of poor CRO agency engagements could include:

• Dozens of hours wasted developing low-impact tests
• Many hours of reports illustrating CTR and Engagement Enhancements
• Minimal impact on Revenue, CAC, or LTV
• An internal belief that CRO does not work

CRO can leverage results; however, if done improperly or poorly, it will simply cause distraction from other priorities.

First: What Does a Good CRO Agency Do?

Before looking at identifying the red flags, let’s define what a good CRO agency looks like.

A reputable Conversion Rate Optimization (CRO) agency:

• Connects experiments back to your company’s success
• Through understanding your company’s market (E-commerce, SaaS, etc.)
• Focuses on revenue potential from your tests rather than ease of execution
• Has experience working with pricing, onboarding and checkout processes
• Evaluates success based on revenue generated rather than percentage increase

The first red flag an agency gives you is when they can’t specifically articulate how their service helps generate revenue.

The First Red Flag: Starting with Tools Instead of Strategy

A lot of agencies will sell you on CRO by simply stating the tools they use, such as:

• Heatmaps
• Session Recordings
• A/B testing platforms
• Dashboard reports

Tools are helpful, however they don’t provide you with an overall strategic plan of what to test during your research process.

Be wary of agencies that are very tool focused, such as:

• “We will use the best tools for CRO”
• “We run experiments using superior industry-leading software”.

In both examples, the agency talks about the tool, before they discuss where the gaps in revenue exist. Any competent team can use Optimizely or VWO, but they will only be successful at testing and executing the best strategy based on your company’s unique needs.

The Second Red Flag: The Agency Cannot Explain in Simple Terms How CRO Work Will Impact Revenue

When interviewing the agency, ask the agency the following question:

“What will your CRO work do for our company in terms of revenue?”

Be prepared for vague, fluffy responses like:

• “We improve user experience”
• “We increase engagement”
• “We optimize the funnel”

Instead, you want to hear the agency provide concrete direct response such as:

• “By reducing checkout abandonment, we’ll increase the number of purchases made by customers.”
• “We’re increasing trial activation rates so that we can improve the trial to paid conversion rate.”
• “We are testing pricing and packaging in order to achieve higher average revenue per customer”.

If the agency refuses to talk about the revenue aspect directly, it’s not a good sign.

The Third Red Flag: One Size Fits All CRO

Not all CRO is the same for every business.

Ecommerce CRO is very different from SaaS CRO.

If the agency has the same playbook for all of the following business types:

• Shopify stores
• B2B SaaS products
• Marketplaces

Then the agency does not have a complete understanding of CRO.

RO Agencies That Specialize In Software as a Service

A professional e-commerce CRO agency will create pointedly specific strategies to address one or both of the types of transactional sales funnel:

• Transactional Sales Funnel – E-commerce
• Lifecycle Sales Funnel – SaaS
• To do anything else is of limited value

Warning Sign #4: Excessive Focus On A/B Test Quantity

Many CRO agencies will promote “10 test results each month” or “Fast Test Expansion” or “Continuous Experimentation” as a way to entice clients into using their services.

While the high number of tests provided may sound impressive, this can be an indicator of low-impact A/B testing. Low-impact testing means that less impact is being made on generating revenue. It is typically the case that revenue-producing experiments are:

• More difficult to build
• More costly to produce
• Longer to validate
• Agencies with high test volumes typically do not want:
• Pricing tests
• Checkout changes
• Onboarding flow redesigns

Because all of those tests take a significant amount of depth of thought and coordination.

Warning Sign #5: Use of Vanity Metrics to Define The Success of a CRO Program

If the agency uses the improvement of click-through rate, scroll depth, or time spent on website as the basis of determining the CRO program’s success, be cautious because while these metrics have merit, they don’t directly relate to paying salaries.

A reliable e-commerce CRO agency will link success with their clients back to factors that directly relate to sales: conversion to purchase, trial to paid, revenue per visitor, retention/expansion.

If the agency states that the revenue metrics are out of scope, it’s time to find another agency.

Warning Sign #6: They Do Not Support Pricing and Checkout

Pricing and checkout are the components of e-commerce where revenue is generated and often lost and yet, pricing and checkout components are among the most politically sensitive and thus, many e-commerce CRO agencies will avoid them.

However, the opposite is true because pricing and checkout components have potential for the largest CRO-related revenue impact.

If a CRO agency states either:

We do not work with pricing (or)

Checkout is out of scope;

Assuming they are working with other parts of the funnel to drive growth, they are avoiding the work that can most directly impact their client’s revenue.

Red Flag #7: No Specific Framework for Prioritization of Experiments

Question:

How do you determine what to test first?

Weak response:

• We are going to test (begin) with ‘quick wins’.
• We constantly test things that are the easiest to do.
• We think of new ideas together, (brainstorm together about ideas).
• A strong response/reference will indicate the following elements:
• Impact on Revenue
• Where there are Bottlenecks in the Funnel
• Where users have Conflicts of Intent
• Cost of Delay

If a CRO agency does not utilize a prioritization model, they will likely only conduct cosmetic types of tests.

Red Flag #8: Failure Cannot Be Discussed

To have true CRO, Failure Must Be Present.

Not all tests will win; in fact, many ‘high-impact’ tests will lose before winning.

• Be very cautious about agencies that:
• Only share Test Winning Case Studies.
• Do Not Discuss Failed Experiments.
• Claim Monsters of Wins.
• An Honest Agency will discuss:
• Why a Test Lost?
• What was Learned from the Loss?
• How the Learning is Applied to Future Tests.
• Being transparent about failure indicates a mature CRO agency.

Red Flag #9: CRO is Considered a Marketing Add-On

CRO is where all 3 areas of your business intersect together:

• MKTG (Marketing)
• UX (User Experience)
• Product
• ENG (Engineering)
• If an Agency Views CRO as:
• Landing Page Optimization ONLY
• Copy Tweaks ONLY
• Design Tweaks ONLY
• They’re Missing Half of the System.

This is especially Important for SaaS companies to note, as CRO typically resides inside the Product.

Red Flag #10: No Defined Ownership or Collaborative Model

Questions to Ask:

• Who is the person responsible for making Experiment Decisions?
• Who will Implement the Changes?
• How are you going to work with Engineering?
• Responses to these Questions usually result in:
• Slow Turn-Around
• Passing Blame
• Halted Experiments/Testing.
• A CRO Agency with a strong collaborative model is Defined. They are not just Ideation

Positive Indicators

What Are The Indicators of a Quality CRO (Conversion Rate Optimization) Agency?

When an Agency Uses a “Revenue First” Perspective

When speaking with you, an agency who uses terms like LTV (Lifetime Value), Payback Period, Margins, Money, etc. to talk about your UX is showing you positive signs.

Their Experience/Fluency in Various Business Models

A Quality CRO Agency Will Differentiate Between Ecommerce CRO, SaaS CRO, B2B & B2C Decision Cycles, and will not use the same playbook for different types of businesses.

They Will Design a Strategic Framework for How to Select Experiments to Run

A good CRO agency will typically run a smaller amount of experiments, but will run experiments that have a greater impact.

They Are Willing to Support Risk

A Good CRO Agency will challenge assumptions, redesign your checkout process, and change your pricing strategy, and these areas are where the most CRA leverage exists.

To Determine the Value of a CRO Agency, Look for Additional Indications

If you are looking for Value from a CRO Agency, you are likely going to discover the following popular keywords that are being searched for;

• Conversion Rate Optimization Services
• Conversion Rate Optimization Pricing

Be Cautious of;

• Low Monthly Retainers with Unrealistic Expectations
• Pricing structure based upon number of tests delivered
• Fixed Pricing packages with No Customization

A Quality CRO Agency will always price based on time, thought and execution rather than quantity of tests.

Questions to Consider When Selecting a CRO Agency

Use These Questions As Filters:

• What Experiment Has Had The Greatest Revenue Generation For The Clients Similar To Us?
• How Did You Decide What To Not Run Experiments On?
• What Metrics Define Success For You?
• How Do You Handle Failed Experiments?
• Will You Be Modifying Pricing, Checkout or Onboarding Process
• How Are You Customizing Your CRO Methodology To Meet The Needs of Ecommerce vs SaaS?

Quality Answers Are More Important Than Quantity of Answers.

What Is Responsible For The Widespread Perception That CRO Doesn’t Work?

The Following Are Some Reasons CRO Fails:

• Incorrect Optimization
• Diverse Success Metrics Applied To Assess Performance
• Lack of Real Risk Associated With Optimizations
• Seeking Immediate Results From Executives

When Performed Correctly, CRO Will Be Among The Highest ROI Leverage Points Available For Growth.

Final Thoughts – Choosing a CRO Agency is a Revenue-Driven Decision

CRO Agency Selection Is NOT A Design Decision, Tool Decision, Or Marketing Experiment; Rather, It’s A Revenue Decision.

An Excellent CRO Agency Allows Your Customers To Buy Easier From You, Reduces Friction During The Decision-Making Phase Of Their Purchasing Journey and Aligns Customer Experience To Customer Intent.

An Ineffective CRO Agency Handles Your CRO With Pretty Pages.

Select Thusly.

Extended FAQs