## Testing Marketing Hypotheses: How to Find, Select, and Successfully Launch Them

Co-Founder Ai

# Introduction

Continuous idea generation and validation help businesses grow, but not every idea ensures growth. To identify those ideas that truly facilitate business expansion, hypotheses are created and tested. This process helps determine the potential outcomes of implementing an idea.

Hypothesis testing works for any ideas—from launching a new department within a company to changing the font on a website’s homepage. In this article, we will share our experience in testing hypotheses with examples of client tests and their results.

# What is a Hypothesis and Why It’s Important to Know How to Work with Them

A hypothesis is an assumption about a possible solution that can lead to the desired outcome. For example, to increase a website’s conversion rate to inquiries, you might launch a proactive chatbot. However, it’s unclear which chatbot mechanism—automatic qualification or perhaps a lead magnet—will be more effective. These are the hypotheses.

In marketing, hypotheses are formed and tested at all stages of a company’s development. They help to:

• Discover a working business model and scale the business
• Explore new platforms and audiences
• Boost product activation
• Enhance customer retention and loyalty
• Analyze the effectiveness of marketing channels

Before implementation, all hypotheses are tested. This approach helps save time and budget while reducing the risks of failures.

For instance, an online school launches a hypothesis: “A gamified pop-up on the homepage will collect ten times more leads than other site mechanisms.” Marketing tests this hypothesis with a group of students and monitors sales changes. If the experiment yields positive results, the proposal is scaled.

# How to Formulate a Hypothesis and Test Its Effectiveness

Hypotheses arise from ideas but are specific and measurable. They are always linked to business goals and rely on concrete metrics. For example:

• Idea: “Reducing the number of steps in the registration form will increase registration conversions.”
• Hypothesis: “Reducing the registration form by one step will increase conversion rate from 5% to 10%.”

To transform an idea into a hypothesis, follow four stages:

1. Define the Goal: Specify the desired outcome. For example, the goal “collect more leads on the website” sets the direction but isn’t a hypothesis itself. It lacks a specific aspect to influence and a metric to assess success.

2. Choose a Metric Within the Goal: Analyze the customer journey and identify which metric to influence. For example, examining the sales funnel and identifying stages where conversion drops significantly—these “bottlenecks” are critical areas for improvement. Decisions on which funnel stage to address should align with the team’s annual strategic goals.

3. Assess Improvement Potential: Based on data and analytics, estimate how the metric can be improved. For example, if the conversion from demo to inquiry is 15%, and industry research shows an average of double, there’s potential for a 15% increase.

4. Identify Impact Methods: Explore changes that could enhance the metric. Suppose customer surveys reveal issues with form completion. Simplifying the form could be the key to increasing conversion rates.

5. Formulate the Hypothesis: Based on collected data, propose a specific change. For example, “Adding address auto-fill in the order form after selecting it on an interactive map will increase order conversion from 5% to 10%.”

# How to Learn to Find Viable Hypotheses and Prioritize Them

Testing hypotheses requires time and resources, so it’s crucial to consider only promising assumptions before testing. Here are characteristics of a workable hypothesis:

• Measurable Outcome: For example, it’s challenging to assess the effectiveness of a lead magnet compared to a discount code directly. However, you can compare conversion rates from pop-ups with promo codes versus lead magnets.

• Achievable Goal within Budget and Time: If previous campaigns indicated that pop-ups with discounts perform better, it’s worth testing them by allocating budgets for advertising and time for design and analytics.

• Test Element and Target Metric in Formulation: For instance, “A pop-up on the homepage will increase button clicks from 1,000 to 2,000.”

• Impact on Main Metric: The hypothesis should influence the key metric, allowing you to understand how the target indicator has changed after testing.

• Market Segment Specificity: Hypotheses should be tailored to specific market segments because each industry has unique customer paths, competitors, tools, sales channels, and metrics. Analyze the customer journey using a Customer Journey Map (CJM) and formulate hypotheses for subsequent tests based on the findings.

Growth teams start working on hypotheses by conducting research—collecting data from the marketing funnel to identify where conversions drop. For example, users visit a blog to read articles but don’t submit an inquiry form after the text. The team then generates hypotheses that could enhance this conversion rate. These brainstorming sessions occur weekly, followed by meetings with leadership to defend the hypotheses, key metrics, and what constitutes a successful outcome. The teams then move into a sprint to implement the changes.

# Methods of Prioritizing Hypotheses

## ICE

This method works best when you have a list of hypotheses and need to quickly assess which ones can bring the most benefit with limited resources.

How to Use:

1. Evaluate Each Hypothesis on Three Parameters: Impact, Confidence, and Ease.
2. Assign a Score from 1 to 10 for Each Parameter.
3. Calculate the ICE Score: Multiply the scores: Impact × Confidence × Ease.
4. Compare Hypotheses Based on Their ICE Scores and Select Those with Higher Scores.

## RICE

Use this method when it’s essential to consider the audience reach influenced by the change. For example, a new feature might be used by 30% of your audience (300 out of 1,000 people). Conversely, another feature might be used by only 5% (50 users).

How to Use:

1. Add a Reach Parameter to ICE.
2. Evaluate Reach: How many users will the change affect?
3. Calculate the RICE Score: Reach × Impact × Confidence × Ease.
4. Choose Hypotheses with the Highest RICE Scores.

## Kano Model

This method is ideal for understanding which features or improvements are most valuable to customers and enhance their experience. Align marketing with the product team when applying this model.

How to Use:

1. Group Features into Categories:

   • Basic: Necessary but do not elicit excitement (e.g., data calculation in a calculator).
   • Performance: The more, the better (e.g., an AI robot in a smart speaker that speaks five languages).
   • Delighters: Unexpected but can significantly improve product perception (e.g., a calculator that can read handwritten inputs or accept voice commands).

2. Gather Customer Feedback: Understand how customers perceive each feature.

3. Analyze Data: Determine which features provide the most value to customers.

4. Prioritize Features: Start with Basic (must-have), then Performance, and finally Delighters.

## Lean Prioritization

Use this method when you need to quickly determine which features or improvements to implement first. It’s commonly used by startups or in situations with limited resources requiring rapid movement.

How to Use:

1. Organize Hypotheses or Features into a Matrix: One axis for “Customer Value” (or impact on business goals) and another for “Ease of Implementation.”

2. Place Each Hypothesis or Feature in the Matrix Based on Its Scores.

3. Identify Four Quadrants:

   • Do Now: High value and easy to implement.
   • Plan: High value but difficult to implement.
   • Simplify: Low value but easy to implement.
   • Don’t Do: Low value and difficult to implement.

4. Focus on Hypotheses in the “Do Now” Quadrant: They promise the highest value with the least resource expenditure.

5. Lean Prioritization Helps Focus on What’s Truly Important: Allows companies to be more agile and responsive to market changes with minimal resource investment.

Choose a prioritization method based on your initial data. If data is sparse and evaluations are subjective, opt for qualitative methods like the Kano Model and Lean Prioritization. Conversely, if you’ve gathered substantial and reliable data, consider quantitative methods. Data can be collected using web analytics or data collection tools available on your site.

You don’t have to stick to a single method. Customize them to fit your tasks and create your prioritization principles. Large companies often develop custom methods based on internal risk and business priority evaluations. These methods can also be used to improve the task backlog or assess specific mechanism importance.

# How to Test a Hypothesis

After evaluating and prioritizing hypotheses, it’s time to test them. In marketing, A/B testing is commonly used for this purpose.

A/B Testing involves comparing two or more versions of an element (such as a lead form, advertisement, or banner) that differ by minimal changes. These tests determine which version better attracts users, increases conversions, and generates profit.

Example: A marketer aims to increase the number of clients for an online service. They want to understand whether a full-screen pop-up or a right-side pop-up is more effective. To do this, they conduct an A/B test:

1. Prepare Multiple Design Variations: For example, create different pop-up versions with varied messages.
2. Define the Target Audience: Select website visitors interested in a specific topic who haven’t interacted with managers or tested the service yet.
3. Launch the Test: Show group A visitors the first pop-up variant and group B visitors the second variant.
4. Analyze the Results: After a set period, compare metrics like conversion rate or CTR (click-through rate) for each variant and choose the one with better performance.

# Assessing the Results of Running a Hypothesis

Similar to initial evaluations, you can use calculators to compute final results. When data is insufficient for automated tools, manually perform a Z-test.

Z-Test Steps:

1. Formulate Null (H₀) and Alternative (H₁) Hypotheses:

   • H₀: “There is no difference in conversion rates between the promotion types.”
   • H₁: “Promotion with a discount code increases conversion rates.”

2. Collect Data: Measure conversion rates for each group.

3. Conduct the Z-Test: Determine if the difference in conversion rates is statistically significant based on a chosen significance level (usually 0.05).

4. Analyze Results: If differences are not significant, accept the null hypothesis; otherwise, accept the alternative.

5. Perform Additional Checks: Compare related metrics, such as the number of users adding items to the cart after opening emails.

6. Make a Decision: If the hypothesis proves effective, implement the change. If not, set it aside and move to the next one. Revisit and refine set-aside hypotheses later as needed.

This approach enables data-driven decision-making, even with limited data.

# Methods of Prioritizing Hypotheses

## Methods of Prioritizing Hypotheses

Refer to the Methods of Prioritizing Hypotheses section above.

# Conclusion

Testing and properly prioritizing hypotheses are crucial steps in driving business growth and optimizing marketing strategies. By leveraging methods like ICE, RICE, the Kano Model, and Lean Prioritization, teams can effectively determine which hypotheses to pursue, ensuring that resources are allocated to initiatives that offer the most significant potential impact.

Implementing robust testing frameworks, such as A/B testing, allows businesses to validate their hypotheses, making informed decisions that align with their strategic goals. Tools like Co-Founder Ai can assist in this process by providing insights and automation, making hypothesis testing more efficient and effective.

Embrace hypothesis-driven marketing to unlock investment opportunities, attract angel investors, and connect with venture capital firms. By continuously testing and refining your strategies, your startup can achieve sustainable growth and secure a competitive edge in the market.