A/B Testing Landing Pages#

When to Use#

Use this skill when planning, running, or analyzing A/B tests on landing pages.

Core Rules#

1. Test One Variable at a Time#

Each A/B test should change exactly one element: the headline, the CTA copy, the hero image, or the pricing display. Testing multiple changes simultaneously makes it impossible to know what drove the result.

2. Start With High-Impact Elements#

Test in this order: headline, primary CTA, hero image, pricing display, social proof placement. These elements have the highest potential impact. Testing button border radius or footer color wastes statistical power.

3. Run Tests Until Statistical Significance#

A test is not complete when you feel confident — it's complete when you reach 95% statistical significance with at least 200 conversions per variant. Stopping early because one variant looks better is the most common A/B testing mistake.

4. Segment Results by Traffic Source#

A headline that wins for paid traffic may lose for organic traffic. Always segment results by traffic source, device type, and new vs. returning visitors before drawing conclusions about a winning variant.

5. Document Every Test#

Maintain a test log with hypothesis, variant description, run dates, sample size, result, and what you learned. Teams that document tests compound learning over time. Teams that don't repeat the same mistakes.

Quick Reference#

ElementExpected LiftHeadline5–30%CTA copy5–15%Hero image5–20%Form length10–50%Social proof position5–15%

Common Mistakes to Avoid#

• Stopping tests after seeing a promising early result — early leads often reverse
• Testing on insufficient traffic (under 1,000 visitors per variant)
• Not accounting for day-of-week effects — run tests for full 7-day cycles

Test Your Product with Racoonn#

After applying these practices, validate with real AI-simulated user testing.

Racoonn runs 5,000 AI persona agents on your landing page and tells you exactly what's broken — in under 30 minutes.

→ API coming soon — Join the waitlist for early access: racoonn.me