This step-by-step tutorial shows you how to create and run A/B tests (split tests) to optimize your landing page conversions.

What You’ll Learn

By the end of this tutorial, you’ll know how to:

1. Create A/B test experiments
2. Design test variants
3. Configure traffic distribution
4. Track and analyze results
5. Declare a winning variant

Time Required: 15-20 minutes
Difficulty: Intermediate
Prerequisites: At least one published landing page

What is A/B Testing?


A/B testing (or split testing) shows different versions of your landing page to visitors and measures which performs better.

Example: Test two different headlines to see which one generates more signups.

Why A/B Test?


6. 📈 Increase conversions: Find what works best
7. 💡 Make data-driven decisions: Remove guesswork
8. 💰 Improve ROI: Get more from existing traffic
9. 🎯 Understand your audience: Learn what resonates

Before You Start


What You Need


10. ✅ Published landing page with traffic
11. ✅ Analytics enabled for tracking
12. ✅ Conversion goal defined (form submit, button click, etc.)
13. ✅ Hypothesis: What you want to test and why

Recommended Minimum Traffic


For reliable results, you need:

14. Minimum: 100 visitors per variant
15. Better: 500+ visitors per variant
16. Ideal: 1,000+ visitors per variant

Low traffic? Tests will take longer but still work.

Step 1: Choose What to Test


Good Test Ideas


Start with elements that have the biggest impact:

High-Impact Elements:

17. ✅ Headline: The first thing visitors see
18. ✅ Call-to-Action (CTA) button: Text, color, size, position
19. ✅ Hero image: Visual appeal and relevance
20. ✅ Form fields: Number and type of fields
21. ✅ Social proof: Testimonials, trust badges, stats

Medium-Impact Elements:

22. ✅ Body copy: Length, tone, benefits vs features
23. ✅ Layout: Single column vs multi-column
24. ✅ Colors: Brand colors vs contrasting colors
25. ✅ Pricing display: Format, emphasis, discounts

For This Tutorial


We’ll test two different headlines to see which generates more email signups.

Hypothesis: A benefit-focused headline will convert better than a feature-focused headline.

Control (Original): “Professional Email Marketing Software”
Variant A: “Get 10x More Email Opens in 30 Days”

Step 2: Access the Experiments Section


26. Go to Landing Pages in WordPress admin
27. Find your published landing page
28. Hover over the landing page title
29. Click Create Experiment

Alternative path:

30. Go to ShahiLandin → Experiments
31. Click Add New Experiment
32. Select the landing page you want to test

Step 3: Set Up Your Experiment


You’ll see the Create Experiment screen.

3.1 Experiment Name


`
Experiment Name: Headline Test – Benefits vs Features
`

Tips for naming:

33. Descriptive: Explain what you’re testing
34. Include date: “Headline Test – Nov 2024”
35. Version numbers: “Homepage Test v3”

3.2 Description (Optional)


`
Description: Testing benefit-focused headline against feature-focused headline to improve email signup conversions.

Hypothesis: Benefit-focused messaging will resonate better with visitors looking for results.
`

3.3 Select Parent Landing Page


`
Landing Page: [Dropdown menu – select your page]
`

Choose the landing page you want to run the test on.

3.4 Set Conversion Goal


Define what counts as a “win”:

`
Conversion Goal: Email Signup Form Submission
Goal Value: $10 (optional – assign dollar value to conversions)
`

Common goals:

36. Form submission
37. Button click
38. Purchase
39. Download
40. Scroll depth
41. Time on page

3.5 Traffic Split


Configure how traffic is distributed:

`
Traffic Allocation:
○ Control (Original): 50%
○ Variant A: 50%

Total: 100% ✓
`

Options:

42. 50/50 split: Standard for A/B test (recommended)
43. 33/33/33: For A/B/C test with two variants
44. 80/20: To minimize risk, show variant to fewer visitors

For this tutorial, use 50/50.

3.6 Test Duration


`
Duration Settings:
☐ Set maximum duration
☐ Set minimum sample size
`

Optional but recommended:
`
☑ Set maximum duration: 30 days
☑ Stop test automatically when statistical significance reached
`

Click “Create Experiment” to continue.

Step 4: Create Your Variant


After creating the experiment, you’ll be taken to the Variant Editor.

4.1 Name Your Variant


`
Variant Name: Benefits Headline
`

4.2 Edit Variant Content


You have two options for editing:

Option A: Visual Editor (Easier)


45. Click Visual Editor tab
46. You’ll see a preview of your landing page
47. Click on the headline to edit
48. Type the new headline: “Get 10x More Email Opens in 30 Days”
49. Click Save Changes

Option B: HTML Editor (More Control)


50. Click HTML tab
51. Find the headline in the code:

`html

Professional Email Marketing Software

`

52. Change to:

`html

Get 10x More Email Opens in 30 Days

`

53. Click Save Changes

4.3 CSS Changes (If Needed)


If you want to change styling too:

54. Click CSS tab
55. Add or modify styles:

`css
.hero h1 {
color: #ff6600; / Change headline color /
font-size: 48px; / Make it bigger /
font-weight: bold;
}
`

4.4 Preview Your Variant


56. Click Preview button
57. New tab opens showing Variant A
58. Verify changes look correct
59. Test on mobile (resize browser or use DevTools)

Important: Only change the ONE element you’re testing. Don’t change multiple things at once or you won’t know what caused the difference.

4.5 Save Variant


Click Save Variant button when satisfied.

Step 5: Review Experiment Settings


Before starting the test, review your configuration:

Experiment Summary


`
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Experiment: Headline Test – Benefits vs Features
Landing Page: Email Marketing Page
Status: Draft

Variants:

60. Control (Original): 50% traffic
61. Benefits Headline: 50% traffic

Conversion Goal: Email Signup Form Submission
Goal Value: $10

Duration: 30 days (or until statistically significant)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
`

Pre-Launch Checklist


Before starting, verify:

62. ✅ Conversion tracking is set up correctly
63. ✅ Variant shows only ONE change from control
64. ✅ Traffic split adds up to 100%
65. ✅ Landing page is getting adequate traffic
66. ✅ Test duration is reasonable (2-4 weeks typically)
67. ✅ Analytics is enabled

Step 6: Start Your Experiment


When ready to launch:

68. Click Start Experiment button
69. Confirm you want to start
70. Status changes from “Draft” to “Running”

Your test is now live! 🚀

What Happens Now?


71. Visitors are randomly assigned to Control or Variant A
72. Each visitor consistently sees the same version (via cookie)
73. ShahiLandin tracks views and conversions for each variant
74. Statistics update in real-time

Immediate Actions


Don’t:

75. ❌ Stop the test too early
76. ❌ Make changes to landing page during test
77. ❌ Start other tests on same page
78. ❌ Check results every hour

Do:

79. ✅ Let test run for at least 1-2 weeks
80. ✅ Monitor for technical issues
81. ✅ Continue driving traffic to the page
82. ✅ Check weekly for statistical significance

Step 7: Monitor Your Results


View Experiment Dashboard


83. Go to ShahiLandin → Experiments
84. Click on your experiment name
85. View the statistics dashboard

Understanding the Dashboard


You’ll see:

`
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Running for: 7 days

Control (Original)

86. Visitors: 523
87. Conversions: 42
88. Conversion Rate: 8.03%
89. Confidence: —

Variant A (Benefits Headline)

90. Visitors: 508
91. Conversions: 61
92. Conversions Rate: 12.01%
93. Confidence: 95.2% ✓

Winner: Variant A
Improvement: +49.6%
Statistical Significance: Reached ✓
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
`

Key Metrics Explained


Visitors: Number of unique people who saw this variant
Conversions: Number who completed the goal
Conversion Rate: Percentage who converted (conversions ÷ visitors)
Confidence: Statistical confidence that this variant is truly better
Improvement: How much better than control

Statistical Significance


What it means: How confident you can be that the difference is real, not random luck.

Confidence Levels:

94. < 90%: Not significant – keep testing
95. 90-94%: Marginally significant – might work
96. 95%+: Statistically significant – reliable winner ✓
97. 99%+: Highly significant – very reliable

Wait for 95%+ confidence before declaring a winner.

Step 8: Analyze the Results


After 1-2 Weeks


Check your experiment dashboard. You’ll likely see one of these scenarios:

Scenario 1: Clear Winner (95%+ Confidence)


`
Variant A is winning with 95%+ confidence
`

Action: Proceed to Step 9 to declare winner

Scenario 2: Trending But Not Significant


`
Variant A shows 12% conversion vs 8% control
But confidence is only 87%
`

Action: Keep test running, need more traffic

Scenario 3: No Clear Difference


`
Both variants performing similarly
Control: 8.1% | Variant A: 8.3%
Confidence: 42%
`

Action: Either continue testing or accept that this change doesn’t matter

Scenario 4: Variant is Losing


`
Variant A: 6.2% conversion
Control: 8.1% conversion
Confidence: 94% that control is better
`

Action: Stop test, implement original (control wins)

Step 9: Declare the Winner


When you’ve reached statistical significance (95%+ confidence):

9.1 Review Final Stats


Double-check:

98. ✅ Confidence level is 95% or higher
99. ✅ Both variants have minimum sample size (100+ each)
100. ✅ Test ran for at least 1 week (preferably 2)
101. ✅ Results are consistent (not wildly fluctuating)

9.2 Declare Winner


102. Click Declare Winner button
103. Select the winning variant
104. Add notes about the test (optional):

`
Notes: Benefit-focused headline improved conversions by 49.6%. Visitors respond better to outcome-based messaging. Consider applying this principle to other pages.
`

105. Click Confirm

9.3 What Happens After


106. Experiment status changes to “Completed”
107. Test stops running
108. All traffic now goes to original landing page (no automatic changes)
109. Results are archived for future reference

Step 10: Implement the Winner


Important: ShahiLandin doesn’t automatically update your page. You must manually implement the winning variant.

How to Implement


110. Go to Landing Pages
111. Edit your landing page
112. Update the content with the winning variant:

`html


Professional Email Marketing Software

Get 10x More Email Opens in 30 Days

`

113. Click Update
114. Clear all caches
115. Verify changes on live page

Why Manual Implementation?


This gives you control over:

116. When to implement
117. How to implement
118. Whether to implement partially
119. Whether to test further first

Advanced: Running Multiple Tests


Sequential Testing (Recommended)


Test one element at a time:

120. Test 1: Headline (complete)
121. Test 2: CTA button color
122. Test 3: Form fields
123. Test 4: Hero image

Benefits:

124. ✅ Know exactly what caused improvement
125. ✅ Build cumulative improvements
126. ✅ Learn systematically

Parallel Testing (Advanced)


Test different pages simultaneously:

127. Landing Page A: Test headline
128. Landing Page B: Test pricing
129. Landing Page C: Test social proof

Don’t test multiple elements on the SAME page simultaneously unless you know multivariate testing.

Best Practices for A/B Testing


Testing Strategy


130. Test big changes first: Headline, CTA, hero image
131. Test one element at a time: Isolate variables
132. Run long enough: At least 1-2 weeks, 100+ conversions
133. Reach significance: Wait for 95%+ confidence
134. Document learnings: Keep notes on all tests

What to Test (Priority Order)


High Priority (Test First):

135. Headline
136. CTA button (text, color, size)
137. Hero image or video
138. Social proof (testimonials, stats)
139. Form length (number of fields)

Medium Priority:

140. Body copy length
141. Benefits vs features
142. Pricing display
143. Page layout
144. Color scheme

Low Priority:

145. Font choices
146. Icon styles
147. Footer content
148. Minor spacing tweaks

Common Mistakes to Avoid


❌ Stopping too early: Need adequate sample size
❌ Testing multiple things: Can’t tell what worked
❌ Not waiting for significance: Random fluctuations mislead
❌ Ignoring mobile: Test on all devices
❌ Testing without hypothesis: Test with purpose
❌ Making changes during test: Invalidates results
❌ Peak Effect bias: Early results often don’t hold

When to Stop a Test


Stop when:

149. ✅ Statistical significance reached (95%+)
150. ✅ Minimum sample size met (100+ per variant)
151. ✅ Minimum duration elapsed (1-2 weeks)
152. ✅ Results are stable

Or stop if:

153. Maximum duration reached (4-6 weeks)
154. Technical issues invalidate results
155. Business requirements change

Calculating Sample Size


Need to know how long to run your test?

Simple Formula


`
Sample size needed per variant =
(Baseline conversion rate × 0.5) × 1000

Example:
If current conversion rate is 5%:
(0.05 × 0.5) × 1000 = 25 × 1000 = 25,000 visitors total
= 12,500 per variant
`

Use Online Calculators


Easier option: Use A/B test calculator

156. Optimizely Calculator: https://www.optimizely.com/sample-size-calculator/
157. VWO Calculator: https://vwo.com/ab-split-test-duration/
158. Evan Miller’s Calculator: https://www.evanmiller.org/ab-testing/sample-size.html

Input:

159. Current conversion rate: 5%
160. Minimum detectable effect: 20% (relative improvement)
161. Statistical power: 80%
162. Significance level: 95%

Output: Sample size needed and estimated duration

Troubleshooting


Test Not Getting Traffic


Problem: No visitors or very few

Solutions:

163. Verify experiment status is “Running”
164. Check landing page is published
165. Drive traffic through ads, email, social media
166. Wait longer – low traffic takes time

Conversions Not Tracking


Problem: Zero conversions for both variants

Solutions:

167. Verify conversion goal is set up correctly
168. Test conversion manually (submit form, click button)
169. Check analytics is enabled
170. Review browser console for JavaScript errors

Results Not Statistically Significant


Problem: Test running for weeks, still no significance

Solutions:

171. Traffic too low – need more visitors
172. Variants too similar – make bigger changes
173. Baseline conversion rate very high – harder to improve
174. Continue running or declare test inconclusive

One Variant Gets All Traffic


Problem: Traffic not splitting 50/50

Solutions:

175. Check traffic allocation settings
176. Verify experiment is running (not draft)
177. Clear caches
178. Test in different browsers/incognito mode

Summary


You’ve learned how to:

179. ✅ Plan and set up A/B test experiments
180. ✅ Create test variants with controlled changes
181. ✅ Configure traffic distribution
182. ✅ Monitor experiment progress
183. ✅ Analyze results and reach statistical significance
184. ✅ Declare winners and implement improvements
185. ✅ Follow best practices for reliable testing

Start testing and optimizing your landing pages today!

Related Tutorials


186. How to Set Up Conversion Tracking
187. How to Analyze Landing Page Analytics
188. How to Optimize Performance
189. How to Create Your First Landing Page

Additional Resources


190. A/B Testing Experiments Feature Guide
191. A/B Testing Issues Troubleshooting
192. Analytics Tracking Feature Guide

—

Questions? Contact support or visit our documentation.