A/B testing, the only marketing lever that doesn't lie
In consulting, I've watched SMEs lose months deploying "best practices" lifted from US blogs: "Tuesday emails convert 23% better", "Long subject lines beat short ones". When you test those claims on a real Senegalese base, the result flips half the time.
A/B testing is the only honest method: isolate a variable, measure, decide. But you have to do it right. Most A/B tests I see on LinkedIn are statistically void: samples too small, false winners, confidence intervals ignored.
Here is the rigorous method we apply at Kolonell.
The sample size rule
For an A/B test on email subject (the most-tested variable) to be statistically significant, you need at least 5,000 sends per variant, so 10,000 total. Below that, statistical noise drowns any real signal.
For an 800-contact SME, that's impossible in a single send. Three options:
- Stack multiple sends until you hit 10,000 total
- Test on the full base 50/50 split across 3 consecutive sends
- Skip subject A/B and test higher-effect variables (CTA, send day)
The 5 variables to test in ROI order
| Variable | Typical effect | Min sample | Difficulty |
|---|---|---|---|
| CTA text ("Book" vs "Request quote") | +30-80% CTR | 2,000 / variant | Easy |
| Send day (Tuesday vs Thursday) | +15-40% open | 5,000 / variant | Medium |
| Subject (short vs long) | +5-25% open | 5,000 / variant | Medium |
| Send hour (9am vs 4pm) | +10-30% open | 4,000 / variant | Easy |
| Button color | +3-12% CTR | 8,000 / variant | Hard |
Always start with the CTA. It needs the smallest sample and delivers the largest lifts. A Dakar consulting SME pushed CTR from 2.1% to 5.8% over 3 successive CTA tests — nearly tripling email-generated pipeline.
Step-by-step in Brevo
1. Write the hypothesis first
"I believe CTA 'Book my audit' will convert better than 'Request a quote' because the commitment level is lower."
2. Randomly split the base 50/50
Brevo handles it natively in the campaign editor (A/B test option).
3. Lock the winning metric before sending
CTR if testing CTA. Open rate if testing subject. Never both — the false-positive risk explodes.
4. Wait 72 hours before concluding
Need a professional website?
Kolonell builds websites that attract clients, optimized for the Sénégalese market. Free quote in 2 minutes.
Weekend opens represent 15-25% of total in Senegal.
5. Check statistical significance
Use an online calculator (Evan Miller's A/B Calc, AB Testguide). If p-value > 0.05, the test is inconclusive — don't deploy.
Real case — Dakar e-commerce SME
Base: 3,200 active contacts. Goal: lift average CTR from 2.3% to 4%+.
8-week test plan:
- W1-2: CTA "Buy now" vs "See the product" → winner "See the product" +42% CTR
- W3-4: Tuesday vs Thursday day → no significant difference
- W5-6: 30-char vs 60-char subject → winner 30-char +18% opens
- W7-8: 11am vs 5pm send hour → winner 5pm +21% opens
Final CTR: 4.7%. Monthly email revenue: 920,000 FCFA → 1.8M FCFA. Kolonell investment: 480,000 FCFA for the test plan + reporting.
FAQ
When can I A/B test seriously?
From an active base of 2,000 contacts for CTA tests. 5,000+ to test subjects and send days.
Is free Brevo enough for A/B testing?
No, native A/B is on Lite (19 EUR/mo) and above. On free, you manually split the base into two lists and run 2 campaigns.
Do I need a statistician to analyze results?
No. Evan Miller's A/B Calc (free, online) computes the p-value in 10 seconds. If p < 0.05, the winner is valid.
How much does Kolonell charge for a 2-month A/B test plan?
350,000 to 600,000 FCFA depending on test volume (4 to 8 experiments + weekly reporting + recommendations).
---
Want a rigorous A/B test plan delivered within 7 days? WhatsApp +221 77 596 93 33 or free quote.
Mohamed Bah
Fondateur, Kolonell
Passionate about digital and entrepreneurship in Africa, Mohamed has been helping Sénégalese businesses with their digital transformation since 2020. Founder of Kolonell, he believes every SME deserves a professional and accessible online présence.
