1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

[TUTORIAL] How A/B testing earned me an extra $10,000 in 2015

Discussion in 'Making Money' started by healzer, Apr 19, 2016.

  1. healzer

    healzer Jr. VIP Jr. VIP

    Joined:
    Jun 26, 2011
    Messages:
    2,703
    Likes Received:
    2,361
    Gender:
    Male
    Location:
    Somewhere in Europe
    Home Page:
    Hi guys :)

    Today I feel like doing something great, but also sharing it with my beloved BHW family.
    So instead of making some ebook and selling it for $99 on WF, I'll be posting my most important experience from the previous year.

    Long story short: Thanks to some simple tricks I was able to get an additional $10,000.- in pure profits in 2015 from one source.

    This is how it all started (early 2013):
    Those who are close to me know that I'm very much into A/B and MultiVariate testing (this is the simple trick/tactic).
    It all started with the release of YouTube Calculator, which is a tool I'm listing on BHW's marketplace. The idea was to test various headlines and then tracking how many people clicked the "buy" or "order" button. I don't remember exactly what the headlines were like, because it was just a test and I don't have a backup from that website now, but it looked something like this:

    "The ultimate YouTube calculator tool!" vs "Dominate YouTube today!"

    After 100 clicks I saw a pattern emerging: the second headline had about 60 clicks, while the first headline only received 40 clicks. Okay... a test based on just 100 clicks isn't really valid, but I made an assumption on the stats at hand. Now think for a second... having 20 more clicks because of a slightly different headline?? SERIOUSLY?!!!
    Back then the tool was sold for $7, so those that's $140 extra each month, just because of a damn headline lol...!

    ----- side note -----
    "How did you do the testing?"
    In the old days I was just using php; there was the landing page and a checkout page. When a user visited the landing page, the php code would select a random headline (from an array), store a cookie identifying that headline, and printing it on the page. When the user clicked the button -- then the checkout page loaded -- another php script would then read the cookie's value and write it into a file. After 100 clicks I processed the file, by counting the occurrences of each variation -- that's how I got those 60 and 40 clicks I mentioned above. Here's a sample of a code (feel free to use it).

    1) Landing page (index.php)
    The code below contains an array of my <H1> tag variations (headlines), then it'll pick a random variation and store it in the cookie (you can choose a cookie name ofc).
    Code:
    <?php
    $versions = ["The ultimate YouTube calculator tool!", "Dominate YouTube Today!"];
    $random = array_rand($versions);
    setcookie("unique-cookie-name", $random, time()+31536000,'/');
    echo "<h1>$random<h1>";
    ?>
    
    2) Checkout page (order.php)
    This will check if the user has a cookie set, upon arriving on the order page; and it will append the value of the cookie to the file (on your server).
    You can also write other stuff such as their IP address etc... This is not a coding tutorial, so I won't go deeper on PHP stuff :)
    Code:
    <?php
    if (isset($_COOKIE["unique-cookie-name"])) {
    $variation = $_COOKIE["unique-cookie-name"];
    file_put_contents('./tracking.log', $variation.PHP_EOL, FILE_APPEND | LOCK_EX);
    }
    ?>
    
    Finally you can use your favorite text editor to analyze your tracking.log file (ctrl+f and count how many time each headline occurs in that file).
    You can also store all this info in a database, do some data mining on it and whatever you can think of... that's up to you...

    !!!!!!!!!!!!!!!!!!!!!!!!!!
    If you only have 10 clicks then -- u won't have much trustworthy data; BECAUSE:
    a) we are generating the headlines in a random manner, not a round-robin fashion; there exists a (small chance) that the first 10 visitors will only see your first variation. So your 10 clicks will contain faulty data.
    b) 10 results isn't statistically significant, even 100 isn't much of a deal, but 500 is another story. So get as much data as you can in a short period of time.
    !!!!!!!!!!!!!!!!!!!!!!!!!!

    Then sh!t got crazy (4th quarter 2013):
    Most of you know me from my Pinterest software business on BHW, and it was a big game changer.
    It gave me insight into A/B testing. I wasn't very serious with the testing for my YouTube soft's website... But now things changed. I realized, looking back at those numbers that I had to do something, more seriously. I started experimenting with the logo. This was in the those early days...

    The logo

    It's very important for a startup business/product/service to test the brand, especially the logo; because YOU DO NOT DECIDE which one is the best one, but the MARKETPLACE (your customers) will decide which one looks the best!!! -- so instead of ASKING them (through surveys) which logo looks the best (everyone has an opinion; but opinions don't mean shit around here) --> no... you TEST them :) Ok I'll be honest, I know photoshop but I'm no expert, I suck at photoshop and it's not easy to find someone who can get you 4 logo's in less than a day for a small price (I didn't want to pay $200 back then)... my hustler mentality switched on and I developed 4 logos myself; they weren't bad, but oh god... I don't want to show them to you right now haha.

    Even at this stage I didn't take A/B testing very seriously: because my only concern was finding a good logo to use... that's it. I had no clue that I was actually doing professional A/B testing; for me it was just a test to find the best looking one (actually, I was caught up with improving the software & improving the sales page; and NOT focused on gathering data). Subconsciously I knew I had my php script installed, just as previously, and after about 3-4 weeks I peeked at the results; what I saw was that there was one logo that had gotten me more sales than the other ones. Cool right? The end result for me was that I had a logo... then I just threw away the data... #YOLO lol.

    I don't exactly remember, nor do I still have those numbers (stupid me...), but that one logo performed about 20% better than all other 3 logos.
    That's how I closed the year 2013, and 2014 began. Much of 2014 was spent on further product development, without any A/B testing (because I was too blind back then to realize what it really meant)...

    #-*BAM*@

    I hit a pole and I heard a voice, it was telling me...

    No, that's now how it was lol.

    I don't remember hitting a pole, but I do remember what happened mid 2015: I listed the tool on JVzoo (just for the sake of having an affiliate system). The upside was that they offered some great "buy" buttons !! And they offer about 10 buttons for you to choose from... I couldn't pick which one to put on my sales page... But wait! We can test the 3 best looking ones! Right?? That's what I did !
    Here's what they looked like:
    ***best looking ones: based on my worthless opinion.

    [​IMG] [​IMG] [​IMG]
    Which one do you think got more sales??

    But before I spoil the results... let me tell u a bit more about this test...

    I have been running this test for quite some time, it wasn't a 30 day test... I have also used it on the latest software launch (February 2016).
    So this data is actually based on two campaigns; for the same product. This exact same test (with these 3 buttons), I have tested them on 3 different landing pages (from mid 2015 until March 2016)... I am still, even today, blown away by the results!

    SPOILER ALERT ---> It was the third one (the black one).
    I've created a pie chart representing the clicks/conversions for all three buttons:

    [​IMG]


    #1 has 27.8% of the clicks
    #2 has got 34% of the clicks
    #3, wins with 38.2%

    So imagine you were in my spot... (close your eyes)...
    You have a $100 product.
    You make 30 sales per month.
    Your monthly revenue is $3000 (pure math).
    - You then start playing around & testing stuff on your website (such as your buy button text/image)...
    - Apparently your new buy button gets 4.2% more clicks than the original version (a very legitimate scenario)...
    - Those 4.2% EXTRA conversions result in 1-2 extra sale(s) each month; that's an additional $200 in your pocket...
    - That's an additional $2400 per year; and enough money to go on a decent vacation, for many of us!
    *****
    In my case, A/B testing headlines and buy buttons added me an estimated value of $8000 in 2015 (since I started testing mid 2015). What if I started the test much sooner and made my discovery much sooner in my career? --> for 2015, that number would've been a $16000 extra!

    *ehm* if you're using JVZoo buttons, don't think that the 3rd image above will also be your winner. It probably won't, so do your own testing!

    TL;DR
    I told you my story & experience... allow me to give you my final two cents...
    Creating more value to your business doesn't start with getting more traffic! You will be blown away if you knew how many sales I make and that I have less than 100 daily visitors; no joke, take a look:
    [​IMG]

    In some industries, such as AdSense or affilites it's important to have A LOT of visitors so you can ramp up your conversion rate (get more clicks/sales/leads). But it all depends on the quality of your traffic. If your traffic is cold, then you'll have a hard time. But most of my traffic are people who were sold on the software even before they hit the sales page; so the conversion rate is VERY high -- even though it's high, there is always room for improvement! That's what I believe. The devil is in the detail after all.

    1. Creating value; getting more sales; making more money... it all starts (at least for me) with GREAT customer support! Even thought I'm a human and I can't be online 24/7, but I do my best to help users/clients as much as I can; I don't even ask if they are a customer, the fact that I CAN help someone solve some problem is enough for me. What else would I be doing with my time? Playing video games? No thanks, I rather help someone out, no matter how small the problem is.

    2. What I learned from online marketing is that you have to test, test & test. Test the most basic things; you don't need to create 5 different landing pages (although if you're serious then you should); but for many of us, it's more important to get some quick cash flow going, before spending more time doing dramatic setups & deep/complex funnels. What's the purpose of having a perfect 10-layer funnel if nobody will buy at the end? Test the market! Test what they like & dislike; learn from that and move on!

    FAQ
    Should I test my Amazon affiliate button-links? YES.
    Should I test my Clickbank buy buttons? YES.
    Should I test my e-commerce order/checkout buttons? YES.
    Should I test my headlines? YES.
    ...
    Test early on, test from day 1; what you don't know is probably hurting you.
    This is your business, it's your website... you should know EVERYTHING about it.
    You should know exactly why you are using button image X and not button image Y. Don't give me the excuse "oh I like X more than Y, that's why I use it"... Don't listen to your web designer for advice, they are DESIGNERS, not conversion experts. But you are!

    /// I hope have inspired someone today; especially the fact that you have to do the simple things extremely well.
    /// Q&A -- ask anything, I'm here to help :)

    Cheers!
    Ilya (aka Healzer)
     
    • Thanks Thanks x 20
  2. gabixxx

    gabixxx Junior Member

    Joined:
    Dec 23, 2007
    Messages:
    135
    Likes Received:
    70
    From your case study, it would seem that you have done extensive split tests on JVZoo...

    And yet both the products listed under your name in JVZoo (Pinbot and Youtube Calculator) have less than 10 sales each...
     
  3. nisdee

    nisdee Regular Member

    Joined:
    Nov 16, 2014
    Messages:
    354
    Likes Received:
    122
    Location:
    Rainbow Land
    Good read !

    Wow today I've read 5 sick threads (including yours) from people who really invested their time to write something useful and interesting, that haven't happened in a long time

    Big thanks from me :)
     
  4. healzer

    healzer Jr. VIP Jr. VIP

    Joined:
    Jun 26, 2011
    Messages:
    2,703
    Likes Received:
    2,361
    Gender:
    Male
    Location:
    Somewhere in Europe
    Home Page:
    That is true.
    I have many people promoting those products, but they didn't make much sales.
    I do have a couple of partners who are generating a decent amount of sales, but those are unlisted (private listings).
    I'm only using JVzoo's buttons, not their links.
    The reason is because they take a % form each sale; I don't like that --> my default checkout is PayPal.

    Does this answer your question?

    Thanks bro! :)
    Could you PM me the other threads that inspired you?
     
  5. dgusic

    dgusic Regular Member

    Joined:
    Feb 28, 2015
    Messages:
    328
    Likes Received:
    166
    Tl;dr. Test everything
     
    • Thanks Thanks x 1
  6. ToxicBlack

    ToxicBlack Regular Member

    Joined:
    Mar 25, 2016
    Messages:
    223
    Likes Received:
    57
    Occupation:
    Programming custom bots and tools.
    Location:
    botland
    Well, this will be reason why my eshop getting much worse results with new theme than with old. I need to do some testing. Even I thought about it in the past. Such a shame on me!
     
  7. healzer

    healzer Jr. VIP Jr. VIP

    Joined:
    Jun 26, 2011
    Messages:
    2,703
    Likes Received:
    2,361
    Gender:
    Male
    Location:
    Somewhere in Europe
    Home Page:
    Yea it happened to me many times before!
    Although A/B testing an entire theme is a horrible operation; but you see for yourself which results your theme brought, it's always an option to go back to the previous one :)
     
  8. and5rey

    and5rey Regular Member

    Joined:
    Oct 17, 2011
    Messages:
    279
    Likes Received:
    28
    Congrats on your success and thanks for sharing this with us! BTW, if you use WordPress, the Divi theme from ElegantThemes has already integrated A/B testing options in its page builder (since Divi 2.7), so you could try this, too (you can read more about it here: http://www.elegantthemes.com/blog/theme-releases/divi-2-7).
     
  9. Zwielicht

    Zwielicht Moderator Staff Member Moderator Jr. VIP

    Joined:
    Aug 31, 2013
    Messages:
    7,462
    Likes Received:
    13,127
    Gender:
    Male
    Occupation:
    Death
    Location:
    Riverside, California
    Home Page:
    Nice share.

    There's so much of an emphasis on just "getting traffic" that not as many people consider optimising the traffic they already have. Hopefully this will help out some of those members who are dealing with low CTRs and conversion rates on both their ads and their landing pages.
     
  10. soccerlover

    soccerlover Jr. VIP Jr. VIP

    Joined:
    Jun 12, 2014
    Messages:
    3,378
    Likes Received:
    1,754
    Gender:
    Male
    Occupation:
    Seo Analyst :D
    Location:
    ♥♥♥ BHW ♥♥♥
    Home Page:
    Good Share actually this will improve my CTR for sure!
     
  11. Adrian112

    Adrian112 Supreme Member Premium Member

    Joined:
    Jan 16, 2014
    Messages:
    1,373
    Likes Received:
    2,642
    Nice, A/B testing is something you can apply to every branch of your business and you can always improve something.
     
  12. healzer

    healzer Jr. VIP Jr. VIP

    Joined:
    Jun 26, 2011
    Messages:
    2,703
    Likes Received:
    2,361
    Gender:
    Male
    Location:
    Somewhere in Europe
    Home Page:
    Yes exactly!
    Everyone is trying to get more & more traffic in the hope to get more earnings; but if they would just take some time to optimize what they already had, it would be a life changer :)
     
    • Thanks Thanks x 1
  13. ChanzGrande

    ChanzGrande Elite Member

    Joined:
    Feb 16, 2008
    Messages:
    2,487
    Likes Received:
    1,178
    Occupation:
    Accountant
    Location:
    Northern Woods Counting Money
    This is a great deal of information! Glad to see how much you learned in the past few years, and that you would share your story with the rest of us. Congrats on your success as well.

    Having worked in direct response for most of my adult life both online and in traditional settings the value of split-testing cannot be over-stated. I can't recall the number of times myself or my clients started with an unsuccessful campaign, and gradually tweaked elements of it over time via split-testing until these became serious bread winning opportunities. Without the split-testing activities all of these campaigns would have ended up in the trash bin, but thanks to effective side-by-side comparison data (and having the budgets to acquire the data), we were able to turn losers into winners.

    So this activity is often more than just a peak optimization strategy. Sure it works wonders on already successful activities, but it's also the ideal way to find winning campaigns to begin with, and is largely the premise behind most forms of online advertising. If you're doing IM in any regards, and not split-testing as many variables(landing pages, images, article content, headlines, ads, auto-responder followups, etc...) as possible, then you are leaving MOST of your money on the table.

    Thanks OP!
     
    • Thanks Thanks x 1
  14. Shadexpwn

    Shadexpwn Elite Member

    Joined:
    Sep 12, 2010
    Messages:
    1,671
    Likes Received:
    452
    Gender:
    Male
    Occupation:
    Professional Showrunner
    Location:
    In the Shade
    Over the broad scale A / B scaling is a viable way to increase income with little risk.

    Even switching affiliate networks to variate the same offer with somebody else would surely lead you into a better understanding of what to be promoting to your audience.
     
  15. healzer

    healzer Jr. VIP Jr. VIP

    Joined:
    Jun 26, 2011
    Messages:
    2,703
    Likes Received:
    2,361
    Gender:
    Male
    Location:
    Somewhere in Europe
    Home Page:
    That's actually a very nice way of putting it into perspective!
    Thanks for the insight :D
     
  16. Ste Fishkin

    Ste Fishkin BANNED BANNED Jr. VIP

    Joined:
    May 14, 2011
    Messages:
    2,058
    Likes Received:
    10,211
    Came here expecting guru retardation... left with knowledge.

    good job OP :)
     
    • Thanks Thanks x 4
  17. healzer

    healzer Jr. VIP Jr. VIP

    Joined:
    Jun 26, 2011
    Messages:
    2,703
    Likes Received:
    2,361
    Gender:
    Male
    Location:
    Somewhere in Europe
    Home Page:
    Lol you are the last person I thought could actually learn anything new around here.
    Jokes aside, thanks :)
     
  18. healzer

    healzer Jr. VIP Jr. VIP

    Joined:
    Jun 26, 2011
    Messages:
    2,703
    Likes Received:
    2,361
    Gender:
    Male
    Location:
    Somewhere in Europe
    Home Page:
  19. abhi007

    abhi007 Jr. VIP Jr. VIP

    Joined:
    Aug 31, 2010
    Messages:
    5,853
    Likes Received:
    3,939
    Location:
    Theatre of dreams :)
    Hey don't forget the subscription thing I told you.
     
    • Thanks Thanks x 1
  20. healzer

    healzer Jr. VIP Jr. VIP

    Joined:
    Jun 26, 2011
    Messages:
    2,703
    Likes Received:
    2,361
    Gender:
    Male
    Location:
    Somewhere in Europe
    Home Page:
    Don't worry my friend, we are preparing good material before bringing it out :)
    I'm glad you're interested in serious stuff, it says a lot about you !
     
    • Thanks Thanks x 1