4 Amazing Ways To A/B Test Your App Page Creatives

If you could increase your app’s conversion rate by 71% by changing one graphic, would you do it?

If you haven’t done anything with A/B App Testing for your app you might be missing tons of free traffic.

How do you do it?

If you’re on Google Play, using Google Experiments is the obvious answer. But what if your app is only available for iOS? What if you want to run a test before launching? Let’s review a handful of ways to A/B test so you won’t have any excuse not to!

What should you actually be testing?

When deciding what to test, look for parts of your app page which are obviously neglected. For example, if you have a beautiful icon but your screenshots are weak, test out some new enhanced screenshots with captions and see what happens. People like well designed and cool screenshots you know, the one that have cool colors, text, comments and add-ons like cool characters and snap chat stickers type.

These are the average increases in conversion for each element listed in order of importance:

Where do you check your current conversion rate?


If your app is in the Google Play Store, click under the “Acquisition Reports” tab to see the conversion data on organic downloads. Google Play is also shows benchmark data of how your app performs against other apps in the same category. Which is cool.

If your app converts below the average app in your category, Mr Google will let you know about how many downloads you’re missing.

To check your conversion rate in the iOS store, open up the “Analytics” page and click “Conversion Rate”. Once this is selected, the chart will show your app’s conversion rate on a daily basis.

To make sure you’re only getting data from organic users in the iOS dashboard, remember setup a filter for either “App Store Browse” or “App Store Search”. This will ensure that you’re only collecting data for users organically that were browsing the store.

1. Before and after the testing

This one might be challenging, but it can still be useful to provide insights into which creatives perform better. Now, If you’re serious about improving your conversion (and you should be), don’t just use this method!

PROS: Super easy to do. Just upload the new versions, wait a few weeks, then compare the conversion rate before and after. It’s free.

CONS: Very inaccurate. Check your conversion rates right now and you’ll see that they bounce lot. Even when you’re not making any changes, your conversion rate might fluctuate by as much as 50% every month which can lead to incorrect assumptions. No Pre-launch testing, which means you’ll need to have the app in the store to use this method. And finally, you’ll need to publish the app and give it time to gather enough data before making any conclusions, which is time consuming.

WHY: You should consider this method if:

  • Your app is only available on iOS.
  • You don’t have the budget for paid ads.
  • You don’t have a landing page for the website you can run tests on.
  • You’re absolutely sure the new variations will improve your conversion rate.

2. Google Play Store Listing Experiments

If you’re app is in the Google Play store, then you must do this! With Google experiments, you can A/B test on things like your icon, description, feature graphics, screenshots, and promo video. Setting up a Google Experiment is super simple too!

Go to your Google Play dashboard and click here:

Then, there are two type of tests you can run:

  1. Global – Which means all tests will show to all users. These types of tests are good for assets that do not involve text (i.e. icons) since it will not discriminate anyone who doesn’t speak the language.
  2. Localized – Tests will only show to users in the targeted language. With these, you can test things like description, videos, or screenshots by making sure the people who see these assets can actually speak the language.

Once you’ve decided what type of test to run, choose what you want to test and wait for the results:

Once you have the test setup and running, it should look like this:

After that, all you have to do is wait. When the test reaches a 90% level of confidence, the test will conclude and you’ll have your data. Check Here for Google’s documentation on how to run A/B tests on your store listing if you want to dig more into it.

PROS: Very accurate, it’s free and easy to use! Since you’re running real A/B tests directly in the Google Play store, your results are going to be much more accurate.

CONS: This great tool is only available for Google Play. Note: One thing you can do is to run a test on google, gather data and apply that to your IOS listing. It might (and it should) have some margin of error since the whole infrastructure and environment are different. Still, it would be more accurate than not having a testing at all!

WHY: If your app is available for Google Play, why not?


3. Use third party testing software

If you’re app is only available for iOS and you have the budget for it, try this! Some third party software examples can be SplitMetrics, Storemaven, or Test Nest to set up mock app store pages. All pretty much the same, just some differences in the analytics reporting.

Once your page is setup, you then create several variations of it with different changes to test against each other. Example: You could create two mock app pages, each with a different icon.

From there, you will drive traffic to the variations with Facebook Ads or any traffic source of your choosing to see which one converts the best.

PROS: f you’re looking to test your graphical assets before launching, using third party software is a perfect way to do it. Once they click “install” on the mock app page, you can direct them to the landing page where you can collect their email. Be careful on the wording you use, some users might get upset that your app is not yet ready to download.

You can also test on more variables at once and use attribution links to track user behavior. Note: If you’re using a service like AppsFlyer, you can use their links to track how users behave once they download the app. This is important because different variations may attract users who behave differently and spend more money: For example, perhaps variation A converts 10% better than variation B but AppsFlyer shows that users who saw variation B spend 20% more money!

CONS: It could be expensive. On top of paying for ads you also need to pay for the software needed to run the tests. Also keep on mind that not all clicks on third party store page will convert in the app store. When a user clicks “install” on the mock app page, they will be taken to the real app page where they will have to click it again. During this process, a handful of users will drop off, meaning that you’ll end up paying more for ads running to a mock app page than you would normally do.

WHY: Consider this method if your app is only available on iOS, you want to do pre-launch A/B testing and you can afford to buy the testing software and run a paid ads campaign.


4. A/B test directly in paid ad creatives

For this tactic, you’ll be using paid ads like Facebook Ads or any ad network of your choosing to A/B test your creatives. This works a lot like using third party software mentioned above except instead of driving traffic to a mock app page, you’ll be using Facebook Ads directly to test your creatives.

This literally means, you post on your app’s page, group or even your personal profile and ask for opinion. Say something “Which ICON you like the mosta, b, c or d?” and drop a image with the options. Run an a campaign for that post and wait for the feedback!

PROS: Cheaper than with third party software since you wont need it. Also ads might be cheaper since you can run as post engagement. You can test different variables, target users opinions by language, interest and/or country. This is very useful since you can get prepare before launching the app.

CONS: Although this method will give you a good idea of what converts the best, the data will not be as accurate compared to using a real or mock app page. Users won’t see the graphics in the same context as they would on the app store and moreover, when asking an opinion, users tends to have a different perception/mind set compared when they are looking to download your app.

WHY: You should try it along other strategies and possibly the best option if your app is only on IOS.

5. Some other options…

A/B test directly on your landing page:

If you have a separate landing page for your app, you can use it to A/B test different graphics. For example, you can use a service like Optimizely or Google Optimize to run tests on icons, screenshots, descriptions, title, or even your pricing directly on your landing page.

Source: Turnplay

In-app offer walls:

This one will only work if you have other apps you can use to cross promote with.

How it works is you would set up an app offer wall in another app you own to cross promote the app you want to test on. From there, you will A/B test this icon using a service like Apptimize or Optimizely.

Run an A/B testing poll:

Share your graphics with a closed group of people and ask them to vote for their favorite one. You can run tests like these in places like Facebook groups, Subreddits, or even by showing variations to friends and family.


There is no reason for you to not even try this! Just consider that when testing variables, you can also test color patters in different shades and tones. In most cases that would be time consuming and digging too much into details, but you will be surprised that some colors and patters does really impact sales and engagement! A/B app testing requires time and patience so keep the hard work!


Share this post