Skip to content

A/B Thumbnail Tests 🧪

Not sure which version of your game’s thumbnail will perform best? We can help you run an A/B thumbnail test during the Soft Release phase of your game to find the best performer.

How Does the A/B Test Work?

An A/B thumbnail test compares two versions of your game’s thumbnail by showing each one randomly to different players on the platform. This helps us understand which version attracts and retains players better. Each test round runs for a minimum of three days to ensure we gather enough reliable data.

If you’d like to start a test, you can share up to three thumbnail options with us via email. Send the thumbnails to developersupport@poki.com and we’ll help you select the top two to test first. We limit the number of tests you can run to ensure you’re focusing on high-quality, well-considered designs. Since we can only test two thumbnails at a time, running too many rounds can unnecessarily prolong the soft release phase. Our goal is to help you find the most effective thumbnail efficiently, without delaying your game’s path to global release. Once the test is done, we will share the test results, including all the metrics, with you via email too.

What Metrics Do We Track for A/B Tests?

We track multiple indicators to evaluate which thumbnail performs better:

  • C2P (Conversion to Play Rate): Indicates how many players start playing after clicking. Most of the time, the C2P results are pretty close between thumbnails, but in general, a higher C2P suggests the thumbnail sets the right expectations about the game.
  • Average Playtime: Shows how long players stick around. Attracting players is the first step and one important aspect of the thumb, and another important aspect is to show that the exception created by the thumbnail matches the real game experience, so players choose to stay and play the game.
  • Video Ad Impressions: Tells us how often ads are viewed during sessions that start from each thumbnail. This generally correlates with longer and more engaged play sessions.
  • Total Playtime (Total Playtime = C2P × Average Playtime): This is our most important metric as it tells us which thumbnail attracts and retains players best overall, making it the best representation for your game.

When Do We Run A/B Tests?

We only run A/B tests during the soft release phase, as it’s the ideal moment to evaluate how different thumbnails perform. At this stage, your game is already receiving a moderate traffic boost, which provides a solid volume of data to work with. You’re also in the process of gathering player insights and making final optimizations before the global release. Testing during this phase ensures that you’re setting your game up for long-term success with the most effective visual presentation.