For years in industry conferences and in every panel and lecture, we hear the industry leaders speaking about; how critical it is that we will base any decision on data and not on gut feeling.
The problem is that up until now the A/B testing solutions were complicated, expensive and required development resources but the worst part is that the results, in many cases, are not conclusive - so there is no clear path of constant improvement in the results.
Even after crossing that hurdle of investing the time and resources, usually, it was discovered that there are very limited features that can be tested in the game and that it is almost impossible to see the full-scale effect and to take into consideration the most important and essential KPIs - IAP LTV, user-level Ads revenues, retention, conversion, ARPPU and more.
It required a lot of work from data scientists and product design teams to prepare and design the tests and the testing mechanism, while at the end of the day it was hard to decide if all that work was actually solving anything and helping improve the user’s Holistic LTV.
Sounds familiar?!
Over time we got a lot of feedback from Game of Whales users that they are looking for an easy, fast and clear way to commence game experiments, to be able to test more than 2 groups and to be able to run several experiments in parallel. But the most emphasis and clear need was getting clear, black and white results that takes all parameters into account and point the game developer to the optimal option for the experiment.
As we always tell our industry colleagues, Game of Whales is all about making game developers life better, and happier :) so we decided to make the easiest, most reliable A/B testing solution possible, without any restrictions and not too much of a hassle, just a few hours of development time and your game is good to go with any test you would like to create!
Yes we know, sounds amazing!
But we didn’t stop there.
Understanding that setting the A/B testing is only half of the way - Analyzing the results, is the real challenge even for experienced data teams, during the beta launch of this feature the interesting feedback we got from Game of Whales users, the game developers, was that they are less interested in receiving an enormous amount of data, a deep view on the user behaviour in each group, they cared more about knowing at the end of the day, which of the scenarios will bring them higher Holistic LTV, IAP & Ad revenues, without hurting their Retention KPI.
And this is exactly what we did! We eliminated all the “Too much data”, deep view, events, and long lists and just decided to present the most important KPIs for each group.
This aligned perfectly with our approach and the rest of our platform features. In Game of Whales we design every feature to require the minimum amount of work from our clients, and prefer to let the AI and ML do most of the work, while our clients just monitor the AI work and constantly see Game of Whales maximizing game monetization performance.
Yes! working with our A/B testing can be that easy.
We also thought, Hey! why should we limit our A/B testing solution to only two groups?!
Run an unlimited number of tests simultaneously with up to 10 groups per test, Game of Whales AI system will divide the users evenly into the groups according to their country and OS. So you know that the results are not affected by Geo or difference in the groups behaviour.
- Create any A/B testing without spending expensive development time.
- Test any game logic, both from client or server.
- Test different ad logics and frequencies.
- Run an unlimited number of tests simultaneously with up to 10 groups per test.
- See KPI results of each group in real time.
- Get a clear answer for every experiment - which group WON the best performance reward.
We wanted to share some of the most surprising results our clients got when using Game of Whales A/B testing tool (bear in mind that setting up a test is soooo easy that it's hard not to test the strangest things :))
- Removing tutorial increased the Holistic LTV (retention decreased, Ads revenues no change, conversion went up!)
- Showing less interstitial ads leads to higher LTV as the eCPMs went up .
- Adding banners and interstitial ads didn't affect retention/churn, but increased both IAP and Ads LTV.
- Aggressive offers in a shape of a pop up increased IAP LTV.
- Decreasing in game items prices increase IAP LTV.
- Matching different bundles flags very fast what is the most relevant on user level.
Results are shown in real time on the dashboard, with all the major KPI’s, super easy to understand what your next game change should be! And most important - you now have a chance to test anything even on a small scale (say few thousand users) and get a clear answer if it performs better or not.
Head over to our A/B documentation to read about the implementation process and the setup.
Or just send us a mail to [email protected] and schedule a demo to see how simple it is to increase your LTV!
