Abstract
The League of Legends API provides a wide range of game and player information, including data on current ranked statistics. Through continuously tracking of millions of League of Legends players, we built a database to investigate ranked progression. We supplemented this data with Mobalytics usage data to get a better understanding of how ranked players improve and what impact Mobalytics has on progression.
On the NA server, we analyzed how 110,426 Mobalytics users and 1,613,396 non-Mobalytics users climbed over the ranked time period between May 3, 2018 and November 13, 2018. We found that 53.7% of Mobalytics users climbed at least one division, whereas 42.3% of non-Mobalytics users climbed at least one division, showing that Mobalytics users are 27.0% more likely to climb than non-Mobalytics users. This was consistent from Bronze to Diamond with the highest percentage increase in Bronze at 33.1%, and the lowest in Platinum at 23.0%. We also split our Mobalytics users into groups based on how much they had used Mobalytics over the user’s lifetime and found that frequent Mobalytics users were 17.8% more likely to climb than less occasional Mobalytics users, and 42.6% more likely when compared to all NA ranked players.
Introduction
Over 10 million players play League of Legends (LOL) competitively with a desire to climb the ranked ladder. Many players complain about being stuck in a specific tier and not being able to climb higher. The initial motivation behind Mobalytics was to create a tool to help all players improve by assessing their strengths and weaknesses, and then giving them guidance on how to progress.
The goal of this study was to investigate how players in LOL traditionally climb and understand the role that Mobalytics plays in helping people climb. We wanted to answer the question of how likely people are to climb based on their division and what factors are common amongst those who actually climbed. This study is based on 2018 Season 8 ranked data from the North American Server.
League Of Legends Ranking
In League of Legends, ranks are assigned through both a tier and a division. In Season 8, the tiers were, in increasing order, Bronze, Silver, Gold, Platinum, Diamond, Master, and Challenger. Divisions ranged from 5 (V) to 1 (I), often being denoted in Roman numerals, with 1 being the highest and 5 the lowest (not applicable to divisions above Diamond). Tiers give a general standing of where a player lies relative to the rest of the population, and divisions give a more granular relation of players within a tier.
On the LOL ladder, tiers are not equally populated and the graph is heavily right skewed, with the majority of players being in Bronze to Gold. This behaviour can also still be seen in the distribution for the current ranked Season (Figure 1):
Figure 1 – Ranked Distribution across tiers for Season 9, data from League of Graphs1 on May 16th, 2019. Note that Iron Tier did not exist during season 8.
Because of this, Bronze up to low Gold is often referred to as “Low Elo”, while Diamond+ is often referred to as “High Elo”. Rather than being strict definitions, these terms are more often used to give a general estimate of skill level. Low Elo refers to players that are still focusing on learning the fundamentals of the game, while High Elo players are much more affected by additional factors such as the current meta. This distinction is important to make because it’s likely that something that may help a Low Elo player climb is too basic for a High Elo player, and something that may help a High Elo player climb can have a negligible effect in Low Elo due to major mistakes in the fundamentals.
Study Group Setup
Because people that use outside resources to try to climb may have a different mentality about ranked and climbing, it’s important for us to separate our player groups into different categories. This goal is to reduce some of the bias in the user’s intent towards ranked and learning.
Since we currently only have information about players using our platform, we split players into Mobalytics vs. non-Mobalytics users, the definition of each of these groups is as follows:
- Mobalytics Users: A Mobalytics user is anyone who has registered at Mobalytics, regardless of how often they’ve used the platform.
- Non-Mobalytics Users: A non-Mobalytics user is someone who has not registered for Mobalytics.
Skill Progression
We defined progression in skill and rank according to three criteria:
- A person who goes up at least one division when comparing the starting and ending rank in Season 8 will be considered someone who climbed.
- A person who has climbed a tier is simply someone who has moved up a tier relative to when we first captured their rank. This is regardless of if they went from Silver I ->Gold V or from Silver V-> Gold V*.
- A person who has climbed a division is someone who has moved up a division relative to when we first captured their rank. This is regardless of if they went from Silver II->Silver I or from Silver V->Silver I.
Methods
10,000,000+
players play League of Legends (LOL) competitively with a desire to climb the ranked ladder.
This study is based on 2018 Season 8 ranked data from the North American Server. All data was gathered using the Riot Games API2. The total number of players analyzed is 1,723,822 with 110,426 of them being Mobalytics users. We only considered a sample of Mobalytics users who were with us before October 31st, 2018, which gave the player about two weeks to interact with our platform before the ranked season ended. Our user interaction data with Mobalytics comes from Mixpanel, and the metadata associated with each account tracks how often they’ve performed specific events.
Gathering Mobalytics User Data
We used the Mixpanel API3 to gather the metadata for all Mobalytics users. The LOL account associated with a Mobalytics user is the current account selected in the Mobalytics settings. We then filter out the users who’ve created their Mobalytics accounts up to and including the 31st of October, 2018.
With these users, we also pull information on the following total events:
- Dashboard checked
- GPI checked
- Pre Game used
- Skill Details checked
- Advice used
- Game Data updated
The total number of Mobalytics actions performed by this user is simply the sum of the total actions performed in each of those categories from their time of registration. Since this data was pulled on the 8th of March, 2019, the total data does not represent the exact number of actions performed in Season 8, but it is a good indicator of Mobalytics usage over the bulk of Season 8.
Gathering LP Data
Starting May 3 2018, we have been tracking Season 8 ranked LP data for LOL players, allowing us to get the starting and ending rank for each summoner. We then infer if they climbed, how many divisions they climbed (dropping a tier is just a negative number), and if they climbed an entire tier or just a division.
Each LOL account has an identification number associated with it, called the summoner ID, as well as an ingame name, called the summoner name. The LP data is stored internally for each summoner ID, and we used a summoner name to summoner ID map to extract the data for the summoner name associated to a Mobalytics account.
We’ve also limited our analysis to the Bronze through Diamond tiers, since the Master and Challenger tiers have only one division each and progression from one to the next does not work the same way as in the rest of the LOL ranked system. We focused on the NA region since NA alone has enough sample size for us to perform a reliable analysis with.
Results
General Climbing Statistics
In this section we aim to get a general understanding of what climbing statistics look like for Mobalytics and non-Mobalytics users.
Percentage Of Users That Climbed
The first statistics are a comparison of the percentage of users in either group that have climbed throughout the ranked season. The results can be seen in Table 1a below. Note that % climbed a tier and % climbed a division add up to be 100%, since if a player climbed but did not get to the next tier they are binned into the group that climbed a division.
1,723,822
is the total number of players analyzed with 110,426 of them being Mobalytics users.
Table 1a – Climbing statistics for Mobalytics and non-Mobalytics users (Overall, Bronze, Silver)
Table 1b – Climbing statistics for Mobalytics and non-Mobalytics users (Gold, Platinum, Diamond)
From Table 1a, we can see that 11.4% more users climb with Mobalytics compared to users who don’t use Mobalytics. Relative to 42.3% , that’s a 27.0% increase in the number of users that climbed.
Figure 2 shows the percentage of users who climbed for both Mobalytics and non-Mobalytics groups, split up into the tiers from Bronze-Diamond. It also shows the difference between Mobalytics to non-Mobalytics users, relative to the lower percentage.
Figure 2 – Percentage of Mobalytics users who climbed in each Tier (blue), percentage of non-Mobalytics users who climbed in each Tier (orange), and the relative difference between Mobalytics and non-Mobalytics users (green).
From Figure 2 we see that, although less people climb as we go up in tier, the relative difference remains significant, with the maximum being at Bronze with 33.1%, the minimum in Platinum at 23.0%, and in Diamond the value at 27.5%.
Mobalytics User Climb Breakdown
In addition to the above, we analyzed how users climbed based on how often they use Mobalytics. To do this we will split the Mobalytics user into different groups, based on how many total interactions they’ve made with Mobalytics. These groups are:
- 0-9 total interactions
- 10-49 total interactions
- 50-99 total interactions
- 100-149 total interactions
- 150-199 total interactions
- 200+ total interactions
Table 2 shows the breakdown of the percentage of users that climbed in Season 8 vs the number of total interactions up until March 8, 2019.
Table 2 – Breakdown of the percentage of users that climbed at least one division along with the average number of divisions climbed across different Mobalytics usage buckets.
If we look at the low and very low group and compare it to the medium-high to high interaction group we see that more Mobalytics interaction results in a higher percentage of users that climb. It is important to note, however, that even the lowest climb percentage in the very low group is 8.9% higher than the non-Mobalytics user reference, showing a 21.0% relative difference, and the high group is 18.0% higher than the non-Mobalytics ranked players, showing a 42.6% increase in relative difference.
27.0%
more players climb with Mobalytics relative to players who don’t use Mobalytics.
Discussion and Conclusions
Summary Of Findings
In a study of 110,426 Mobalytics users, and 1,613,396 non-Mobalytics users we found that 53.7% of Mobalytics users climbed at least one division compared to 42.3% of non-Mobalytics. This is an increase of 27.0% relative to the non-Mobalytics users. These results remained true even after we accounted for the ranked tier of the user, with the largest increase in Bronze (33.1%), and the lowest in Platinum (23.0%). Even in Diamond, the relative increase was 27.5%, which shows that Mobalytics has a significant impact on user improvement across low to high levels of play.
We then considered how these statistics change within different groups of registered Mobalytics users themselves based on usage frequency. We found that only 51.2 % of Mobalytics users who had very few interactions with the platform climbed, whereas 60.3% of the users who had high interactions with Mobalytics climbed. Furthermore, the percentage of Mobalytics users that climbed increases as people use the platform more and more.
One of the fundamental questions we had to the validity of this study was: are people who use Mobalytics inherently more serious about climbing the ladder? From the interaction analysis we also found that users who have simply signed up for Mobalytics climb more than users who have never even signed up for Mobalytics, with a relative increase of 21%. This is probably because players who sign up for Mobalytics are more motivated to climb, and are also more serious to achieve their goals. Hence, users who sign up for Mobalytics but rarely use the platform can be considered a good control group, because then may have a more serious attitude towards climbing when compared to the rest of the ranked LOL. If we compare this control group to people who use the platform more frequently, the chance of climbing increased by 17.8%. This increase goes up to 42.6% when comparing to all ranked NA players, not just the control group.
Study Limitations
It is also important to consider the limitations of our analysis, and point out potential sources of error as well as opportunities for future research. First, we were not able to line up LP gains in climbing with people’s historic Mobalytics usage. We were only able to capture total interactions over a long period of time and compare them to LP gains over that period of time. Due to the nature of account creation and Mixpanel data logging, there are also some potential errors in the window of time where Mobalytics user data was examined. This error comes from the fact that all user data was pulled after the end of the season and would include usage up until that data as part of the sample size. Therefore, Mobalytics interactions should be used more as an indicator of Mobalytics usage (which is why we created the different buckets of interaction from very low to high) over the Season 8 time period that we analyzed, rather than a direct number. For future analysis it would be an improvement to have season specific interaction counters, or even to go back and analyze each user’s history individually over the period in which we tracked their ranked progression and Mobalytics usage.
Another source of error is also related to the way that Mixpanel assigns user data. When we make a pull, it means we are looking at a user’s current summoner name and region that is linked to their account. That means that, potentially, a user could be looking at another player’s account and we could be tracking their improvement instead. This problem also goes the other way though, since a summoner could’ve changed their name after Season 8, and we would not link their Mobalytics account to their Season 8 ranked account. However, the user interface of Mobalytics is set up so that players use their own account and can track their own progress over time as well as get tips when they are in a game. If they use a different account, they would forfeit these features, so we are confident that this problem is a minor source of potential error.
Additionally, we also only considered the NA player base and only Mobalytics users who play on NA accounts. There might be differences in culture, work ethic, or climbing mentality across different regions, which could affect both how many users from both groups climbed. Since Mobalytics is only available in English, a language barrier exists in regions where there are multiple languages and/or English is not the standard language. For this reason, sticking to the already large North American sample pool was the best way to achieve meaningful results. It would be interesting to see if different regions would use Mobalytics in slightly different ways, depending on how well they can read and understand the text.
Finally, our ranked data only started on May 3rd of 2018, whereas Season 8 officially started in January of 2018. Therefore, there’s a substantial period of time where a user could’ve played and climbed, and then not have climbed more afterwards. Players usually also get placed in a lower starting rank than where they ended the previous season, which means that users may have been able to play and climb to reach their previous season rank before we started tracking. This problem, however, affects both Mobalytics and non-Mobalytics users, and could have an affect on the absolute percentage of users who climbed in both groups, but likely wouldn’t result in major changes in the conclusions, since the sample size of users we considered was extremely large.
Future Directions
Given the results of the study we can say that Mobalytics usage has a significant impact on if a user will climb or not, even after controlling for rank and motivation. However, there are still open questions, such as “Does continuing to use Mobalytics indicate a higher dedication to climbing?” and “How do users who consistently use Mobalytics compare to users who consistently use another platform to improve?” These questions could not be answered within the scope of the data collected in this study.
Future research should look to repeat this experiment for Season 9, and address some of the above listed sources of error in this study. Additionally, we would consider games played along with divisions climbed, so that it’s possible to account for number of games played having a significant impact on if/how much a user climbs. This would allow us to address three other major factors we hypothesize affect a player’s ability to climb: when players play, how long they play, and what champs they play.
We hypothesize that playing late at night or too many games in one session can lead to fatigue, which can have significant effects on performance and climb. This is common due to the strong desire of wanting to play and win, players often find themselves playing “one more game” to get that elusive win and not go to bed on a loss. On the other hand, playing too few games in a session could also lead to worse results, since users may not be warmed up properly and may not achieve the benefits of a proper session. The effect size of session start time and duration on a player’s performance could be an important finding, since it can help make personalized recommendations to users for when and how much they should play. This will be different based on each person’s previous experience and comfort levels.
Finally, we hypothesize that players who have smaller and more focused champ pools have a higher chance of climbing. Conventional coaching wisdom says that to climb you should have a small champion pool that you feel very comfortable on, which would allow you to play using muscle memory every game. Most players fall victim to the illusion of choice with the sheer number of options available with the 144 champions. They make the mistake of thinking that a champ which is believed to be strong vs the one their opponent selected will give them a better chance of winning. In reality, their ability to play a champ they are less comfortable on hurts their chances more than it helps, even if the selection is a potentially better counter match up.
References
We would like to thank all the players of LOL for providing such a rich data set for this study. All Data was gathered courtesy of Riot Games API.
- https://www.leagueofgraphs.com/
- https://developer.riotgames.com/
- https://developer.mixpanel.com/docs
If you are interested in using any of our original images, graphs, cliff notes, or infographic, feel free to check out this link.
42.6%
is the increased chance of a frequent user of Mobalytics to climb vs an average NA ranked player, and the difference between a more frequent and less frequent user is 17.8 %.