Company X wants to know how long players are spending playing different modes of their new third-person shooter game. The game has four modes: single-player campaign, online multiplayer, and local and online horde modes. An hour before Company X’s weekly update meeting, a researcher is asked to show data representing the time spent (in hours) playing these game modes during the game’s first week following launch from a small random sample of players. A time-spent analysis is important and of interest because it demonstrates what activities/game modes players are engaging in and for how long, which can help guide game development during production as well as post-launch to ultimately see if the game is matching design intention.
The researcher was not worried about generating such data with little prior notice, because Company X collects an exuberant amount of data via telemetry. The figure below was presented along with the statistical analyses of the data, which were paired-samples t tests because of the within-subjects design (each group consisted of the same players).
Furthermore, the researcher calculated confidence intervals (the I-shaped bars on the graphs), which are ranges of scores constructed such that the true population will fall within its range in 95% of samples, for each group. The true mean would simply be the average time spent in each of the game modes of every player to play the game during its first week (as opposed to just the small random sample used in this example). Since the researcher doesn’t know the true mean of the entire population of players, he/she doesn’t know if the sample values (means) here are a good or bad estimate of this value. So, rather than fixating on these four means in the sample, the researcher could use an interval estimate instead, utilizing the sample means here as the midpoint, as well as setting a lower and upper limit.
Essentially, the researcher calculated upper and lower limits for each game mode in the sample. Again, since this is only a small sample of the population of the players of Company X’s new game, we do not know the true mean of the population. However, for example, if Company X gathered 99 more different samples of players to generate 99 more figures similar to the one below (and calculated 95% confidence intervals), the researcher could confidently say that in 95/100 of those samples, the true mean of the population of players would fall within that range.
The small random sample consisted of 23 players and their average time spent in each of the four game modes: single-player campaign, online multiplayer, and local and online horde modes. While the figure alone can show differences visually, further analyses can validate whether there are statistically significant differences between the modes.
Based on this sample, players spent more time, on average, playing single-player campaign (M = 7.37, SE = .900) than local horde mode (M = 1.53, SE = .455). This difference, 5.84 hours, BCa 95% [3.875, 7.803], was significant, t(22) = 6.166, p = .000. Players also spent more time playing single-player campaign than online horde mode (M = 3.87, SE = .614). This difference of 3.5 hours, BCa 95% [1.180, 5.820], was significant, t(22) = 3.128, p = .005.
While there was no significant difference in time spent playing single-player campaign compared to online multiplayer, players spent more time, on average, playing online multiplayer (M = 8.35, SE = .857) compared to local horde mode. This difference of 6.82 hours, BCa 95% [4.727, 8.908] was significant, t(22) = 6.763, p = .000. Players also spent more time, on average, playing online multiplayer than online horde mode; the difference of 4.48 hours, BCa 95% [1.844, 7.013], was significant, t(22) = 3.664, p = .001.
Additionally, players spent more time, on average, playing online compared to local horde mode; the difference of 2.34 hours, BCa 95% [-3.674, -1.004], was significant, t(22) = -3.634, p = .001.
Conclusion
To summarize, players spent more time playing both single-player campaign and online multiplayer than both local and online horde modes. Additionally, they spent more time playing online horde mode than local horde mode either by themselves or with friends. Again, the confidence intervals are an important piece of information here, because they allow Company X to confidently assume what the true mean of the player population would be for each game mode, and, ultimately, generalize findings based on this small random sample of players.
For more on paired-samples t tests and confidence intervals:
Field, A. (2013). Discovering Statistics Using IBM SPSS Statistics. London, England. SAGE.