64

How to tell what advice is actually good? An experiment

by The Tank Man on The Tank Man 1674062216051


Writing another post, I realized it was long, abstract, divorced from the actual game, and probably unlikely to help readers actually improve and climb. Which is not to say I won't post it, I still like it as a rant, but you might reasonably ask of a post "is someone who reads this more likely to climb than someone who hasn't?"

Someone once wrote they liked something I posted and it helped them get to plat, and I remember thinking "well that's nice, but I wonder if they would've climbed anyways". So how do you tell advice was good?

Imo it's a surprisingly hard question. You might try to approach it with logic, reading the advice to see if it makes sense. I don't think that's enough! It might help with identifying advice to try, but without testing it against playing the game you can't tell if it merely sounds good or is actually good.

Ofc that's sort of the point of this site - put what people say next to what they do - but for telling if advice is good in general you'd need a sort of experiment.

The experiment

To address the problem of well maybe these people would climb anyways, you'd need to put some people in a control group: tell them just do whatever you were gonna do anyways good luck.You don't merely want advice that makes people climb; you want advice that makes people climb more than they would've in the control group.

Now the interesting part - what advice do you test? Even as a writing exercise this already forces more rigor than the usual bsing imo, because you have to think will this really help people. You can't get away with anything that sounds good but turns out overly complicated without much benefit. You have to find the highest value interventions.

-Mute all? Years ago turning chat off would've been my number 1 suggestion because it's extremely simple and sometimes pretty helpful. Muting seems more widely accepted and popular now though, so I'm not sure how many people it's new to. And this brings up an interesting point - advice new in 2015 might be totally known (therefore low impact) in 2025. I do still think this experiment is worth doing though, even if what helps people climb in 2023 might not be true forever, because identifying what advice is new and helpful in the current year is itself interesting.

-Champ pool? Narrowing your champion pool to a few champions is also somewhat known but imo underrated. Especially the point of it, which is that once you have a decent number of games on your champion you stop focusing on playing/learning your own champion and pay more attention to what enemy players are doing and learning the game as whole. It also seems like the sort of advice where it'd be easy to track what type of person it helps: you could plot not only the rank people started and ended at, but also how wide/narrow their champ pool was.

-Coaching? Doesn't really generalize because obviously coaches are different, but I'm very curious if any coaches could, for example, spend a few hours watching replays one on one with each person in their 20 person sample, helping them climb more than the control group and canned advice groups.

-Play support? I've seen people claim support is easier than the other roles such that when you swap from another role to support you'll go up multiple divisions. The best evidence for this is probably players like Huhi, Zven, and Busio swapping to fill an easy role without taking an import slot, but this would be evidence the typical soloq player could swap to support and climb.

-Copy a high elo player or streamer? This may overlap with narrowing your champ pool, but there are a lot of resources for learning from people good at a champion.

-Watch replays (better)? One question is if any particular approach to replays, like "what happened before your first death", is much better than simply telling people "watch your replays" with no further guidance.

-Play more games? To get good you've gotta play the game...although in the long term how much you play is only a function of how much you enjoy the game, so idk if this advice actually does anything.

One last problem: what if someone says ok, this ALL sounds good, this year I'm going to mute all, narrow my champ pool, get coaching, main support, find high elo streamers, analyze my replays, play tons of games, rank 1 here I come!! I mean...you can ask people to participate in an experiment both for great science and their own climb, but it's harder to tell people to abstain from potentially helpful advice. That might make the experiment better but hurt their climb.

The solution could be either to say well please just abstain anyways, or maybe to test advice that's costly enough to implement that you don't naturally do it all. For example getting coaching is pretty hard - you have to go find a streamer or discord or whatever - so it's an intervention people in other groups probably are not getting.

Another solution could be to measure how well people follow the actual advice. For instance if people in the "narrow your champ pool" group turn out to only narrow their champ pools by the same amount as everyone else, then what? Maybe the advice isn't necessarily wrong, but it's so well known that just telling people isn't enough to change anything.

In a way the constraint that advice has to be somewhat costly to follow makes things more interesting, because who cares if easy advice works or not, it costs you little to implement so you might as well. Whereas before embarking on a rigorous training program of waking up every morning by drinking pickle juice in a cold shower then learning Korean to translate showmaker while dodging every lobby without a perfect teamcomp, you might want to know if these painful steps are really worth the cost.

Even advice that seems simple/easy might be costly - for instance it might turn out that it takes most people a lot of effort (at least at first) to play only one champ.

Note that we've moved the goalposts from "is someone who reads this more likely to climb than someone who hasn't" to "is someone who reads this and implements it more likely to climb than someone who hasn't", which is less elegant and really less useful. Oh well. To keep it strictly about reading the advice, you'd have to keep the advice every group gets secret from the others, which might be hard or might not idk.




LegendsOfLeague isn't endorsed by Riot Games and doesn't reflect the views or opinions of Riot Games or anyone officially involved in producing or managing Riot Games properties. Riot Games, and all associated properties are trademarks or registered trademarks of Riot Games, Inc.