What's LUCK got to do with it? (2)

 

A few days ago, I itemised the major ways in which LUCK manifests itself in our game, defining the concept as 'positive or negative points swings arising from entirely unpredictable events'.

I pointed out that I believe LUCK plays a huge role in Fantasy Premier League (probably far larger than most participants would like to admit).

In this follow-up, I want to go over the relationship between skill and luck in the game, and how they appear to be distributed across the community of FPL players.

The Scope of LUCK

So, how much is whatever success we achieve in the game down to luck, and how much can be credited to skill?

Three observations are relevant here:

i)    Most players 'plateau' after a certain number of years playing the game: they improve little by little, year on year at first; but then they reach a consistent level. You would expect luck to be completely random; it shouldn't grow steadily for a while, and then suddenly become stable. So, I believe this early development trajectory that almost all FPL managers (the better ones, anyway!) exhibit is a fairly accurate indicator of their level of skill.

ii)   After players have reached this plateau, their variance over the years usually remains fairly small, their scores almost never straying up or down from their average by more than 150 or 200 points, and in many cases by much less than 100 points. Those year to year differences might be accounted for by fluctuations in the total points produced in a season (some years are significantly richer than others: the global average scores are usually a pretty good guide to this - unfortunately, FPL doesn't aggregate the weekly averages into an annual total for us, but you can add it up for yourselves), they might be caused by small variations in a player's skill (or, more probably, in the amount of effort and attention they put in), or they might be mostly down to variations within the normal range of 'average' luck.

iii)  People's best-ever scores are often 200-300 points above their typical average 'plateau' score.... or even above their previous best score. This suggests that the range of variation for 'above average' luck is ± 300 points per season. (And the range of possible variation for 'exceptional luck' may be far greater still....)

My feeling is that one's points return (as a proportion of whatever the 'global average' for a given season may be) probably varies little or not at all for reasons of skill - once we've learned our way around the game. 

Once we've allowed for those annual shifts in the 'points available', our individual annual totals - deriving from skill - will tend to be extremely stable. But they will often vary - sometimes quite substantially - for want of attention to the game. All sorts of life circumstances may occeasionally prevent us from devoting much time to football. (This is my main handicap. I live in an adverse timezone, with poor local TV coverage; and I travel a lot. Watching live games is difficult for me; sometimes. even catching up with highlights shows is impossible. And when I'm on the road, I occasionally even miss a weekly deadline to update my team.)

If we factor in those two constraints, that it may have been an unusually high-scoring or low-scoring season, and that we may have knowingly made some missteps through not always being able to focus on the game enough....  the variation from one year to another will be very, very small; perhaps ZERO.

Or.... it would be, but for LUCK. And we see so many people's scores - good, consistent manager's scores - regularly swinging up and down by 50 or 100 points or more each season, I think this must be inescapable evidence of luck operating at its mildest level.

For most people, most of the time, their luck in the game is going to be pretty 'average'. And if they do get a few weeks of spectacularly good luck, these will, most of the time, be more or less balanced out by a few weeks of spectacularly bad luck: the net PLUS or MINUS contribution of luck over the season won't be that great... most of the time. But in a small proportion of cases, it can be much bigger; it can occasionally be HUGE.

How much are results attributable to skill alone?

If the above analysis is correct, we ought to be able to get a fairly good sense of most people's general skill level from finding their average 'plateau' score - once their performance has appeared to be fairly stable for at least 5 years.

From randomly scanning the top few 10s and 100s of thousands quite extensively over the last 5 years or so, I have inferred that the peak skill level possible probably corresponds to an annual points total of around 2,400 (a little above or a little below, depending on those inevitable wobbles in 'available points' from year to year).

And across the board, for all skill levels there will be an identifiable corresponding points score (if we can filter out those two distorting variables of up-and-down Premier League performance and sheer luck). 

And I would imagine (although this is not provable; at least not without an awful lot of data-crunching, which is not possible for me) that there are probably rather more 'unskilled' than 'skilled' players in the game. Well, I think that's bound to be true, simply because there are such large numbers of debutants entering the game every year. But even among moderately experienced players, there seem to be an awful lot who obviously don't have much idea what they're doing. I think we'll probably see that something more than half of FPL managers are at least slightly 'below average' in their skill level. So, if we represent that with a distribution curve, we won't see a nice symmetrical 'bell' shape as in the typical illustrative 'normal' distribution...

No, the curve for FPL skill distribution will have its hump displaced a little to the left of its median point.

Moreover, there is a limit to how bad you can be (unless you're deliberately trying to achieve a low score rather than a high one; there are some dingbats like that out there...), and it's substantially more than a zero points score. So, the left side of the curve will drop off quite steeply, and won't approach zero.

The right side of the curve, however,.... well, that might be a little contentious. My hunch is that it's likely to be a little more extended than the left side; but there is still a hard limit (even if we can't see it), and while the drop-off will be less steep, I don't think it will be very tapered; there are limits to how much you can know about the game and how sharp your judgements can be. I don't believe there is a rare, 'superhuman' level of FPL skill found only in a handful of players. (If there were, we'd see those players consistently maintaining an average score above 2,500 or 2,600... but we don't find people like that.)

The skill curve for FPL probably looks something like this...

What about a curve showing the distribution of luck?

Well, in a fair world, we'd like to think that it would look pretty much like our first example: the nice neat 'bell curve', showing an equal amount of good and bad fortune.

Alas, no: I don't think so. Call me a pessimist or an embittered man (fair enough...), but the subjective experience of luck in the game for most of us certainly seems like there's much more BAD than GOOD. 

Now, probably to a large extent this is a misperception arising from cognitive biases. We want to take credit for the good things that happen in the game, so we convince ourselves that we foresaw those and deserved them and they weren't really lucky at all; whereas we bitterly resent every setback, and tend to overexaggerate its impact.

Nevertheless, I think there may in fact be at least a little bit of a skew towards the less-than-average side of the median here too. And, as with the skill curve, there's definitely a lower limit which will compress the tail-off on the left side of the graph. (It might be just about theoretically possible to score less than zero in one gameweek [though I doubt if anyone has actually suffered that]; but not for a whole season!)

Thus, I contend that our luck curve probably looks fairly similar to our skill curve - with the hump displaced slightly to the left; the tail-off on that side greatly compressed, more gently sloping on the right.

On the right side of the luck curve, however, there is no obvious limit. In practical terms, clearly there must be one; it can't tend towards infinity, because there is a limited pool of points available in the game and thus a maximum possible score.... if you were perfectly lucky all season (and I'm fairly sure we're never going to see anyone get anywhere near that limit). [This, by the way, was one of the main reasons why I chose to discuss 'luck' and 'skill' in correlation to actual points scores. If we used a more subjective measure of defining 'luck', such as fairness/unfairness, the curve could stretch to infinity in both directions!]

Our luck curve, then, won't have exactly the same profile as the skill curve: its shape on the left side might be a little less squashed; and on the right side, I think it's going to be much more spread out, with a gently descending slope.... and a very, very long thin tail.

Our point on the skill curve is what we're really interested in: we want to feel validated - we want to know how good we are, how much we deserve our points and ranking success.

But that pesky element of luck interacts with the skill curve, adding or taking away points if we've been more or less lucky than 'average'.  And we don't simply overlay those curves on each other matching at the median; the median of the luck curve has to slide along the skill curve so that we can see how much impact luck might have at any given skill level.

This is why it doesn't really matter that much if the skill curve has a long thin tail on the right or not. The fact is that, whatever the exact shape of the curve, there are going to be such small numbers of people at the highest skill levels that it is vanishingly unlikely that any of them will ever enjoy super-high levels of luck.

It becomes more and more likely that we will see examples of 'well above average' luck... and 'extraordinary luck'.... and 'super-colossal' luck the further back up towards the centre of the skill curve we go.  

And this effect continues on and on,  beyond the centre of the curve into the 'below average' and 'poor' zones of the skill distribution. It is here that that long thin right-tail to the luck curve might make itself most vexingly felt: some really quite bad players, maybe not many but some, can potentially reach the uppermost levels of the global ranking just by having outrageous luck.

That means that you're going to find the highest concentration of 'very skillful' players in the global rankings around about the 2,400 points mark, and perhaps 100 points either side of that; and such players are going to become increasingly rarer as you move up through that last 200-300 points towards the global champion. I suspect they'll be in a minority above 2,500 or maybe even 2,450; and certainly starting to get pretty thin on the ground much above 2,600. (And that's in a 'normal' year. As I pointed out in my earlier post on this topic, last year was a year of so many freak events that the impact of luck was much greater than usual. And so the proportion of quite average, or even below-average players in the top half million, and especially in the top 100,000, was extraordinarily high.)

I'm not saying that the global champion from last year - or any year - is no good; or that the top 100, 1,000, 5,000 or whatever are no good. But it is possible - likely, even - that a majority of people at those upper levels are not the most skillful, but are just 'quite good' or 'average' or perhaps even 'a bit below average' players who happened to get very, very lucky that year. And no-one can reach that kind of eminence without enjoying way above-average luck over the season.

But wait - it's actually even worse than that. I would suggest that the more skillful you are in the game, the less likely you are to benefit from good luck.

Hear me out. If you're skillful, you make only good choices. If good choices produce good results, that's not entirely lucky; you anticipated and deserved some of that points haul. (Less skillful players might have made the same pick on impulse, or following someone else's recommendation, without realising how good a choice it was - or not understanding why it was. For them, the benefits were a pure windfall, 100% luck!)  But if, on the other hand, good choices produce bad results (as they all too often do), that's really very bad luck.

By being good at the game, you're necessarily setting yourself up to experience more bad luck, and less good luck. 

And the converse is true of the weaker players. Luck tends to have its biggest impacts with really bad choices that unexpectedly come good. (I'm not suggesting that anyone should make deliberately 'bad' choices in the hope of achieving some lucky breaks. But... there is maybe a case for being a bit more of a risk-taker, if you hope ever to reach those higher reaches of the rankings.)

Sure, yes, this is just a thesis. I don't have the data resources to prove it. And so much of it is subjective or definition-dependent, I think it's essentially unprovable. But it is a compelling idea - carefully thought-out, and, I hope, persuasive....

The hard news for FPL enthusiasts (who almost all want desperately to believe that their 'success' is entirely deserved, derives solely from their shrewd judgement) is this: 

The game rankings are not a  pure meritocracy - they depend far more on luck than anything else. You can achieve good points and rankings by being lucky without being very skillful at all; but you can't achieve good points and rankings by being skillful if you're not also - at least a bit - lucky.