- The Daniel Ratings rate past performance; they do
not predict future results.
This distinction is definitely the thing most often overlooked
by people commenting on this system (and probably every other system
as well). Being aware of the distinction between these two purposes is
crucial to understanding any ratings system.
Here's a synopsis of the two main types of system:
- A power index, also called a predictive system. It's purpose is
to predict the future. A high ranking in a power index says that a
team is strong, and likely to beat a team ranked lower. The highest
ranked team in a power index is the "strongest".
- A performance index, also called a retrodictive system. It's
purpose is to rate past performance. A high ranking in a performance
index is akin to saying that team is in "high in the standings"; the
highest ranked team in a performace index would be in "first
place".
The Daniel Ratings are performance index.
Therefore, you will often see one team ranked above another team
it's obviously going to lose to next week, especially early in the
season. The reason is simply that the ratings are not a measure of
strength, but of performance. They say what the team's accomplished,
not what they're capable of.
And, I'm sorry, but the strongest team in the world hasn't
accomplished much if they've only beaten a bunch of chumps. A weaker
team that's defeated some nonchumps is going to be ranked
higher.
- Margin of victory is not a factor; only wins and losses
matter.
I get an awful lot of comments saying, "How can you rank X so
low? They killed everyone on their schedule!" The reason, simply, is
the Daniel Ratings only judge a team based on wins and losses. Scores
do not factor into the system at all.
But why only wins and losses?
Well, as I mentioned above, the Daniel Ratings are a performance
index. But what is performance? There are many, many aspects of the
game that you could take as an indicator of peformance: wins, road
wins, scoring margins, offesive yards, etc. A good performance is
really just whatever you want to reward. Well, it turns out that the
perforamce I wanted to reward was wins. I wanted to say who did the
best job at winning, not who did the best job at winning by large
margins.
But it's not quite as simple as me just waking up one day and
deciding to only use wins and losses; there's a reason for it. When
it comes time to choose or seed teams for the postseason, I feel that
it is wrong to use anything other than wins or losses, except as a
tie-breaker. Winning is the name of the game in football. I feel
that a system that uses any other factor goes against the tradition
and culture of football, and field sports in general. It's almost
universal in such sports that winning is the only factor that matters
when determining who won the "round" (be it a regular season, a
playoff series, a match, or whatever).
To illustrate:
- To make it to the NFL playoffs, the only aspect of performance
that matters is wins (except as a tiebreaker, and deep one at
that).
- To win the World Series, the only aspect of performance that
matters is the wins. You can outscore your opponent in the World
Series 55-27, and still lose. (In fact that happened once: in 1960.)
- To win a volleball match, the only thing that matters is to win
games. You can outscore your opponent 132-75 in a match yet still
lose the match three games to two. Likewise, in a tennis match, the
only thing that matters is to win sets. And in a set the only thing
that matters is to win a game.
The point is, in a lot of the sports we deal with every day, we only
care about wins. Winning is the name of the game.
I realize that some sports, like auto racing, do use points. It
could be possible to win the This-Year's-Corporate-Sponsor Cup in some
some racing league without winnnig a single race. So it's not a moral
travesty to use margin of victory to choose postseason teams, but for
team field sports like football, the tradition is that winning is the
only thing that matters.
- An early game counts just as much as a late one.
For much the same reason that I didn't want to use margin of
victory, I also didn't want to consider the date when the game was
played.
It's pretty much the same argument: traditionally, sports don't
weight recent games higher when deciding who wins the round and
advances. An early season win in the NFL counts just as much as a
late season win; Game 1 of the World Series counts just as much as
Game 7, etc. If we want a system that chooses or seeds teams, it
should weight all games equally, regardless of date.
This can sometimes confuse people who think rating systems ought to
be a computer version of the human polls. Human polls nearly always
reward recent wins and penalize recent losses disproportionately more
than earlier ones. The Daniel Ratings don't do this, so sometimes
you'll see a team with a large change in the polls but hardly any
change in the ratings.
- The Daniel Ratings uses no a priori
information about teams.
The Daniel Ratings are completely based on wins and losses from
the current season. They use no prior knowledge of how good teams
are.
This confuses people sometimes, especially early in the season,
where there isn't a lot of data. Sometimes, a generally-accepted
strong team will play a weak schedule early. The Daniel Ratings has no
prior knowledge of the team, so all it knows is that they beat a bunch
of patsies, which certainly this doesn't qualify that team for a high
rating.
As the season goes on, and more information comes in, and the
supposedly strong teams play other supposedly strong teams, these
sorts of oddities happily fade away somewhat.
- The Daniel Ratings rates each team's peformance
over their entire schedule.
Oftentimes, I hear comments like, "X beat Y, so why do you have
Y ranked above X? Your rankings suck!" There's a good reason for it:
the Daniel Ratings never look narrowly at one or two games to
determine a team's rating; it always rates a team's peformance against
it's entire schedule. (More to the point, no game result can trump
any other result. No single game result can constrain the order of
the rankings.)
The fact is, teams have vastly different strengths of schedule.
And sometimes, the disparity is strong enough that even if X beat Y,
and has a better record than Y, Y could still be ranked above X. The
Daniel Ratings will look at a team's whole schedule, and will reward Y
for beating hard teams, and if it's enough Y will be ranked higher.
The fact that X beat Y cannot trump those other games on Y's schedule;
it cannot force X to be ranked above Y.
And if you think this is retarded, think again: it's manifestly
impossible to do this consistently for all teams without producing
absurdities. I'll repeat myself. It's impossible to do what
you ask consistently for all teams.
This whole issue is one of my pet peeves, actually. It never
ceases to amaze me how much people believe that The Better Team Always
Wins. Sure, almost everyone would say, "I know the better team
does not always win." Yet they often turn around and
make arguments (like the one above) that are predicated on it, at
least in part.
- They're called the Daniel Ratings, and not the Daniel
Rankings, for a reason.
Ranking is just a side effect of
rating. Let me explain.
In the Daniel Ratings, every team gets a rating. The rating of the
team is it's measure of performance. Winning a game can never result
in a team's rating going down. Losing a game
can never result in a team's rating going up.
That's as it should be: you're measuring performance, and your overall
performance can't get worse if you win a game (and remember, winning
is only aspect of performance in this system).
Ranking, on the other hand, is not a
measure of performance. Even if you win a game, your ranking can still fall. Why? Because some other team
previously ranked below you might have won a tougher game, thus
gaining more rating points than you, thus
overtaking you.
So if your team wins, but still goes down in the rankings, you shouldn't be crying "I won, but my
ranking went down." I assure you, according to the Daniel Ratings,
your peformance has improved. Your rating has
gone up. It's just that some other team's rating has gone up more.
- These ratings are for fun.
Really. I do these rankings because the amuse me. I find
these ratings amusing because it takes a seemingly silly concept
(transitivity) and produces something that's surprisingly
reasonable.
I introduced these ratings into a culture where it was common to
taunt good teams who lost an early game with long chains of
transitivity. These ratings are actually a sort of parody of that
thinking, and people who took part in those sorts of arguments should
find these ratings interesting.
However, these ratings have flaws, so they
shouldn't be taken too seriously.
Having said that, if someone comes in and attacks the Daniel
Ratings from completely baseless grounds, I'm going to defend them. I
don't care much if you disagree with details, but I won't stand for
unfair criticism based on misguided thinking. "Your rankings suck
because they have Y ranked above X," is anecdotal evidence, not a
well-reasoned argument, and oftentimes isn't even valid because of the
complainer's tunnel vision.
A reasonable argument like, "I can see how Y's schedule is
stronger, but it looks like they might be getting a little too much
credit for beating this team here," is likely to get a "probably" from
me.