Wednesday, August 28, 2013

New and Improved TAVA Methodology

TAVA or Total Adjusted Value Added is a metric designed to show which QBs are truly playing a valuable role in helping their team win ball games, which QBs might be winning but in more of a game manager role, and which QBs are holding their team back. 

TAVA = "(weighted) WINS ADDED" + "(weighted) POINTS ADDED"

WHAT IS "WINS ADDED"
WINS ADDED or WA is a calculation that measures how much a QB is adding to help his team win. By looking at an entire season's worth of data, one can see a clear correlation between QB Support and a team's chance of victory. And this makes sense: a team that rushes for 250 yards, allows three points, and scores two defensive TDs should have a pretty good chance to win, regardless of QB performance.

To get to WA, I perform a logistic regression to estimate, at any given amount of QB support, what is the expected chance of victory. Then, I take that percentage chance of winning (Pw) and put it into one of the following equations.

If the QB Wins, I use this equation:

   (1-Pw)    
  ((Pw)+.5)

If the QB loses, I use an inverse equation that looks like this:

      (-Pw)      
  ((1-Pw) +.5)

What this ends up doing is giving each QB a WA score for each game where they were the only primary QB. Here are a few examples of what the score would be if a QB won or lost a game with a certain QB Support

QB SUPP
% Chance of Win in 2012
Result
WA
1.00
11%
WIN
1.46
2.00
25%
WIN
1.00
3.16
50%
WIN
0.50
4.00
69%
WIN
0.26
5.00
86%
WIN
0.10


QB SUPP
% Chance of Win in 2012
Result
WA
1.00
11%
LOSS
-0.08
2.00
25%
LOSS
-0.20
3.16
50%
LOSS
-0.50
4.00
69%
LOSS
-0.85
5.00
86%
LOSS
-1.34

WHAT'S WRONG WITH WINS ADDED
The problem with WA is that it unfairly punishes QBs who may have played well, but just barely lost a game as well as QBs who played excellent and helped make an otherwise close game a complete blowout win. It overly rewards QBs who win close games (like Andrew Luck and Matt Schaub in 2012), even if, sometimes, it was only close because they played average or below average. This is why TAVA also has POINTS ADDED.

WHAT IS "POINTS ADDED"
POINTS ADDED (PA) is a much simpler calculation. I take the data from a single season, and I plot out the QB Support numbers against the margin of victory or defeat. This gives me data points at like 0.5, -7 (a QB who lost by seven points with a QB Support level of 0.5) or 2.5, 10, etc. etc. I use a linear regression to create an equation that tells me, given the QB Support level, what is the expected points a QB's team should expect to win or lose by.

For example, a QB might win with a QB Support level of 5.5. That's fine, but the equation for PA says that his team should have expected to win by about 13 and a half points, and so if they only squeaked out a win with a last second field goal, well, then his PA score would -10.5 points in that game. Likewise, a QB who lit it up and led his team to a 24 point victory would have a PA score of 11.

WHAT'S WRONG WITH POINTS ADDED
Well, it's the opposite problem as WA. Whereas WA puts too much emphasis on just whether or not the game was won, PA puts too little. A QB who throws a garbage time TD to lose by 10 instead of 17 (or win by 17 instead of 10) improves his PA score just as much as the QB who throws a game winning TD to go up by 4 points with 3 seconds left.

TAVA: A GOLDILOCKS METRIC
TAVA takes the positives and negatives from WA and PA and balances them by combining them. Again, it takes the 512 QB performances over the season, gets an average and a standard deviation for both WA and PA. Then, each QB can receive both a PA and WA weighted score based on how many standard deviations their scores are above or below the average. Combine those weighted scores and VOILA! You have their TAVA scores.