Creating TAVA

At this point, I have explained my useful, but problematic “Wins Added” (WA) score as well as my equally useful and equally problematic “Points Added”  (PA) score. One places too much emphasis on whether the end-result is a win or a loss, the other places too little emphasis on the same thing. It shouldn’t kill the suspense if you realized that I did the obvious: I combined them. This post explains how. And it’s in this post that many fans will have their eyes glaze over and go “huh?” and the actual math-nerds will go “That’s really all you did? Seriously?” This gives me the opportunity to leave everyone disappointed.

The problem with combining the scores is that I wanted to give each statistic exactly equal weight. The other issue is that both statistics are cumulative, meaning that the more games you play, the more wins or points you can “add.”

CONTROLLING FOR GAMES PLAYED

First, I divided both “Wins Added” and “Points Added” for each QB by the numbers of games they played where they received scores. At this point, each QB that I am measuring has a WA/Gm and a PA/Gm score. Thus, a QB who played 8 games won’t automatically be at a disadvantage.

WEIGHTING EACH FACTOR

Second, I calculated the average WA/gm, which should be around 0. As it turns out, it’s -0.008. Then, I found the standard deviation for WA/gm, which is approximately 0.152. I did the same thing for PA/Gm, which has an average of 0.391 and a St/Dev of 0.127.

My third step is to give each QB a “weighted” WA/gm and “weighted” PA/gm. This is done by simply calculating how many standard deviations a QBs WA/gm is above the average and doing the same calculation for the QB’s PA/gm. Now, I have two separate statistics that are each on a standardized scale. I combine them, and the sum is that QBs TAVA. Voila!

Calculating "POINTS ADDED"

The points added matrix shows how QBs performed across QB Support Levels, not just in terms of wins and losses, but also in terms of margin of victory.

                            WINS                                  LOSSES

			21	14	7	3		-3	-4	-8	-15	-22
QB SUPPT		22+	15	8	4	1		-1	-7	-14	-21
0.5 or Lower		1.000	1.000	1.000	1.000	1.000		0.897	0.846	0.667	0.513	0.000
0.5 to 1.499		1.000	1.000	1.000	0.969	0.922		0.813	0.656	0.438	0.234	0.000
1.5 to 2.499		1.000	1.000	0.948	0.885	0.688		0.479	0.313	0.156	0.052	0.000
2.5 to 3.499		0.990	0.941	0.833	0.696	0.529		0.441	0.245	0.127	0.029	0.000
3.5 to 3.999		1.000	0.944	0.778	0.611	0.417		0.250	0.139	0.056	0.000	0.000
4.0 to 4.999		0.939	0.864	0.576	0.348	0.227		0.076	0.030	0.000	0.000	0.000
5.0 to 5.999		0.717	0.417	0.300	0.133	0.067		0.017	0.000	0.000	0.000	0.000
6.0 or higher		0.533	0.311	0.222	0.089	0.044		0.044	0.000	0.000	0.000	0.000

	NO QBs SCORED HERE			Better than 30 - 50 %  %
	Better than 90% +			Better than 20 - 30 %
	Better than 75 - 89 %			Better than 10 - 20 %
	Better than 60 - 74 %			Better than   0 - 10 %
	Better than 50 - 60 %			Worst Performers

For each game, a QB receives a QB Support level. Additionally, each game is won or lost by a specific amount. By looking at those two variables, the above matrix can place each QB’s game in a cell. The number in that cell represents the percent of QB Performances (1) in that QB Support range and (2) that yielded a worse result (a result to the right of that cell). The number in that cell is then the QBs “Points Added” score for that individual performance.

WHAT ARE THESE PRETTY COLORS TELLING US?

These colors are telling us that QB Support has significant effect on a team’s outcome. As we increase in QB Support (going down the Table), we see that it’s increasingly more common for QBs to win, and win by larger margins. The diagonal shifting of those yellow and light green cells show the results average QB performances secure for a team given different support levels.

WHY THESE CUTOFFS?

            In “Wins Added,” I talked about the ranges of QB Support that I use. But why the cutoffs for the Point ranges. Well, I think the closest games are those decided by a field goal or less. I include in this category any games that go to overtime (tied at regulation). I think it makes a big difference if you need just a FG or a TD to tie or take the lead. After that, I think the biggest difference is whether it’s a one score, two score, or three score game. I did not take two point conversions into account, and I could say more on the topic, but I won’t right here.

            After being up three scores, I feel like much of the difference lies in a team’s approach rather the performance of the QB.  It doesn’t make sense in my mind to put much value on winning (or losing) by 38 points instead of 31 points. The game is out of reach for your opponent (or you) one way or the other and some coaches may put in back-ups while others may try to run up the score.

WHY IS “POINTS ADDED” HELPFUL

Points added is helpful because it gives a more complete picture of the game than “Wins Added.” If a QB Wins with a QB Support level of 4.5, that tells us that he performed better than 23 percent of QBs who lost with a QB Support level in the 4-5 range. If he won by 17 points or 3 points, we know that he either outperformed only those 23 percent who lost or the 86 percent who either lost or won by a smaller margin.

Matt Ryan’s 4 point win over the Cardinals earlier this year might have been his worst game. He squeaked out a win only because the rest of his team put him in a great position to do so. “Wins Added” gives Ryan the same 0.067 for that game that Tom Brady received for a very strong performance against the Rams in a 45-7 blowout. In contrast, “Points Added” gives Ryan a score of .133 for that game and Brady a score .717 for the Rams game.

WHY IS “POINTS ADDED” PROBLEMATIC?

While “Wins Added” places too much emphasis on who won or lost, “Points Added” doesn’t place enough. Consider these two scenarios where QBs received QB Support of 2.75. In scenario 1, the QB, down 30-20 throws a 25 yard TD pass on the final play of the game. His “Wins Added” score stays the same (-.47), but his points added goes up by 0.314. In Scenario 2, the QB is down 30-26, and throws a 25 yard TD pass on the final play of the game. This time, it is no meaningless play. This time, it’s the game-winner. And yet, the QB’s “Points Added” in scenario 2 go up by .288.

There are number of similar scenarios. Not all points are created equal. Pulling within 7 or even 3 points is simply not as valuable as scoring the go-ahead touchdown. “Points Added” corrects some of the problems of “Wins Added,” but it also has its own problems. 

CALCULATING WINS ADDED

Here’s how the entire league performed across the QB Support ranges I use to calculate Wins Added and, accordingly, TAVA.

QB Support WINS LOSSES WIN % 0.5 or Lower 0 39 0.0% 0.5 to 1.499 5 59 7.8% 1.5 to 2.499 30 66 31.3% 2.5 to 3.499 48 54 47.1% 3.5 to 3.999 21 16 56.8% 4.0 to 4.999 51 15 77.3% 5.0 to 5.999 57 4 93.4% 6.0 or higher 43 2 95.6%

“Wins added” is calculated for each game a QB plays. First, we look at how much QB Support the QB received. Then, given how others in that range performed, we can calculate the league’s win percentage in that range.

For each win, “Wins Added” is calculated as simply (1 – Win %). For each loss, it’s an even simpler calculation of (-Win %). These positive and negative values can then be added up for a whole season. 

WHY THESE RANGES?

The first step I had to take in creating “Wins Added” was to look at the win-loss records at each 0.5 interval. Let’s call these “sub-ranges.” What I found was that certain sub-ranges had very similar win-loss records (and margins of victory, but we’ll get to that later), and certain sub-ranges showed a significant difference from the sub-range directly below them. By grouping the similar sub-ranges together and separating them from any sub-ranges that were significantly different, I made ranges of QB Support that made the most sense based on the data.

WHY IS “WINS ADDED” HELPFUL?

“Wins Added” is helpful because, at the end of the day, we want to know who helps their team win. By looking at not just how often a QB wins, but how often that player wins when placed in different situations, “Wins Added” can see which QBs are actually playing an active role in helping their teams win (or lose) games, and which QBs win or lose mostly depending on what support they receive. 

WHY IS “WINS ADDED” PROBLEMATIC?

The problem with “Wins Added” is that it treats every win and loss as equal. When Andrew Luck received QB Support of 0.125 against the New England Patriots, “Wins Added” treats that game as essentially unwinnable and gives Luck a score of 0.0 because they lost, but so did every other QB placed in a similar situation. The fact that Luck’s poor play caused the Colts to not just lose, but lose by 25 gets completely ignored. When Drew Brees received QB Support of -0.275 against the Packers, he received the same score.  The fact that he played incredible and nearly defeated the Packers  (they lost by 1 point) despite receiving such little help from his team is also ignored. He receives the same score 0.0 wins added.

Of course, the same problem occurs with high support wins. Matt Ryan’s 5 INT game against the Cardinals (a 4 point win with QB Support of 5.9) receives the same 0.067 Wins Added as Tom Brady’s strong performance against the Rams (a 38 point victory with QB Support of 5.1).  It is precisely this problem that I attempted to correct (or least balance) with the creation of “Points Added.” 

Why Matt Ryan Should Be in the MVP Discussion

If you want my opinion, Adrian Peterson should be this year’s NFL MVP. In my mind, no other player had their consistently elite performance have such an impact on their team’s success.  If you replace him with an average player at his position, I could legitimately see the 2012 Vikings with a 6-10 record (or worse). Still, Peyton Manning is getting serious consideration for the 2012 MVP award. I’m here to tell you that, if we’re considering a QB for MVP, we need to also consider Matt Ryan.  Look at how the two QBs performed across QB Support levels.

P. Manning	WINS	LOSSES	TAVA
1.499 OR LOWER	0	1	2.23
1.5 to 2.499	1	1	3.17
2.5 to 3.999	2	1	3.28
4.0 to 4.999	4	0	3.72
5.0 or Higher	6	0	1.50
M. Ryan	WINS	LOSSES	TAVA
1.499 OR LOWER	0	2	1.93
1.5 to 2.499	4	0	7.80
2.5 to 3.999	4	1	4.29
4.0 to 4.999	2	0	1.63
5.0 or Higher	3	0	0.99

                                                 

RYAN: SAME RESULTS WITH LESS SUPPORT 

Both QBs were undefeated when QB Support was above 4.0. Similarly, both were winless when QB Support was below 1.5.  Where we see Ryan pull away is in the mid-support games. When QB Support was above 1.5 and below 4.0, Matt Ryan was 8-1, and Peyton Manning was 3-2. Not only that, Ryan’s only loss in this range was the meaningless week 17 game against the Buccaneers where the Falcons had already wrapped up the no. 1 seed.

Both QBs went 13-3. Both QBs had an 8-0 half of a season and a 5-3 half. Both QBs put up gaudy passing statistics (4500+ yards, 65+ %  completion rate, 30+ TDs, etc. etc.). Matt Ryan did it with less support. So why is Manning a serious candidate for the MVP award, and Ryan isn’t even in the conversation.

SHORT MEMORIES FAVOR MANNING

Part of this, I think, has to do with WHEN the 8-0 stretches happened. Ryan and the Falcons opened the season 8-0 before finishing 5-3 whereas Manning and the Broncos finished their year off on an 11 game winning streak. Here’s the catch: that 11 game winning streak had as much to do with the entire Broncos team. Manning had incredible levels of QB Support during this time. Meanwhile, the Falcons 8-0 run was much more dependent on the play of Matty Ice. To those who believe that wins later on in the regular season should “count more,” I would refer to you this year’s Baltimore Ravens (finished the season by going 1-4), the 2009 Saints (8-0 and then 5-3), the 2007 Giants (6-2 and then 4-4), the 2006 Colts (9-0 and then 3-4), and I could go on. 

Let’s take a look at each QBs season divided into halves.

PEYTON MANNING	WINS	LOSSES	EXPECTED WINS	TAVA
FIRST HALF	5	3	4.208	2.33
SECOND HALF	8	0	6.766	3.07
TOTAL	13	3	10.974	2.63
MATT RYAN	WINS	LOSSES	EXPECTED WINS	TAVA
FIRST HALF	8	0	4.311	5.55
SECOND HALF	5	3	3.836	2.32
TOTAL	13	3	8.147	3.90

Seven of Manning’s last eight games were played with QB Support of 4.0 or higher (and four with 5.0 or higher).  By season’s end, the Broncos supplied Manning with nearly 11 “Expected Wins” based on the support they gave their QB. His Expected Wins per game was just below Russell Wilson, making Manning the fourth most supported QB in the 2012 season.

Meanwhile, the QB Support Ryan received was equal to only about 8 “Expected Wins,” making him the 17^th most supported QB for the regular season.  Ryan only had about four “Expected Wins” during that 8-0 start. What does all this mean, though? Here’s a quick comparison of Matt Ryan’s 8-0 start to Peyton Manning’s 8-0 finish with some more “basic” statistics.

	True Points Against	Rushing yards/gm
MATT RYAN FIRST 8 gms (8-0)	16.1	86.5
PEYTON MANNING LAST 8 gms (8-0)	9.4	124.5

Putting aside factors like field position and the kicking game, these two simple statistics, more than anything else, clearly demonstrate the situation.  Ryan had to overcome more points while receiving less help from his teammates in the ground game. Manning provided tremendous value to the Broncos during their winning streak, but they certainly didn’t need an All-Pro QB to win some of those games.

THREE THINGS QB SUPPORT MIGHT BE MISSING

There are a number of things that QB Support cannot statistically take into account. Here are three things it misses and why it might lead us in the wrong direction.

1.       QUALITY OF WRs

It’s very difficult to quantify how much WRs are helping their QB because the play of each position is so intertwined with the other.  Did Demaryius Thomas become a top 3 WR (as ranked by DYAR) this year compared to 62nd last year because he really improved his game? Or was it because Peyton Manning was throwing him the ball instead of Tim Tebow?

I haven’t figured out a good way to take WR quality into account (and I don’t see myself doing so in the near future). Regardless, as good as Decker and Thomas might be, I think most would agree that Roddy White and Julio Jones are probably the most talented WR duo in the league. Because of them (and Tony Gonzalez), Matt Ryan’s TAVA scores will always be a little bit inflated. How inflated is the question that we cannot answer.

2.       AUDIBLES AND CALLING PLAYS ON THE FIELD

It’s not a secret that Peyton is often as valuable for his on-the-field coaching abilities as he is for his personal playing ability. To what extent does Denver’s rushing attack owe its success to Manning’s ability to audible and adjust at the line of scrimmage? I don’t know, and TAVA gives him no credit for this.

3.       MANNING FACES

I suspect that much (if not all) of the QB Support that Manning receives is due to the fact that his teammates get to look at expressions like this. I mean, not taking things like this into account is probably my statistics greatest flaw. 

ROY Debate Part III: Andrew Luck and doing more with less

In this final installment, I’ll discuss how Andrew Luck fared this year across different support levels, and give my final thoughts on the ROY of the year debate. Luck had the highest TAVA of any rookie QB, ranking 5^th overall in the league (2^nd in “Wins Added and 13^th in “Points Added”).

As I did with RGIII and Wilson, I’ve included, as a reference point, how all QBs performed across a few different ranges of QB Support.

QB Support	Wins	Losses	Win %
1.499 or Lower	5	98	5%
1.5 to 2.499	30	66	31%
2.5 to 3.999	69	69	50%
4.0 to 4.999	51	15	77%
5.0+	99	6	94%

And here’s how Andrew Luck performed in those ranges along with his TAVA scores for each range.

A. Luck	WINS	LOSSES	TAVA
1.499 or Lower	0	4	-0.67
1.5 to 2.499	3	1	2.49
2.5 to 3.999	4	0	1.28
4.0 to 4.999	3	0	0.18
5.0+	1	0	0.19

Of the three rookie QBs we’ve been discussing, Luck was the only QB to play in games with a QB Support level of less than 1.5. One of those games was just barely below at 1.31, but three of them were below 1.

THE BEST

Luck added the most value to his team in the low to middle support ranges. Luck played in 8 games where QB Support was somewhere between 1.5 and 3.999. The Colts went 7-1 in those contests, almost always because of a strong outing from Luck and often including late game heroics. Many of these were close games, but given the comparatively low QB Support levels, Luck would have had above average (though admittedly lower) TAVA scores even if he had lost a couple more of these games in close fashion. Furthermore, he had a positive TAVA score in the one loss in this range. It was also a close game, a five point loss where Luck drove the offense to the opponent’s 26 before coming up short on two final pass plays.

Consider this: games where QB Support is below 1.0 are almost surely losses, and games where QB Support is above 5 are almost surely a win. So the games where QBs can make the biggest difference are games where QB Support is between 1.0 and 4.0. When QB Support was 1.0 to 4.0, Russell Wilson was 5-5, RGIII was 8-5, and Andrew Luck was 10-2.

THE WORST

Andrew Luck may have been ranked 2^nd in “Wins Added,” but his TAVA was dragged down by a lower score (ranked 14^th) in “Points Added.” Despite his 3-0 record in games with QB Support above 4, Luck won those games by unimpressive margins of 7, 7, and 4 points. He did enough to help his team win, but not much more.

Luck’s real struggles, however, came when his QB Support was at its lowest. The -0.67 TAVA for extremely low support games represents that. Yes, those four games were almost surely losses anyway, but Luck’s poor play turned losses into blowouts. He lost his three lowest support games by 20, 26, and 35 points. In those three games where his QB Support was under 1.00, his TAVA was an even lower -0.80. Even then, that represents only three games, and perhaps it shouldn’t be surprising that a rookie QB with exceedingly low support was feasted on by the opposing defense.

BUT WHAT ABOUT ALL OF LUCK’S TURNOVERS?!       

There are those that dismiss Luck’s season as, well, lucky. Sure, he won some close games, but he really didn’t play that well. He turned the ball over 23 times. HE’S TIED WITH MARK SANCHEZ FOR INTERCEPTIONS. MARK SANCHEZ! Meanwhile, RGIII turned the ball over only 7 times, and Wilson threw only 10 interceptions and lost 3 fumbles. How does a guy with almost double (or more than triple) the amount of turnovers of his fellow rookies end up with a higher TAVA? What gives?

I mentioned that Luck’s TAVA was lowest in the three games where his QB Support was below 1.0. Let’s take a look at those three games:

AVG QB Support	Avg. True Points Against*	Rush Yards/gm	Pass Atts./ Gm	INTs	Fumbles
0.52	38	70.33	46.33	8	3

*Accounts for TDs scored on a QB’s turnover as well as points created by a QB’s own special teams and defense.

In those three games, Luck was tasked with overcoming, on average, 38 points (and never less than 34) while getting only an average of 70 rushing yards from his teammates. That, and the fact that he started ZERO drives inside his opponent’s 50 yard line, is probably why he threw more than 46 passes per game, often in situations when the opposing team knew he would be throwing. It shouldn’t be a surprise that 11 of Luck’s 23 turnovers came in those 3 games. Furthermore, 10 of those 11 turnovers occurred when the Colts were already losing by an average of over 16 points. One interception occurred when the game was tied.

Keep in mind, Neither Wilson nor Griffin III were ever asked to throw more than 40 passes in a single game nor did they ever face QB Support levels below 1.0. RGIII had to overcome 30 points three times (though never more than 32), but his teammates rushed for over 110 yards in each of those games AND his defense and special teams provided him with strong field position in those games as well. Wilson meanwhile never had to overcome 30 points, and only had to overcome more than 20 points in 2 games.

Another way to highlight the same idea is simply to look how often the three QBs had to throw less than 25 times, 25-40 times, and 40+ times, and what effect, if any, that has on how often they throw interceptions.

<25 Pass Attempts	GAMES	ATTEMPTS	INTs	INTs/Att
R. Wilson	9	183	3	1.6%
R. Griffin III	5	100	2	2.0%
Andrew Luck	0	0	0	0.0%
25 - 40 PASS ATTEMPTS	GAMES	ATTEMPTS	INTs	INTs/Att
R. Wilson	7	210	7	3.3%
R. Griffin III	9	278	3	1.1%
A. Luck	9	285	5	1.8%
40+ PASS ATTEMPTS	GAMES	ATTEMPTS	INTs	INTs/Att
R. Wilson	0	0	0	0
R. Griffin III	0	0	0	0
A. Luck	7	342	13	3.8%

Luck wasn’t just the only one of the three QBs to throw more than 40 times; he did so in almost half of the games he played. Meanwhile, in more than half the games Wilson played, he threw the ball less than 25 times. RGIII’s ability to avoid turnovers is, however, still apparent, even after controlling for attempts/game.

CONCLUSIONS ON LUCK

Like we did with RGIII and Wilson, let’s take a look at how Luck’s TAVA breaks down for each half of the season.

A. Luck		WINS	LOSSES	Expected Wins	TAVA
FIRST HALF		5	3	2.51	1.02
SECOND HALF		6	2	4.21	0.62
TOTAL		11	5	6.717	0.82

Luck was significantly above average for each half of the season. We also see that Luck had less support than either Wilson or RGIII. His Expected Wins are just a hair above RGIII’s, but keep in mind, those are the expected wins for Luck’s 16 games, compared to RGIII’s 14. It’s not surprising that the first overall pick (going to a team who was 2-14 the year before) did not get a whole lot of QB Support or that his expected wins were between 6 and 7. What is surprising is how much value Luck managed to add to the Colts in his rookie season. Even in his first season, Luck showed that he can win the tough games, that he doesn’t need a strong defense or ground game to lead his team to victory.

FINAL THOUGHTS ON THE OROY DEBATE

Luck had the highest TAVA, and his worst games occurred when he was placed in situations that RGIII and Wilson simply did not face. Nevertheless, when things were at their toughest, Luck had significant struggles. It’s also not fair to assume that, if they had faced extremely low QB Support, RGIII and Wilson would have struggled as much as Luck did.

Compared to RGIII, Luck had wider swings between his best and worst performances. Additionally, while RGIII and Wilson showed significant improvement in the second half of the season when he got more QB Support, Luck seemed to play a little bit worse. 

Looking at the QB Support numbers, I would give the award to Luck, though just by a hair and with plenty of ambivalence. Yes, RGIII was more consistent. But no rookie meant more to his team than Luck. No other rookie played such a central role to his team. The lack of QB Support Luck received is a big reason that he threw the ball over 200 times more than RGIII or Russell Wilson, and it’s a big reason why he made more frequent mistakes.

There’s definitely a strong argument for RGIII, who never had the lows that Luck had. If there was ever a time to have Co-OROYs, this would be the year. In terms of Wilson, however, I don’t think he should really be in consideration. He’s shown tremendous promise, especially with how much he continues to improve. Nevertheless, the OROY is an award for a single season and an entire season. As good as Wilson was in the second half of 2012, he simply did not perform at the same level as RGIII and Luck for the entire year. The ability of the top two picks to start strong right out of the gate was so critical to the value they added to their teams. If it’s going to go a QB, it should be Luck or RGIII.