Fantasy Baseball: How Should We Evaluate Starting Pitchers?

There can sometimes be a huge gap between a pitcher's ERA and FIP, leading to different thoughts as to how they actually performed in any given year. How do we know which one is the more accurate number?

Evaluating starting pitchers is a tricky business. There is a metric ton of metrics out there to consider, and sometimes those numbers contradict each another.

If one is going to quantify the value of a pitcher, there are a few numbers analysts use most frequently. Batted-ball data, strikeouts and walks, velocity, and the movement of breaking pitches are all vital components in deciding just how good a pitcher has been.

But two sites, Baseball Reference and FanGraphs, use Wins Above Replacement, and many times, they're very helpful. Different formulas to calculate pitchers' WAR, though, with Baseball Reference using ERA to come up with their number, while FanGraphs uses fielding independent pitching (FIP). The latter is calculated to look like ERA, but strips out all batted-ball data. In other words, FIP only uses home runs allowed, walks, hit by pitches, and strikeouts to create a formula that mirrors ERA.

A pitcher's ERA and FIP are usually somewhat close, but not always. So what do we do when that happens? A third statistic can be used to break the tie.

Baseball Prospectus has a statistic called Deserved Run Average (DRA), which uses a collection of mixed models to game out the most likely contributions of pitchers to the runs they allow. Unlike FIP, it adjusts for home runs and balls in play. DRA is then used by Baseball Prospectus to come up with their own version of WAR, called WARP.

Pitchers Who Saw Good Fortune in 2017

In the first table below, we see the five pitchers in baseball with the biggest difference between their ERA and FIP, where the ERA is the lower number.

Gio Gonzalez Nationals 2.96 3.93 -0.97 3.35 6.4 3.3 5.0
Ervin Santana Twins 3.28 4.46 -1.18 3.74 4.8 2.9 4.3
Andrew Cashner Rangers 3.40 4.61 -1.21 4.81 4.6 1.9 1.4
Jose Urena Marlins 3.82 5.20 -1.38 5.29 2.0 0.2 0.5
Lance Lynn Cardinals 3.43 4.82 -1.39 4.54 2.8 1.4 2.1

If a pitcher has a lower ERA than their FIP, it is generally assumed they benefited from good fortune, as FIP is supposed to be a more accurate reflection of what any particular hurler was actually responsible for. In these cases, a pitcher's bWAR is going to be higher than their fWAR.

The Washington Nationals' Gio Gonzalez had the biggest difference between his bWAR and fWAR last year (6.4 to 3.3). It's rather significant because according to bWAR, Gonzalez's 6.4 was third-highest in baseball. Only Corey Kluber (8.0) and Max Scherzer (7.3) performed better, according to this metric. However, his fWAR was 20th-best mark among qualified starters in 2017.

That's a pretty big gap. Which is the more accurate number?

His DRA of 3.35 puts him a bit closer to his ERA than his FIP, assigning him a value of 5.0. So at least in his case, DRA and WARP agree with the end product of his results rather than the peripherals. The same is true for Ervin Santana, which can be seen above.

However, in the cases of free agents Andrew Cashner and Lance Lynn, as well as the Miami Marlins' Jose Urena, their DRAs and WARPs more closely resembled their FIP, indicating their ERAs may have been the result of some good fortune on balls in play.

Pitchers Who Experienced Some Hard Luck in 2017

Now, let's look at pitchers with the biggest differences in FIP and ERA, where FIP is the lower number.

Jason Hammel Royals 5.29 4.37 0.92 4.78 1.2 2.1 1.6
Jordan Zimmermann Tigers 6.08 5.18 0.90 6.32 0.3 1.1 -1.3
Jeff Samardzija Giants 4.42 3.61 0.81 3.63 2.2 3.8 4.5
Matt Moore Giants 5.52 4.75 0.77 7.06 -0.4 1.0 -2.9
Chris Archer Rays 4.07 3.40 0.67 3.30 1.2 4.6 5.1

In the cases of Jordan Zimmermann and Matt Moore, all of their numbers are poor, so it really doesn't matter a whole lot if they were a bit unlucky on balls in play. But for a pitcher like Jason Hammel, who had the largest gap between ERA and FIP on this end of the spectrum, there's a big difference between a 5.29 ERA and a 4.37 FIP.

For Hammel, DRA essentially split the difference, indicating hewas neither as bad as his ERA indicated, nor as good as his FIP. But for two other pitchers, their DRA more closely aligned with their FIPs, telling us they had a better season than their final ERA would suggest.

Jeff Samardzija's 3.61 FIP and 3.63 DRA are almost identical, while Chris Archer's DRA (3.30) is actually lower than his FIP (3.40), giving him a higher WARP (5.1) than fWAR (4.6). That's significant because given Archer's 4.6 fWAR was tied for the ninth-best season by any starting pitcher in baseball last year. His bWAR (1.2) was 49th out of 58 qualified starters. Again, a big difference.

It's always important to look at as many numbers as possible when evaluating pitchers. Righty-lefty splits against their opponents, recent trends in strikeout and walk percentages, and changes in velocity are sometimes more telling than any Wins Above Replacement figure.

However, these numbers off a nice snapshot as to who may be overvalued and undervalued in season-long fantasy baseball drafts in the coming months.