Last Player Picked

You may have noticed a recurring theme here is dealing with valuing pitchers relative to hitters. Average draft position starts drafting pitchers in round 2 and doesn’t really get going on them until the fourth round. The Price Guide (as well as several other sources) tends to rank the top-tier pitching higher and advocates grabbing Santana in the first round.

(Since it’s a topic I seem to be stuck on, I have appreciated the comments that have been left on the various pitcher/hitter ADP posts. Lots of stuff has been written that has really made me think about this.)

Anyways, being ever vigilant for evidence that supports one approach or the other, I took a look at how Rotoworld valued pitchers. Here are a few SP dollar values from their magazine, with overall rankings included:

5. Johan Santana – $37 (tied with Pujols)
14. Tim Lincecum – $31
19. Roy Halladay – $30
22. Cole Hamels – $29
26. Brandon Webb – $28
29. CC Sabathia – $28

Not quite as aggressive as the Price Guide, but considerably higher than ADP. Actually, those are probably a good indication of what you would end up paying if you were following the Price Guide and the rest of your league was using ADP.

I have mentioned that I think Standings Gain Points (SGPs) are a pretty good method of determining player values for fantasy. However, I don’t use them for the Price Guide based on some shortcomings I have found. This week I’m pointing out some of the quibbles that I have with the SGP methodology:

If you have looked at any of the projection systems that the Price Guide uses, you may have noticed that they look a little conservative. Marcel projects Ryan Howard to lead the league with 119 RBI? No pitcher is expected to finish with more than 16 wins?

These things are not indicative of problems with the projection systems and are, in fact, positive features. While it’s very likely that someone will get close to 20 wins, it is impossible to predict who it will be. I’d bet that someone tops 130 RBI, but I don’t have a clue who it will be. When you are predicting something as unpredictable as baseball stats, the smart route is to be conservative. (Tango explains this fully here.)

So it’s clear that our projections are operating on a different scale than the real life stats. On the projections scale, a player projected for 100 RBIs is elite. In real life, he’s just “pretty good.”

What happens when you mix-and-match the projected scale with the real life scale? Standings Gain Points attempts to find out. On one hand, it’s looking at the stats from last year (the “real life scale”) to determine the relative value of each category. It then takes those values and applies them to a completely different scale — the regressed scale from the projections.

Consider saves, a highly variable stat: Players on fantasy teams will always finish with more saves than what was projected for players before the year. When SGPs look at last year’s stats, they might see about six saves separating each team (McGee came up with 6.2 – 6.3 in his book) and determine that 6 saves are worth 1 SGP.

Remember, however, that the projections don’t project that many saves! And on a scale with fewer saves, the saves that are available become more valuable.

This is a nuance that is missed by SGP, but is caught by the Price Guide. Notice that when the Price Guide determines the value of each save, it is using the scale determined by the projections themselves. Which means it is willing to value saves from the reliable closers more highly than SGP does.

I have mentioned that I think Standings Gain Points (SGPs) are a pretty good method of determining player values for fantasy. However, I don’t use them for the Price Guide based on some shortcomings I have found. This week I’m pointing out some of the quibbles that I have with the SGP methodology:

How many types of fantasy baseball leagues are there out there? Fantasy leagues might have 6 teams or 30 teams or anything in between. They use different stats: 4×4 and 5×5 and 6×6 and hundreds of other combinations. Different leagues start different positions: two catchers, middle infield, middle relievers, etc.

When I was developing the Price Guide to build fantasy values, I obviously wanted something that could be used by people outside of the Standard League. And requiring that flexibility ruled out using Standings Gain Points.

See, SGPs are tied to the final standings of a specific league. They are built based on a certain number of owners, a certain number of hitters, and a certain number of pitchers. The more you deviate from those specific settings, the less accurate SGP prices will be.

Consider Razzball‘s Point Shares, which are essentially a SGP system. A while back I drafted a league that had 6 teams using Point Shares and 6 teams using the Price Guide, tailored specifically to the league settings that Point Shares were based on. When I checked the final results, the Price Guide teams and the Point Shares teams split the points right down the middle. Out of 550 total points, each system got 275 — exactly half. In a league customized for a SGP system, SGPs and the Price Guide can do equally well.

However, if you change any single aspect of that league, the Price Guide will very likely take the top six spots. If you add a couple of new positions or add a couple of teams, the Price Guide values will adjust for it, and SGPs won’t. (This fact became very clear when I setup the Price Guide/Point Shares league, which wasn’t even competitive when I accidentally left out CI and MI.)

I imagine most people can live with SGPs inflexibility: After all, most people aren’t trying to build an online tool that works for any league, they just want a system that works for their league.

But what happens if your league decides to expand from 10 to 12 teams this year? What if they decide to use OBP instead of AVG?

If anything in your league changes from the year before, the accuracy of your SGPs is greatly compromised; but the Price Guide will adjust for whatever settings your league chooses.

I have mentioned that I think Standings Gain Points (SGPs) are a pretty good method of determining player values for fantasy. However, I don’t use them for the Price Guide based on some shortcomings I have found. This week I’m pointing out some of the quibbles that I have with the SGP methodology:

There’s a standard disclaimer in financial investments that “past results do not guarantee future returns.” Just because a stock has done well in the past, it doesn’t mean that it will continue to do well. It’s a principle that applies to baseball as well: A good hitter who is 0/10 against a pitcher might go four-for-four the next time they meet. And I think a big problem with SGP is expecting the next year to be just like the past one.

Since SGPs are based on the standings from previous years, they will be affected by past strategies. If two owners decided to punt saves last year, SGP will factor their save totals into this year’s denominators. If a few teams ran away with stolen bases, that will be included in the prices as well.

But remember, past results do not guarantee future returns. What happens if no one punts saves this year? What happens if stolen bases end up being a tight race, with no one pulling away with a huge lead?

Fantasy owners are dynamic: They are trying out new strategies from year to year. They are zigging if they saw people zag last year. You can’t rely on the same strategies being used from year to year, but that is exactly the assumption that SGP makes.

In his book, Art McGee does make some efforts to account for this problem. He suggests throwing out the top and bottom teams when building your SGP denominators, because these are more likely to be outliers. He recommends using multiple years of data if you have it.

Definitely, the more data you have to work with, the better your SGPs will be. But when I developed the Price Guide, I wanted a method that didn’t require years of historical, league-specific data to be effective.

And speaking of league-specifics, that’s actually my next quibble with the SGP method.

Since the start of fantasy baseball, the constant challenge that participants have faced is trying to determine players’ values. What do you pay for a guy who can get you 30 SB? Is he worth more or less than someone who hits 30 HR? Would you trade him for someone with a 3.65 ERA?

Really, it’s a problem that boils down to this: How do you measuring a player’s contribution, when the contributions are based on 8 or 10 different scales (i.e. the 4×4 or 5×5 stat categories)? The solution is to convert all of the categories to use a common scale.

One way to do this is to use Standings Gain Points. Standings Gain Points are a system developed by Alex Patton and popularized by the recent re-release of Art McGee’s How To Value Players For Rotisserie Baseball. Derek Carty praises SGP as “the greatest, most logical way of valuing players I’ve ever heard of.”

How do SGPs work?
The basic idea behind SGP is to look at the final standings from your league in previous years and determine how much of each stat it takes to pick up an extra point in the standings. How many homeruns would it take to move up one spot in the homerun standings? How many strikeouts separate each team?

If you notice that there are 10 HR separating each team, then you know that, on average, it will take an additional 10 HR to move up one point in the standings. So 10 HR are equal to 1 SGP. Do the same thing for all of the other stats, and all of a sudden you have reduced a player’s contributions from across four or five categories down to one, simple number. Sort players by that magic number and you will have rankings customized for your draft. Convert that number into a dollar value and you are ready for an auction.

To be honest, I think Standings Gain Points is overall a pretty sound methodology. However, I do think it has some flaws that keep it from being “the greatest, most logical way of valuing players” ever.

This week I’ll be taking a look at some of the problems I see with SGP, and also explain why I decided to take a different approach when building my own valuation system, the Price Guide.

Another advocate of drafting pitchers in the first round: Baseball Prospectus’s Player Forecast Manager. Here’s an excerpt from an article I came across:

Q: It looks like the overvaluation of pitchers is back again. After the first day, the valuations were normal, but the PFM is currently showing pitchers as the second, third, fourth, and 6th-8th most valuable players in the game for fantasy purposes.

BP’s Ben Murphy explains that, although it might look like a bug, the high pitcher values are correct based on their methodology.

So why does everyone who runs the numbers come to the conclusion that the top-tier starters are worth taking in the first round? Is everyone overlooking the unpredictability factor?

In last week’s discussions on why the Price Guide’s values differ from what you see in average draft position (ADP), Nick made the following comment:

Regarding risk and projections, I know that Marcel contains reliability scores and PECOTA has a beta score, but I’m not sure exactly how they are incorporated into the projection numbers themselves.

He references a comment from Tango on Razzball:

I would say that you MUST use the “reliability” column provided with Marcels. And, the playing time forecast is the most important thing in fantasy sports, so you have to get that right first.

If you are familiar with the Marcel projections, you may have noticed that each player’s stats have a reliability rating. This doesn’t indicate injury chances or anything like that, but measures the confidence level of the projection. Since the Marcel’s are based on a player’s performance from the past three years, a player who has been in the majors since 2006 will be projected with a higher reliability than, say, Evan Longoria.

So what happens if we adjust our Price Guide values using Marcel’s reliability factor? Will that fix the discrepancy we see in pitcher values?

Here’s an illustration of what changes with the Price Guide’s fantasy dollar values for pitchers when reliability is considered:

Notice what happens when the reliability ratings were added: The values for the top nine pitchers were practically unchanged, because these are the pitchers that have shown themselves to be consistent performers over the past few years (Santana, Sabathia, Webb, Haren, etc.). After those nine, the prices for the rest of the pitchers dropped by about $3-4 each, all the way down to the bottom. Those are the players that have not put up consistent numbers over a large number of innings.

It’s not a drastic change, but it does fit a little bit better with what we see happening in average drafts.

You may remember from a previous post that three of the ten players that the Price Guide diverged from ADP most widely on were RP. Here’s how the draft rankings on those three look with reliability factored in:

Wow! Considering reliability, we practically eliminated a gap of about 20 spots. Papelbon and K-Rod are both eerily close to what ADP suggests now. The change in reliever rankings is easily the biggest effect we see after adding reliability.

I’m hoping to have an option added within the next day or two to use reliability factors when building fantasy values on the Price Guide (at least for Marcel, which are the only projections that include a reliability ranking). Reliability doesn’t make a huge difference, but I think it does help a little. The top talent is pushed up while the lower tiers are flattened out a bit. Due to low IP totals, relievers are discounted especially.

We still haven’t explained all of the SP in the first round. Even after we account for reliability, the Price Guide still thinks top-tier pitchers are worth paying for. But I think this gets us a step closer to the most accurate values.