Sloan Sports Analytic Conference
MIT's Business School Presented the Sixth Annual Sloan Sports Analytics Conference in Boston on the weekend, and Scott Cullen was there, witnessing the uphill fight for advanced statistics to gain widespread acceptance in hockey.
Before we get into this, a disclaimer. It may not be necessary, but if you're not familiar with my work, I'm very much into the statistical analysis in sports. It's what I do and as information becomes ever-easier to acquire, it seems like an area that is ripe for growth as I wrote in the foreward for this year's Hockey Prospectus, but not everyone is a believer.
I am a believer, that statistical analysis can provide valuable information, so my recap of the weekend's proceedings comes from that point of view. It's the same point of view I had last year when I was critical of hockey's use of advanced statistics and that frustration may spill out for years to come until there is more widespread acceptance. It's nothing personal, hockey, I just want the best for you.
With that out of the way, my recap of the 2012 Sloan Sports Analytics Conference and the influence of statistical analysis in sports, especially hockey.
LONG TIME COMING
Bill James, who was honoured with a Lifetime Achievement Award at this year's conference, started writing his Baseball Abstracts in 1977 and he had little to no acceptance until Daniel Okrent wrote a column in Sports Illustrated in 1981. (Long-time fantasy players may recognize Okrent's name -- he organized the first Rotisserie baseball league in 1979.)
Even then, James' acceptance didn't come from within the baseball community. It was a much longer process. While there isn't a definitive date at which MLB teams started using James' Sabermetric philosophies, but the late 1990s Oakland A's -- slightly before the Moneyball era -- are generally considered the trail blazers in this regard.
The thinking from hockey's advanced stat community is that, seeing James' influence on baseball, there should be teams in other sports clamouring for this kind of objective analysis. There are some NHL teams that are using advanced stats analysis, but the league isn't at the point at which statistical analysis has reached widespread acceptance.
When James Mirtle of the Globe and Mail wrote about Moneypuck last September, he spoke with Dan MacKinnon, who is in charge of the Pittsburgh Penguins' analytical efforts. MacKinnon, who was a panelist at the Sloan Conference last year, estimated that maybe half a dozen teams are actively using analytics with maybe another half dozen starting to explore the ideas. Based on conversations with others at the conference this weekend, that estimate still seems to be a fair representation.
Advanced hockey stats face several challenges in gaining wider acceptance. For starters, advanced hockey stats haven't been around that long, when compared to James' advances in baseball stats more than 30 years ago.
Advanced hockey stats are also more complicated due to the nature of a game that involves ten skaters in motion on the ice at once with changes on the fly, making it necessary for any complete statistical analysis to consider strength of teammates, strength of competition and zone starts in order to provide context for the raw numbers.
HOCKEY ANALYTICS PANEL
Additionally, modern stats analysts face challenges of convincing teams that are often run by men in their fifties and sixties that have been involved in the game for their entire lives and have the strong convictions and track record required to be a success in the industry.
As such, it wasn't remotely surprising that Toronto Maple Leafs President and General Manager, Brian Burke, would be so vocal in his opposition to the premise of using advanced stats in hockey. Burke thrives in the panel environment and can dominate the proceedings because of his bold and colourful opinions.
He was joined on the panel, hosted by the NHL Network's Kathryn Tappen, by Bruins GM Peter Chiarelli, former NHLer Tony Amonte, broadcaster (and former player, coach and GM) Mike Milbury and Michael Schuckers, an Associate Professor of Statistics at St. Lawrence University. ("One of these things is not like the others...")
Always good for a lively quote, Burke offered his take early on during the hockey panel, saying, "Statistics are like a lamp post to a drunk: supportive but not very illuminating." It got a chuckle from the crowd, but that may have only masked groans; after all, this was at a conference dedicated to the use of analytics in sports, so the crowd was more stat-savvy than most.
To the chagrin of every stathead, Burke colourfully insisted that numbers couldn't determine a player's character or bravery, with the implication that such things were fundamental to establishing the player's value. While it would be foolish to dismiss character and bravery as desirable traits for a hockey player, the possibility exists that those traits haven't been valued correctly during the Burke era in Toronto.
Mike Komisarek, Colby Armstrong and Colton Orr would all be considered high character guys, and there's no reason to think otherwise, but given their production, isn't at least conceivable that the Maple Leafs might have overvalued the character and bravery aspect of their contribution when signing them as free agents? Just so we're not accused of poking holes at the grinders instead of skill guys, Tim Connolly hasn't yielded a great return on investment either.
I'm not suggesting that Burke doesn't know talent; far from it. He's made a lot of excellent trades in his career, but when heart and grit and character are primary selling features of a player, those who look at the numbers are going to have a hard time justifying big money being paid for intangible benefits.
Peter Chiarelli wasn't quite so bold in his pronouncements, but acknowledged that the Bruins do use an advanced plus-minus statistic, based on scoring chances and that alone is at least somewhat progressive.
The reason that teams might use events, beyond goals, to help establish a player's value is that goals are such rare occurrences in the course of a hockey game that finding evidence of puck possession and territorial dominance needs to be established through other means in order to gain greater statistical relevance. Teams that track scoring chances for and against are adding more data to the mix, so even if that was the entirety of what the Bruins are doing statistically, it's more than just looking at standard goals, assists and plus-minus.
If a team isn't tracking scoring chances for all teams in all games, though, using statistics like Corsi (shots on goal plus blocked shots plus missed shots) or Fenwick (shots on goal plus blocked shots) add more data, which allows for some league-wide comparisons.
So, as Burke dismissed a statistic like plus-minus as useless because a shutdown defenceman on a bad team could have a bad plus-minus -- applying the context that anyone would expect in serious statistical analysis -- it seems more like Burke could be in favour of advanced statistics because they tell a more complete story than simple plus-minus.
Maybe he just hasn't seen the right statistics or they haven't been presented in the best way. I don't think that the shortcomings of antiquated statistics, however, makes for an argument against the use of more advanced stats. On the contrary, it cries for better and more accurate measures.
The point of advanced statistics isn't to come up with a singular plus-minus-style number that provides an all-encompassing value for a player, but relevant statistics, applied in context (considering quality of competition, quality of teammates and zone starts, for example) have to be helpful.
It strains belief to insist that having more knowledge about what a player accomplishes on the ice couldn't possibly be beneficial to a team's decision-making process.
CHAMPIONSHIP OR BUST?
One of the reasons that Burke asserted he wasn't buying the Moneyball concept for hockey was that, in addition to the fundamental differences in the sports, "Moneyball didn't win a championship." In his curmudgeonly way, Burke insisted that the A's should have had a parade before espousing their stat-savvy philosophy in a book.
On the surface it is true that the Oakland A's didn't win a World Series and, as Milbury pointed out, the movie Moneyball didn't pay any attention to the A's trio of starting pitchers (Tim Hudson, Mark Mulder and Barry Zito) that made such success possible. While the A's made the playoffs on a shoestring budget, competing against teams with far greater resources, they did not win a title.
However, it's preposterous to suggest that the concepts of statistical analysis behind Moneyball haven't resulted in titles. The Boston Red Sox hired Bill James in 2003 and World Series titles followed in 2004 and 2007. This from a team that hadn't won a World Series since 1918.
The Dallas Mavericks won the NBA title in 2011 with a stats analyst, Roland Beech (founder of the site
http://www.82games.com), on the bench.
In any case, just because a championship isn't won, that doesn't make the process inherently incorrect. What would one round's worth of home games in the playoffs be worth? That's not enough incentive to maybe consider a smarter method of asset valuation?
In a league that operates with a salary cap, anything that could possibly improve a team's budget decisions ultimately affects the calibre of team that ends up on the ice.
DRAFT
What is the cost of a draft bust? In the same panel, Burke said that all of the GMs have scouted players that they were sure would become NHLers and, by the end of their entry-level contracts, they were never heard from again.
While no amount of stastistical analysis could eliminate all draft busts, wouldn't it be advantageous to improve the value gained from draft picks? What if, instead of one in five picks falling below expectations, a team could make it one in eight? Or what if a team hit a late-round home run once every four years instead of once every ten? It's these incremental improvements that can raise a team's level of competition.
There were a few interesting points made when it came to the draft discussion:
Peter Chiarelli said that the Bruins tend to target heavier players when it came to the draft, finding that a prospects weight had a better correlation to NHL success than height, referencing the advantage for players with a lower centre of gravity.
Teams naturally want to know what kind of teenagers they will be drafting and Burke said that players that are good enough to be high draft picks are almost always captains at some level as they grew up. When asked why he was never a captain (raising questions about attitude or some other red flag), Jaromir Jagr explained that it was because he always played with players that were three years older. Burke said it was the best answer he ever got to that question.
Burke also said that, when he was with Vancouver, he decided he liked Ryan Kesler after seeing him play one shift at Ohio State, telling his scouts he had seen enough and that they should do whatever they needed to get him in the first round. By contrast, Burke said he went to watch Wayne Simmonds play eight games for Owen Sound and didn't see anything in those eight games to indicate that he'd be worth drafting.
These are the uncertainties that NHL general managers face when scouting 17-year-olds. One of the most prominent challenges faced by anyone in the scouting or analysis side of things is determining a player's character, work ethic and desire to get better.
Baseball super agent Scott Boras was on a couple of panels at the conference and, aside from revealing a humanity that was unexpected, the psychology of an athlete was one of the areas that Boras felt was due for further exploration.
For all the physical tools and statistical production that a player might have, it's an ongoing challenge for others (agents, executives, coaches etc.) to know just how motivated a player will be to perform, particularly once they have big money in the bank. The psychology of athletes is a different type of analytics, but one that has been brought up in each of the last two years at the Sloan Conference as being an area for improvement.
AGENTS
What is fascinating is that the acceptance of advanced stats in hockey seems to be coming along a little more easily with player agents, who are trying to find any methods that would indicate their clients are more valuable. It's not unlike the experience that Bill James encountered more than 30 years ago, as James said agents were more receptive to his ideas then too.
One of Amonte's best contributions to the panel was as Burke was lamenting how agents would use numbers to sway arbitrators that might not be terribly familiar with hockey, Amonte piped up that general managers had the same right to establish comparables too.
It's hardly the fault of statistics that one side uses them more effectively to advance an argument in front of a neutral observer.
FUTURE
Now, as down as Burke was on applying the Moneyball movement to hockey, there has been progress. Gabe Desjardins continues to do terrific work on
http://www.behindthenet.ca and he consults with NHL teams. (He had some issues with Burke's comments too.)
Schuckers and Brian Macdonald, an Assistant Professor of Mathematics at the U.S. Military Academy, have presented research at the last couple Sloan conferences and a couple of University of Toronto students, Timothy Chan and David Novati, offered their take on measuring individual point shares for NHL players at this year's conference.
These are all signs of progress. There were more NHL teams represented at the Sloan Conference this year and even if advanced stats are far from receiving the wholehearted acceptance that the stats community would prefer, it is a sign of a movement that is gaining traction; it just might be slower than hoped.
Perhaps the biggest challenge facing proponents of advanced stats is the ability to make the data represent something meaningful and that makes it imperative for those providing the information to sell it in relatable terms. It's not enough to say, "This is Player X's Corsi, so that establishes the entirety of his worth as a hockey player."
Context needs to be applied to tell the whole story.
As challenging as it can be to get buy-in from the front office, it's another level altogether to get coaches and players to accept the data as useful.
There were some interesting points made on various non-hockey panels in this regard.
Mike Zarren, the Assistant GM with the Boston Celtics, talked about how stats analysts might come up with half a dozen points of interest based on their data, which they would then pass on to the coach. The coach might take three or four of those points and express to the players and, ultimately, a player might incorporate one of those tactics effectively in a game situation.
Basketball coach Jeff Van Gundy (a conference All-Star with his quick wit) stated that he would lie to players about their stats if it would help him change their behaviour, but Van Gundy also said that he wouldn't tell a player to take an extra dribble just because the statistics say that a player shoots a few percentage points better after five dribbles, instead of four.
Former major league baseball player Rocco Baldelli, who works as a special assistant for the Tampa Bay Rays, pointed out that even with all the advanced statistical knowledge that exists in baseball, a player has to be willing to apply it if he's going to affect positive change. No matter what information you offer a player, it won't matter if the player isn't inclined to take advantage of the knowledge.
Soccer analyst Alexi Lalas made a good point about incorporating video to help get those points across. It's one thing to tell a player about stats, but another matter entirely to be able to show examples that identify areas for improvement and don't allow a player to make excuses for why the stats might not be in that player's favour.
It can be heartening to see what other sports are doing to improve the level of analysis available but, at the same time, that only increases the frustration for those in the hockey stats community, who would rather have teams make decisions based on the best available information and it's out there, for those that are willing to look.
Scott Cullen can be reached at
[email protected] and followed on Twitter at
http://twitter.com/tsnscottcullen. For more, check out TSN Fantasy on Facebook.
Login
Register
FAQ
Search
