2008-10-21

Projecting goalie performance



The road to the NHL is an arduous one, and all but the supremely gifted must spend time honing their craft in lower professional leagues. This is particularly true of goaltenders. The developing goalie of most interest for Oiler fans is currently Jeff Drouin-Deslauriers (pictured), who played his first NHL game on Friday night, beating the Flames 4-3 right in the Saddledome.

Towards the end of yet another lengthy thread over at
Lowetide, the discussion turned to NHL projections of goalies based on minor league performance. My No.1 fan Slipper posted a number of examples of goalies who had performed at a high level in the AHL before making the link. He concluded:

I really think some where in the range of .920 by the age of 23 seems like it could be the line … that deadline where a goaltender either gets it or not

I decided that rather than “typing guys in randomly over at hockey-reference.com” I would do a little more thorough study. Looking over a current list of NHL goaltenders I found about three dozen who had played at least 100 games in one of the high minor leagues (AHL and/or IHL). I might have missed one or two but I think I got most of them. I listed their career GP and Sv% in their primary minor league, their Sv% and age in their final minor league season of 20+ GP, and then their GP and Sv% in the NHL. It’s an interesting list:

(Note: Two guys played over 100 games in each high minor league so both lines are listed; a third, Kevin Weekes, clearly made a career leap in a briefer stop in the "I" which it would be disingenuous not to mention. Otherwise I made no attempt to blend in the occasional few extra games in the "other" league)

Goalie ********** Minor GP, Sv% (Last, Age)
/ NHL GP, Sv%
---------------------------------------------

Craig Anderson ** AHL: 146, .914 (.919, 25)
/ 79, .904
J.-S. Aubin ***** AHL: 129, .907 (.899, 28)
/ 218, .900
Alex Auld ******* AHL: 150, .909 (.909, 24)
/ 141, .904
Martin Biron **** AHL: 103, .921 (.930, 21)
/ 381, .910
Ilya Bryzgalov ** AHL: 199, .913 (.902, 24)
/ 129, .914
Peter Budaj ***** AHL: 133, .917 (.919, 22)
/ 130, .901
Ty Conklin ****** AHL: 117, .914 (.914, 26)
/ 110, .909
Mark Denis ****** AHL: 131, .908 (.910, 30)
/ 348, .902
Jeff Deslauriers* AHL: 132, .906 (.912, 23)
/ 1, .897
Wade Dubielewicz* AHL: 164, .920 (.922, 29)
/ 37, .920
Dan Ellis ******* AHL: 140, .904 (.894, 26)
/ 51, .918
Manny Fernandez * IHL: 246, .909 (.916, 24)
/ 299, .912
Mathieu Garon *** AHL: 246, .915 (.927, 27)
/ 188, .906
J.-S. Giguere *** AHL: 134, .905 (.917, 23)
/ 416, .915
Josh Harding **** AHL: 118, .924 (.920, 22)
/ 39, .916
Johan Holmqvist * AHL: 193, .905 (.908, 25)
/ 99, .890
Brent Johnson *** AHL: 160, .903 (.911, 22)
/ 229, .902
Jason LaBarbera * AHL: 247, .928 (.933, 27)
/ 84, .902
Patrick Lalime ** IHL: 175, .899 (.900, 24)
/ 399, .905
Pascal Leclaire * AHL: 101, .909 (.915, 21)
/ 117, .910
Manny Legace **** AHL: 119, .906 (.897, 23)
***************** IHL: 116, .912 (.913, 26)
/ 312, .914
Michael Leighton* AHL: 266, .917 (.931, 26)
/ 52, .901
Chris Mason ***** IHL: 124, .910 (.917, 25)
***************** AHL: 146, .914 (.921, 26)
/ 136, .912
Ryan Miller ***** AHL: 172, .922 (.922, 24)
/ 208, .908
Antero Niittymaki AHL: 147, .918 (.924, 24)
/ 132, .899
Andrew Raycroft * AHL: 122, .911 (.917, 22)
/ 201, .900
Dwayne Roloson ** AHL: 179, .911 (.929, 31)
/ 399, .909
Curtis Sanford ** AHL: 156, .914 (.901, 25)
/ 91, .900
Mike Smith ****** AHL: 127, .914 (.917, 23)
/ 60, .908
Jose Theodore *** AHL: 110, .912 (.917, 22)
/ 447, .914
Mikael Tellqvist* AHL: 146, .914 (.921, 25)
/ 92, .895
Tim Thomas ****** AHL: 119, .922 (.923, 31)
/ 168, .914
Hannu Toivonen ** AHL: 128, .920 (.909, 22)
/ 61, .890
Vesa Toskala **** AHL: 156, .909 (.903, 25)
/ 185, .910
Marty Turco ***** IHL: 114, .918 (.916, 24)
/ 388, .912
Steve Valiquette* AHL: 197, .916 (.909, 29)
/ 27, .918
Kevin Weekes **** AHL: 111, .885 (.895, 21)
***************** IHL: 045, .919 (.919, 23)
/ 332, .902
-------------------------------------------

First things first: there's no Brodeurs, Luongos or Lundqvists to be found here. Those guys were correctly fast-tracked to the bigs. No diPietros or Fleurys either. A couple of near misses like Lehtonen (98 GP), Kiprusoff (87 GP) and Vokoun (78 GP) were omitted. Meanwhile, a guy like Olaf Kolzig played his 139 AHL games before they kept Sv% stats in that league. It follows that some of the career minor league Sv% for the older guys may be incomplete.

Now I don't know the current formula for projecting NHL performance for goaltenders -- and I'm not about to pay "Rotowire" to see an archived copy of Behind the Net's valuable research to this end, whazzup with that? -- but surely the expectation would be for a goalie's Sv% to drop. So it is with (only) about 70-80% of all cases. Turns out there's a couple guys who have posted better numbers in the NHL than they did in the minors (Ellis, Fernandez, Giguere, Lalime, Legace); although in the majority of those cases the guy was around his now-established NHL level in his last season in the minors. Ellis is the exception that came right off the board; his .924 that led the NHL last year was .013 better than his best minor league season. Wanna bet on a repeat?

There's quite a few more who are pretty much level (+/- .002, including Bryzgalov, Johnson, Leclaire, Mason, Roloson, Theodore, Toskala, Valiquette so far). So that's fully a third of the entire list whose Sv% stats stayed about level or better, at least relative to their career rate. In about half of these cases the guy had a great last year in the minors. In summary, 11 of the 37 goalies have a better or equal career Sv% at the NHL than in the minors, and 8 maintain a higher average than their last season at the minor pro level.

As for the magic markers: only 12 of 37 recorded a Sv% of .920 or better their last year of 20+ GP in the minors. Another 10 were in the .915-.919 range. So while .920 is certainly nice, it's not necessary. On the other side, fully 12 of the 37 had a sub-career-rate Sv% their last year in the minors.

Final minor season Sv%
----------------------
.890-.894 = 1
.895-.899 = 1
.900-.904 = 4
.905-.909 = 4
.910-.914 = 5
.915-.919 = 10
.920-.924 = 7
.925-.929 = 2
.930-.934 = 3

As for the age of "getting it", a lot of these guys were still buried in the AHL well beyond 23, with many of them gradually improving to somewhere near the .920 level (Anderson, Dubielewicz, Garon, Leighton, Mason, Raycroft, Roloson, Tellqvist, Thomas) before “earning” their chance at whatever age. At the NHL level this group has had varying degrees of success, although it’s fair to say none of them can be considered among the top tier of goalies in the league. Thomas was on the outer boundary of the top ten goalies since the lockout in this blog’s first look at the state of goaltending in the NHL, while Roloson likely* achieved similar status just before the lockout. (*I haven’t done the actual study, but with Sv% of .927 and .933 in 2002-04, Roli was rolling.) Averaging out the entire list, they played their last "full" minor league season at age 25. Just 12 of the 37 (including, speculatively, Deslauriers) were essentially done with the minors by age 23. Only two goalies on the entire list -- Marty Biron and Josh Harding -- met the double standard :) of their last minor league season by 23 with a Sv% of .920; another seven achieved .915 by that age and then graduated, so maybe that's a more achievable "line in the sand". But it's still faster than most.

Final minor season age
----------------------
21 = 2
22 = 6
23 = 4
24 = 7
25 = 5
26 = 5
27 = 2
28 = 1
29 = 2
30 = 1
31 = 2

Looking across the board it's hard to find a common thread. For every general rule there's a solid minority of exceptions. For every Marty Biron, Jose Theodore or Josh Harding who dominated the minors at an early age, there's a Dan Ellis, Manny Legace or Chris Mason who took years to "get it". Oiler fans will note that both Dwayne Roloson and Matthieu Garon fit the "late bloomer" description, as indeed do former Oiler prospects Tim Thomas, Steve Valiquette and Ty Conklin.

As for the youngster currently challenging the veteran Oilers, JDD's AHL Sv% of .912 in 2007-08 is slightly below the mean graduating level; but bear in mind that at 23 he would be among the younger 100+ GP guys to successfully make the jump. Considering this, it would neither surprise nor disappoint greatly if he remains one year away. It would, however, be disappointing to lose him on waivers at this point; his improvement from .888 to .897 to .908 to .912 has been steady, especially considering some of the chaos in the Oilers farm "system"; if he spends another year raising that into the .915-.920 range that would be just fine.

It is incumbent on me to point out that such studies are rife with ambiguities. For one thing, this one is incomplete in that it excludes the guys who didn't need 100 games apprenticing in the minors, most of them top line goaltenders. For another, Sv% may be the best available measurement of goalies, but it's far from perfect. Much depends on team play. Some of these guys may have parachuted into a cushy situation at the NHL level (e.g. Legace, Turco, Mason, Ellis), while others were thrown to the wolves (LaBarbera, Niittymaki, Tellqvist); similarly some no doubt came from great defensive minor league clubs and some from not-so-great (I didn't pursue this, but the goalie's own W-L record would be a strong clue). One might expect such transitions to trigger a similar effect as what happened to Mike Smith last year when he got traded from Dallas (.906) to Tampa (.893), but that in itself is a topic for further research, and for another day. For now I'll throw this information out there and let you draw your own conclusions, which I certainly welcome in the comments section.

23 comments:

raventalon40 said...

I think it's important to consider the farm situation too. For DesLauriers at least, playing on a shifting farm team ( Edmonton (A), Wilkes-Barre Scranton, Springfield, Edmonton(N) ) and Devan Dubnyk ( Stockton, Wilkes-Barre Scranton, Springfield ) it was a problem of sharing ice time. Even earlier than that was the time period when we had to share farm space with the Habs in Hamilton. I'm not sure if the playing minutes had a huge impact on other goaltending prospects, but considering the past tends to repeat itself I'm sure other goalies who took a while to "get it" probably went through a similar experience.

Bruce said...

RT40: Thanks for the reminder, I had meant to return to JDD in the above and forgot. I have added a paragraph near the bottom.

You are absolutely right, it is some combination of nature and nurture. Some guys need time to develop, others need the right opportunity to open up, still others seize the chance early and do their developing right in the NHL (e.g. Cam Ward, damn him). And some guys like JDD have to stay in a holding pattern while their organization gets its act together.

PerformanceOil said...

Bruce,

I was taking a look at the goaltenders as well today (though I didn't get nearly as far as you), and I came to pretty much the same conclusions.

There seems to be two tiers:
Elite goalies, who spend negligible time in the AHL (even if their NHL numbers aren't great), and the next tier which seems almost random in when they get their chance (a lot of times it is after a poor season in the AHL rather than their career best season, that they get the call-up). Most of the early NHL guys who start out badly seem to get there eventually though, so the scouts are obviously seeing something in these guys. From tier two, it seems a grab bag. Some elite AHL performances (though usually not consistently), sometimes at an early age, sometimes not. If I find time, I want to look at the AHL stars who don't make it, and see if there is anything of interest there.

The one thing that seems to be pretty consistent (from casual inspection) is that the better goalies either broke in quite young, or after a long time in the minors/Europe. So, you're either a star or a project, with not a lot in between. Of course, in some cases it might be less lack of talent, and more lack of opportunity that keeps guys in lower leagues as well (not many goaltending jobs available in the NHL).

As far as JDD goes, his chance of making it is certainly pretty bad (since he isn't tier 1). However, it probably isn't any worse than a whole bunch of other tier 2 guys, some of who will make it. If he does become a starter, it's not going to be for several years, most likely.


Oh, it did seem as if Slipper's random number generator was broken when he did his unbiased search.

Sean said...

Here is last years top 20 AHL goaltenders by save percentage.

http://spreadsheets.google.com/ccc?key=pD8REcipK_hlsS6Kn82L2Fg

8 of them are younger and 5 of them are younger with a better SP. R40's point about JDD's wasted season plus injury problems is valid. The wasted development season essentially made the Oilers give him a one-way deal. With another year under his belt, it would be easier to assess how good JDD really is. Either way, only 5 teams have younger better goalies. That leaves 25 with a possible upgrade. Do you think its reasonable to assume he would be lost on waivers?

mc79hockey said...

Either way, only 5 teams have younger better goalies. That leaves 25 with a possible upgrade. Do you think its reasonable to assume he would be lost on waivers?

(Keeping in mind I'm not even necessarily accepting your premise here.)

Yes, when you consider that they'd have to keep him in the NHL to keep him.

Slipper said...

I gotta be honest, I was thinking biasedly when I was doing my 10 player, 5 minute "research" project in that passing conversation over at LT's site.

I initially conjured up goaltenders who had shown competence at a starting level in the NHL and worked backwards.

My bad.

Slipper said...

It's would also be fair to note that, as the conversation had been triggered by oilman in an earlier comment, I was initially seeking players who had split reasonable time between the NHL and AHL within one season.

So I wasn't really aware there was an expectation of either thoroughness or a lack of bias.

Why do you hate me BRUCE!!!!

*sobs! and temporarily quits internet*

;)

raventalon40 said...

One example of a guy who took a while to make it to the big show: Dwayne Roloson.

Bruce said...

Thanks for the list, Sean.

Do you think its reasonable to assume he would be lost on waivers?

Well JDD's no Bryzgalov (last season's waiver steal), but there's gotta be a team or two out there with a current weakness at backup who might take a shot at him. Not to mention a team or two with a real hate on for the Oilers. JDD is just nearing that tipping point where he could emerge as an NHL (backup) goalie in the next year or so; it would seem a shame to have wasted all that development time.

That said, looking over that list of guys with 100+ GP in the AHL/IHL, the large majority of them have changed employers at least once. The exceptions are the young guys who achieved something close to Slipper's .920 at 23 threshold: Budaj, Harding, Leclaire, Miller, Niittymaki, Turco. Others in that category (Biron, Raycroft, Theodore) were given a real shot with their original club before being traded later in their respective careers.

The lion's share of the other guys (PO's "Tier 2" guys) bounced around early and often. The odd sock exception is Wade Dubielewicz who has been Islanders property since signing as an NCAA graduating FA at 25; Doobie's eye-popping rookie pro season with Bridgeport Sound (20-8-5, 1.38, .946 !!) might explain the Isles' patience with this guy, who as of this writing has a .920 career Sv% in both the AHL and NHL, and for all I know is the next coming of Tim Thomas.

While I agree with PO that JDD is more a Tier 2 guy, he has followed the idealized development curve fairly well and has improved his Sv% every year, albeit to a sub-elite level. Still just 23, it would be a dangerous time to give up on him IMO.

On the other hand, the high minors would seem to be a bubbling cauldron of replacement level goalies who simply have to be in the right place and time to get their chance, so it's not like he would be irreplaceable or anything.

Bruce said...

I was initially seeking players who had split reasonable time between the NHL and AHL within one season.

Slipper: I did notice that your examples were partly within one season and partly career rates. And hey, no worries, one man's "cherry picking" is another man's "targeted research". (Or maybe that should read two men's cherry picking, eh, PO?)

Anyway, your initial efforts and assumptions prompted me to do some more rigorous research, which is a good thing, right? Hardly rigorous enough though, in that by focussing on the minor leaguers who reached the 100 GP threshold I deliberately omitted most of the fast-track guys, the ones who are even better than the Harding-Leclaire-Miller class identified in the comment immediately above.

Perhaps later in the season I will extend the study to include every current NHL goalie including the high-draft hotshots and the career Euros.

raventalon40 said...

What do you make of the goalie situation for the Oil Kings?

Oilman said...

Good Work Bruce. It's really what I assumed it to be - not so clear cut.

BTW, I honestly didn't cherry pick the goaltender i used as an example - he was just the first guy that sprung to mind when I tried to remember goalies that have split significant time between 2 leagues in the same year - it doesn't happen all that often in one season, but Giguere did it over 3 consecutive seasons with 4 different teams - so his example was really the best I could have come up with (completely by fluke)...I also remember suggesting Dipietro as another good example before checking his stats and his AHL numbers are phenominal - so what the hell do I know.

dstaples said...

As always, Bruce, your work is a breath of fresh air, for its clarity, its thoroughness, and its careful interpretation.

The Oil's problem in net is that there's one too many guys between Jeff D-D, Roli and Devan Dubnyk.

Clearly, though, Jeff D-D is still on track, and it will be interesting to see how he does this season.

I wonder how much Dubnyk's development would be hurt playing at the ECHL as opposed to AHL level, so long as he played every game at the ECHL? That, of course, would take another study.

I also wonder if Roli hadn't signed for as many years as he did, if the Oil would have brought him back this year? I doubt it, and that's the problem of tacking on extra years at the end of an aging player's contract.

Bruce said...

What do you make of the goalie situation for the Oil Kings?

RT40: Haven't been tracking the Kings that closely this year. Went to few games last season and in a couple of look-sees was severely unimpressed with Dalyn Flette. While I am aware these young guys can often turn a page from one season to the next, his .847 Sv% so far in 2008-09 suggests it might be time to turn the page on him. Whatever, apparently he's now hurt and the Oil Kings have acquired Torrie Jung, a 19-year-old from Kelowna via Lethbridge.

I hope the newcomer will bring a Jungian philosophical bent to the task at hand; he may well have to provide some "unconscious" goaltending for this team to succeed.

Doogie2K said...

Since we're temporarily speaking of Oil Kings goalies, it seemed to me that Alex Archibald was a good goalie in a shitty situation last year, in my three or four chances to see him at the 'Dome. Does that line up with your own impressions of him?

Bruce said...

Doogie: Yeah, Archibald played pretty darn well in most of the games I went to. He had a tendency to let in the odd weak goal at a bad time, especially late in games. Same is true of many goalies who spend too much of the game with the puck in their end ... they get tired.

Whereas Flette has been getting burned early in games, based on the game reports I've read.

BTW, I went to one Hitmen game at Rexall last year. The Hitmen dominated but Archibald played great, took the game to OT when Calgary scored to win it, 2-1 as I recall. On my way out of the building I filed out next to a couple of guys in Hitmen sweaters, and complimented them on their team's performance. Well all they could do was shit on their own team esp. the two players who had just been named as game stars (after Archibald), saying things like WTF were the star-pickers watching, X and Y played the shits tonight, we should have won by four or five goals, etc. And all I could think was "some fans you are." So much for reaching out.

Oilman said...

Hey Bruce...I just noticed the blogroll nod...Thanks. I guess that means I'll have to write something again.

Bruce said...

Oilman: You're welcome. I've enjoyed your early stuff, keep it coming.

On the subject of blogroll maintenance, somehow the link to Doogie2K's site has fixed itself. It sure wasn't any computer wizardry on my part.

Doogie2K said...

I noticed that not too long ago. Must've been a defective hamster.

I dunno, I sometimes get confused by the star picks in junior, but it's no big deal. It's junior. My seats cost $10. Not worth it.

I do get the sense that Plante might be this year's scapegoat/fan-basher pick, judging from the mumblings in Section 108, which is nuts, because he's an obvious Top 4 guy who's largely returned to his '07 form, and gained a bit of speed during the off-season, but I guess a trade demand will do that for you. But at the same time, I've also learned to acknowledge my own biases and try to reign them in: I shat on Seabrook a lot last year, and I maintain it was deserved, but it's plainly obvious that he's gotten a lot better this year, and deserves the top-four slot. Whatever. Just goes to show that at any level, you have good fans and bad fans.

Bruce said...


Just goes to show that at any level, you have good fans and bad fans.


D2K: True that. I suspect most of us are both, with a large variance from player to player. We have our favourites whom we are willing to forgive their occasional transgressions, and our not-so-favourites whose transgressions are all we (choose to) see, or at least dwell on.

But some fans are just arseholes, and I encountered two of them that night. Wearing Calgary sweaters ... who knew?

raventalon40 said...

I wonder how much Dubnyk's development would be hurt playing at the ECHL as opposed to AHL level, so long as he played every game at the ECHL? That, of course, would take another study.

I also wonder if Roli hadn't signed for as many years as he did, if the Oil would have brought him back this year? I doubt it, and that's the problem of tacking on extra years at the end of an aging player's contract.


Depends on how many guys come out of the ECHL with an NHL career, I suppose.

As for the Roli contract, I agree and disagree. We can see with the Geoff Sanderson/Curtis Glencross situation that the Oilers will usually not bring back guys they feel are expendable. There's no way the Oilers could have predicted the DesLauriers waiver situation, unless it was thoroughly intentional to have him back up Garon this year, which I doubt.

However, the fact that the Oilers haven't bought him out yet or engaged in a salary dump (per se, trading for a low round pick) is due to some perceived usefulness - the fact that season is still young, and also that DesLauriers is a rookie and Garon ended the last season injured.

All this should mean that due to the season being young and the fact that there are particular uncertainties regarding the goaltending situation, there is still a good chance that Roli would have been around anyway.

Doogie2K said...

All this should mean that due to the season being young and the fact that there are particular uncertainties regarding the goaltending situation, there is still a good chance that Roli would have been around anyway.

Yeah, but he might have been a couple of mill cheaper.

Bruce said...

Depends on how many guys come out of the ECHL with an NHL career, I suppose.

It's not unheard-of, in fact at a guess there's a higher percentage of goalies coming out of the lower tier minors than position players. A less than exhaustive search found at least 10 current NHL goalies -- Dan Ellis, Jaroslav Halak, Olaf Kolzig, Jason LaBarbera, Patrick Lalime, Joey MacDonald (new Islanders' goalie excluded from previous study), Dany Sabourin, Curtis Sanford, Mike Smith, Steve Valiquette -- played at least a significnat chunk of one season (20+ GP, to a max of around 50 career GP) in the ECHL, or 1/6 of all goalies and almost 1/4 of North American trained 'tenders. I'd call that pretty significant.