September 24, 2009

Welcome To Our World

What goes into the construction of Golf Digest's annual Hot List? As Mike Johnson details, a lot more than people think

The sheer volume of equipment Hot List judges have to review is staggering.

The sheer volume of equipment Hot List judges have to review is staggering.

Let's get one thing straight. Do not construe any of the following as whining, complaining or anything of the sort. I always tell people one thing about my job -- as cool as they think it is, it's actually way cooler than that. And at no other time during the year is it cooler than when our annual Hot List process heats up.

Still, most folks have no idea what goes into our annual review of golf equipment. Most think we swat a few balls and come up with a list. Not. So. Much. The photograph you see here was born out of our photo editor being staggered by the number of clubs flowing into our offices here in Wilton, Conn. Of course, what you see in this picture represents only about two-thirds of the product that we will eventually review. And no, we don't keep the clubs. Golf Digest policy is straightforward: Any clubs that stay in the bags of equipment editors must be paid for. After Hot List testing clubs are either returned to companies or, if the company wants, donated to various charities.

So, how do we actually do the Hot List? Here's an inside look -- and welcome to our world.

To paraphrase Vince Lombardi, the Hot List is not a sometime thing -- it's an all-the-time thing. Fittingly, how we put it together is not unlike how manufacturers produce golf clubs -- it takes the efforts of a great number of qualified people over a lengthy period of time. And it starts pretty much as soon as the previous Hot List issue is printed.

For starters, we listen -- a lot. We take in praise and complaints; kudos and the concerns. We speak with people in the industry—manufacturers, retailers, golfers and readers, with many changes based on what we hear. For example, our reader surveys over the years revealed the Demand category carried the least weight with readers. So what started out as 20 percent of the score is now down to 5 percent. Conversely, almost everyone felt Performance/Playability wasn't being given enough weight. So we boosted that.

To enhance our knowledge of products and technologies, each of the four judges has flown more than 25,000 miles this year, getting sneak peeks in research and development labs and formal presentations on equipment from the bright minds at the equipment companies. This is not some sales presentation, either. Companies regularly send us away with 100-plus-page white papers explaining everything from club moment arm to d-values to specific gravity. Make your head hurt? Mine too. Which is why two years ago we took a crash course in physics from Dr. John McPhee, a professor of systems design engineering at the University of Waterloo in Canada, to help further our understanding of such things. I still have my notebook from that course -- and I refer to it often.

Come August the process kicks into full swing. The visits to manufacturers become more frequent (since August we have either visited or been visited by 14 equipment companies -- yes, we were invited and yes, we pay our own way -- including picking up the dinner tab when we go out with any company representative). We recently went through our preliminary stage where we narrowed the initial field of more than 350 products by play-testing clubs prior to our summit in October. True, it's the fun part of the job. But after a while even banging balls on the range or course can become somewhat tedious. Overriding that, however, is the fact that with each decision we make on a club we are well aware we are having an impact on what a consumer may purchase as well as a company's business. And while we don't let that sway our decision, it does help keep us focused on the job at hand.

This year the process hits its high point during a 10-day summit at the Casablanca Resort in Mesquite, Nev., in late October, where we will have 20 player panelists play-test more than 150 finalists and then discuss the merits of each. During this time we will spend two days speaking to our academic panel to further evaluate technology trends and claims. Our retailer panel will be on hand to discuss what will make a product hot in 2010. In all, approximately 20,000 balls will be struck -- with our panelists often found beating more balls during their lunch breaks to better understand the subtle differences between products. The judges, in addition to talking with the panelists and gathering their input, will enjoy a little light reading in the form of more than 1,000 pages of technical data.

And of course, we'll also be busy reporting back to readers in the form of blogs and even video from the summit, all in the interest of enlightening them on the process.

The summit, however, is in many ways more the start than the finish. After returning to our offices we huddle in a conference room for the better part of two weeks for voting. Like judges in chambers (and with the tenacity Tiger Woods reserves for major championships), we together debate the merits of each individual product over each of our four criteria. In other words, nearly 1,000 separate debates. In the end, we reach a consensus on each category that becomes our Hot List. I recall one time when Jerry Tarde, the editor-in-chief and chairman of Golf Digest, saw us there late one night and asked how long we had been at it. I remember telling him and adding, "After all, now's not the time to get lazy."

Honestly, there is no good time to get lazy on the Hot List. True, big companies tend to dominate the list, but as I tell people, large companies are usually large companies for a reason -- they tend to make pretty good products. But we also are thrilled to discover the hidden gem from the company no one has ever heard of. Our mission statement for this project is simple: "Helpful to the reader, fair to the industry, true to ourselves." Readers are counting on us to help point them in a good direction for spending their hard-earned cash. Manufacturers are counting on us to give every product a fair shake. And we're counting on ourselves to do the most thorough job we can. As NASA flight director Gene Kranz once said, "failure is not an option."

Welcome to our world.