Stephen J. Smith was perplexed in golf's time-honored struggle between a caddie's recommendation and his own opinion.
The caddie had been clear: 6-iron. But Smith, a 43-year-old attorney from Dublin, Ohio, with a 7.4 Handicap Index, felt more like 5-iron. So naturally, he ignored the caddie and went with his gut.
The 5-iron flew half a club too far, settling behind a tree root and leading to an infuriating 6. The caddie knew Stephen J. Smith better than Smith knew himself.
But the caddie wasn't scowling, nor did he wrap a consoling arm around his man and steady him as they walked to the next tee. The caddie, as it turns out, doesn't walk, and isn't a he or a she. It's a kind of machine called Arccos Caddie that can live inside your smartphone. And, having processed more than 70 million shots and 368 million geo-tagged data points, it not only knows Smith and his tendencies, it has the potential to know everything there is to know about every golfer and every shot that could ever be hit in every situation on any golf course from any lie and in any weather event there could be.
This is golf coming face to face with Big Data, a meeting that could change the game's future, in this case restoring the bond between player and trusted caddie, even if the caddie is artificially intelligent.
Since the invention of the pencil, golf has been leaning in toward its numbers, and with the emergence of Big Data, machine learning and artificial intelligence, the game is about to get smarter than it has ever been. Smarter players, smarter course operators, smarter equipment manufacturers. The measurement and analysis of golf's numbers will eventually change every element of the game.
Thanks to rapid advancements in computer science, A.I. has been making big news in all forms of business and marketing since its evolution from sci-fi fantasy just a few decades ago to everyday reality. A.I. is how Amazon knows what brand of toilet paper you need to order before you do, or how Netflix finds you the perfect movie for a Friday night despite the fact you've never heard of Giovanni Ribisi. A.I. is the foundation for commercial applications such as IBM's Watson, that Stephen Hawking-like black box that crushed a couple of all-time "Jeopardy!" champs in a special man vs. machine match in 2011. A.I. is what drives your GPS system, and it's what will be driving your driverless Google car in the not-too-distant future.
What A.I. says is, we are our data, and the numbers never lie. And now it has found in golf a natural place for its capabilities. Despite being a sport known for its volumes of words and mountains of images, golf is really a numbers game. And the numbers are huge, the very embodiment of Big Data. There is no more vivid example in golf than the PGA Tour's ShotLink system, which tracks every shot hit every week and produces player data in 653 statistical categories, including the revolutionary strokes gained statistic, a predictive-analytics model that calculates how a player is performing from every place on the course compared to the field.
Data at that level has been the exclusive domain of tour professionals—until recently. Over the past few years, in-round stat programs, like those from leaders Arccos and Game Golf, have tracked almost three million rounds and more than 250 million shots, providing amateur golfers with statistical profiles for every club in the bag. However, given that the GHIN system records 50 million rounds a year, it's clear that golf is barely scratching the surface of Big Data. This growing surge of information might be the game's most promising frontier.
"Golf is one of the better sports for data," says Game Golf founder and CEO John McGuire. "It's very data-driven. Our job is to take the data, contextualize it, and make it useful. Data on its own is no good; data on its own is dry. It's the stories you tell from the data. That's what's relevant."
Perhaps the biggest step in golf's Big Data revolution came this May when Arccos launched Arccos Caddie as part of a partnership with Microsoft using their Azure Cloud. The Arccos 360 system ($249.99) works with a smartphone and measures shots using sensors placed in the grip of each club. The Arccos app is free, and Arccos Caddie is available for an additional fee. For now, Arccos Caddie provides club recommendations only off the tee. Why only off the tee? Because those recommendations can be made before a round begins, which makes it legal in the eyes of the USGA. If digital recommendations were "live" during a round, like on approach shots, they would violate Rule 14-3, which governs the use of artificial devices and unusual equipment, says Thomas Pagel, the USGA's senior director of rules and amateur status.
"Golf is still a game of skill and judgment, and anything that can give a player an advantage and diminish that judgment is a problem," Pagel says. "The compilation of two or more data points to provide some recommendation that takes that judgment away from the player, that's where the issue comes in."
What the rules-makers are wrestling with is this burgeoning interface between human knowledge and computed data. The work that Arccos Caddie does theoretically could be done by a human, but certainly not as fast. For example, Arccos Caddie analyzes not only a player's data but the data of all similar golfers in its system and all golfers who have played that particular hole. Based on GPS information, weather forecasts, topographic maps and proprietary algorithms, it calculates the terrain and the forecasted wind and temperature and assesses their effects on your location. This all goes into the mix before it spits out a club recommendation and displays your predicted score, your odds of making par with that club and your odds of hitting or missing the fairway or green. And it does that in less than three seconds. For example, as Smith played the 175-yard 14th hole in our test, Arccos Caddie picked a 7-iron with a 45 percent chance of hitting the green, a 29 percent chance of coming up short and 19 percent missing left or right. The odds of making par: 52.3 percent. He took the advice, hit the green and made 3. It doesn't read greens. Yet.
"The knowledge base for Arccos Caddie is growing rapidly, but I would say even today, its understanding of golf data is already well beyond the collective understanding of us as humans," said Sal Syed, Arccos co-founder and CEO. "It's on another planet. The weird thing about machine learning is, it's very hard to explain how it's doing what it's doing. Even the people who built it can't do it."
This machine mind is everywhere around us, and this new field of predictive analytics is what's commonly called machine learning. In short, it's the super-fast use of algorithms, computer-based decision trees, if/then propositions and rules that eventually point to a decision.
"This can get very complicated very fast," says Raghu Machiraju, interim faculty director for translational data analytics at Ohio State University. "There's no explicit mathematical model that could tell you on this day on this hole you should use this club, because there's so much randomness in the process. So you create computing elements that are actually inspired by the neurons in your brain. It requires a tremendous amount of data, but eventually a lot of the learning in this context is not that dissimilar from the way you yourself would learn anything."
In a TED Talk two years ago, data scientist Jeremy Howard noted that A.I. already is better in some instances at diagnosing cancer than expert pathologists are. "We now know that computers can learn to do things that we actually don't know how to do ourselves. . . . The better computers get at intellectual activities, the more they can build better computers to be better at intellectual capabilities, so this is going to be the kind of change that the world has never experienced before."
That is precisely the informational precipice upon which the rules of golf sit at the moment. There is a virtual data explosion that could let all golfers know not only how far they hit every club but whether they chip better with a 50-degree wedge than a 54-degree. It knows with cyborg certainty that a 25-handicapper on a short par 4 will score on average a half-stroke lower hitting a 3-hybrid instead of a driver off the tee, even though his 3-hybrid goes 42 yards shorter than his driver (see chart, below). Is that an unfair advantage? The ruling bodies want to maintain the human element in the game's strategic decisions, Pagel says.
"More and more data points are going to exist that can be aggregated together," he says, "so what does that mean for playing the game? The rules shouldn't be viewed as a hindrance to technology and innovation. They provide structure so that we're able to be thoughtful about where the game might go and ensure that it doesn't become robotic."
Still, delineating what kinds of information are allowed can be murky, and it risks alienating a generation that consumes more data in a single morning than its ancestors from a century ago did in a lifetime.
"I think as we introduce data and, more importantly, the analysis of data, I look at that as more efficiency," said Mike Downey, director of sports technology engineering for Microsoft. "It's not about making things easier; it's about making your time more efficient. Golf lags behind all sports when it comes to those efficiencies. I can understand the reservation that we don't want to let technology make everyone a great golfer, but that's not going to happen."
Maybe, maybe not. Golf's data surge creates powerful information, self-knowledge that can have a profound effect not just on your scores but on your enjoyment of the game. Big Data, A.I., machine learning are all ways that, if properly mined, might increase interest in the game. (Data mining and machine learning already are shaping how courses are maintained, how sponsors gauge the use of their marketing dollars, even how clubs and balls are designed.) And this is without even considering the social aspects of a data-linked golf community, key features to the Arccos and Game Golf apps. Already these apps feature virtual contests between golfers playing courses all over the world at the same time. Arccos even lets you text your buddies with a diagram of how you just played your last hole.
"Bringing the Cloud to golf helps it stay relevant to the times," Syed says. "If golf resists these developments, then golf won't be a part of our lives. It seems a little backward to think that that should be disallowed."
Back to Mr. Smith and his test run with Arccos Caddie. It was easy to see him beginning to develop a rapport with the app's cold, hard numbers, eager to see what it would recommend at one moment, disagreeing with it the next. After a pulled 3-wood into a hazard, Smith felt he wasn't alone: "I'm sensing some disappointment from my phone right now," he said, joking only a little.
At the very least, the confirmation effect of the app's advice was palpable—and Smith still wanted more. "I would have liked to have had it down in Myrtle Beach last week," he said. He also found himself wishing it was there with recommendations on approach shots, particularly after one flared tee ball left him with a blind approach and water in play.
Syed says those expanded capabilities are coming to Arccos Caddie, perhaps before the end of the year. For now, though, the USGA clearly is still not completely endorsing the idea. But Syed thinks he has a solution: "I've looked at the rules, and nowhere do they define that a caddie has to be human," he says. "So maybe there's a loophole."
GETTING ADVICE FROM A CADDIE WHO HAS SEEN 70 MILLION SHOTS
In the charts below, Arccos Caddie uses artificial intelligence to make tee-shot recommendations for three golfers playing a short par 4. Perhaps surprisingly, it recommends driver for the longer hitters and 3-hybrid for the shortest. You might have a theory on why, but even Arccos doesn’t know exactly how its program's machine brain does what it does.