NFL teams (but not the Eagles) are starting to embrace a second intelligence test that can help with draft evaluation

Manning or Leaf?

Twenty-one years ago, that was the great debate leading up to the 1998 NFL draft. Two potential franchise quarterbacks, and one team -- the Indianapolis Colts -- trying to decide which of them was The One.

As most everyone knows, the pair ended up having disparate careers. Peyton Manning became one of the greatest quarterbacks in NFL history; took his teams to the playoffs 15 times in 17 seasons; won two Super Bowls; threw more touchdown passes than anyone in history; already has a spot reserved in Canton’s class of 2021.

Leaf, well, Leaf turned out to be the quintessential bust. Went a pick after Manning to the San Diego Chargers and played just four years, lost 17 of 21 career starts and threw just 14 touchdown passes and 36 interceptions before quickly fading into oblivion.

But in the months leading up to that draft, Manning or Leaf? seemed to be a flip-a-coin decision.

Scott Goldman was a graduate student at Hofstra University at the time, working on PhDs in clinical and school psychology. But he also was a football fan and closely followed the Manning-Leaf debate.

“Leaf had all of these physical measurables,’’ Goldman said. “He had the stronger arm. He was bigger, taller, faster. He had that -- quote, unquote -- upside potential that everybody covets.

“Peyton had less of those physical attributes, but a long history of winning [at the University of Tennessee], a father who was a Hall of Fame quarterback.

“I just thought, isn’t it interesting all of these different data points. I wondered about intelligence. Wondered if anybody had compared that. I did some research, and what I found was, I think we can build a better mousetrap.’’

Goldman’s mousetrap -- the Athletic Intelligence Quotient, a test that measures an athlete’s sports-specific cognitive abilities -- took a while to build. Nearly 15 years, to be exact.

But eight years after finally taking it to market, the AIQ is gaining acceptance among professional sports teams, including many in the NFL. Goldman’s company, Athletic Intelligence Measures, has more than a dozen NBA and major league baseball teams under contract.

The AIM team, which also includes fellow psychologists and AIQ co-developer Jim Bowman, and Alex Auerbach, will be at the NFL Scouting Combine in Indianapolis next week for the eighth straight year administering the test to prospects there. About a quarter of the NFL’s 32 teams currently are AIQ clients.

The Eagles, despite an emphasis on analytics in recent years, are not yet one of AIM’s clients.

Neither executive vice-president of football operations Howie Roseman nor vice-president of player personnel Joe Douglas would talk to the Inquirer for this story, though another club executive said they view it as “similar to the Wonderlic.’’ The Wonderlic test is a 50-question written test that has been given to combine participants for years.

Said Goldman: “I wish [the Eagles] were [a client]. To be honest, with their forward thinking, I think they would really get it. But unfortunately, I have never had the opportunity to talk to anyone from their staff about it.’’

Goldman and his team have moved very deliberately in marketing and promoting the AIQ test. They don’t have a marketing budget. They make no pie-in-the-sky promises about their test data.

“Mark Cuban had this line on Shark Tank once, which is where I get most of my business knowledge from,’’ Goldman said. “He said overnight sensations are 10 to 15 years in the making. That’s what it was for us. We wanted to be ethical practitioners and good scientists.

“You can’t see intelligence. What you can see is output. A guy runs the 40-yard dash and you can look at your stopwatch and say, ‘Gosh, that guy’s fast.’ But you can’t really see intelligence the same way.

“We just felt it was really, really important to make sure we were accurately measuring what we were claiming to measure.’’

What the AIQ attempts to measure is an athlete’s intelligence, particularly as it pertains to on-field performance, both the speed with which he potentially might be able to make a meaningful contribution, and also what the analytics website profootballfocus.com calls “career approximate value,’’ which puts a number on how much a player can impact a team’s won-loss record over the course of his career.

“We’ve always tried to let our science speak for itself,’’ said Bowman, a certified school psychologist who has worked extensively with children and adolescents for more than a decade.

“We’ve also tried to be very cautious. There are a lot of people out there who will say this is the information that you have to have.

“We don’t do that. We’re trained as psychologists. What we say is this is just one additional piece of information. Just like the stopwatch is going to tell you how fast somebody is going to run the 40, you’re not necessarily going to draft somebody based on only that.

“I think teams have responded positively to that because they respect the fact that we aren’t trying to sell them a bill of goods here. We’re just saying, ‘Look, we can give you information about how this athlete thinks and learns and processes information. And we think that will be valuable to you, and would you like to see what that will look like?’’’

Goldman, Bowman, and Auerbach first started testing players at the combine in 2012. They signed their first NFL client a year later. That number has grown each year since.

Goldman initially expected to see a correlation between high AIQ scores and draft status. But that didn’t turn out to be the case.

“I was like, shoot, maybe we missed the mark,’’ he said. “Or maybe what we’re measuring isn’t important.

“But five years later, we looked back at the data. What we found was that the people with the higher AIQ scores, they got on the field sooner. They started earlier in their careers. And their careers were longer.

“The average career of an NFL player is just 2.7 years. But five years later, the guys who had the higher AIQ scores were still in the league vs. the lower ones who weren’t.’’

The AIQ’s credibility got a big shot in the arm last year from Cleveland Browns quarterback Baker Mayfield.

Goldman declined to give the test results of any specific player -- “We’re very mindful of relationships,’’ he said. But two league sources who work for teams that are AIM clients, confirmed a Sports Illustrated report from last year that Mayfield had the second highest AIQ score ever by a quarterback, and one of the top 100 scores among the 4,000-plus athletes that have been tested in the last eight years.

Mayfield, who was selected by the Browns with the first pick in the 2018 draft, started 13 games last year and broke the NFL rookie record for touchdown passes in a season with 27.

“If you think of sports as an unsolvable puzzle, which is how I tend to view it, it’s like, well, what kind of cognitive abilities do you need to solve it?’’ Goldman said.

During the 15 years they were developing the test, Goldman and Bowman looked at other “unsolvable puzzle’’ environments, including first-responders, special forces, and fighter pilots. In fact, the test is used by a branch of the U.S. Army Special Operations Forces.

“If you’re a first-responder and have to open a door and have no idea what’s on the other side of the door, what are some of the cognitive abilities that would really be helpful?’’ Goldman said.

The test, which is taken on a tablet and takes about 35 minutes to complete, has been described as part puzzle and part video game. It includes 10 subtests in four categories: visual spatial processing, reaction time, decision-making, and learning efficiency.

Visual spatial processing includes tasks that require athletes to mentally organize visual information efficiently and effectively. Reaction time measures an athlete’s speed in response to stimuli and assesses his or her ability to make snap judgments, detect differences, or compare information.

Decision-making measures the speed and accuracy with which the athlete is able to make decisions. Learning efficiency measures the ability to store information into long-term memory and then retrieve that information later.

“I see (the AIQ) test as (being) one piece of the (evaluation) puzzle,’’ said Auerbach, a clinical and sports psychologist for the University of Arizona athletic department, and a former assistant coach at the University of Rhode Island. “But my perspective is that perhaps it captures a piece of the puzzle that is going to be slightly more connected to sports performance than would a written test or something like that.

“The big difference is the tasks that we’re asking people to complete, I think, are going to be more representative of the mental tasks an athlete will be asked to complete as they engage in their sport.’’

The Wonderlic test, which was designed to assess the aptitude of prospective employees in a range of occupations, still is given to prospects at the combine. But many people inside the league and out question the value of a written test with respect to evaluating football players.

“It’s very unlikely that an athlete is going to be asked to engage in any fill-in-the-blank numbers-sequencing task that you might see on the Wonderlic, like ‘3, 6, blank, 12,’ ’’ Auerbach said. “That’s just really not a skill that’s going to be called upon in a sports performance.

“Whereas, with the AIQ, athletes are asked to rotate a shape in their mind and match it up with shapes they see on the screen. That sort of ability to manipulate an object in space and understand how turning an object around will or won’t fit with what you’re looking at is perhaps more of a dynamic skill set.’’

Said Goldman: “The thing about written tests is they’re dependent on things like language and verbal behavior. While language is a component in most sports, it’s not essential in live time. Once the ball is snapped, you’re not really talking to yourself or one another. You have to really visually process the field.

“The other thing is, a lot of these language tests can be culturally biased. That’s one of the things Major League Baseball loves about us is they can go in and test kids in Latin American countries, even though some of those kids haven’t received anything beyond a third- or fourth-grade education, and still get meaningful data to help them with their decision-making.’’

Goldman and Bowman wanted to develop a test that wouldn’t be influenced by formal instruction and would be more applicable in dynamic fields such as sports in assessing things like reaction time and decision-making speed.

“Reaction time is a critical component in football,’’ Bowman said. “For defensive linemen and linebackers, there is a significant relationship between reaction time and sacks.

“You can’t measure reaction time with a paper and pencil. [With the AIQ] you look at the screen, an image appears on it, and as soon as you see it, you have to push a button as quickly as you can. It measures average reaction time in milliseconds, as well as how accurate they are.’’

After testing players at the 2012 and 2013 combines, Goldman and his team were invited to training camp by one club in July 2013. The team’s offensive and defensive coordinators took the AIQ.

“One of the coordinators struggled, and I’m thinking, ‘He’s not going to want this,’ ’’ Goldman said. “But he said, ‘You know what, this is why I do this or why I do that.’ He was able to see why he naturally compensated.

“For example, one of our tests is about looking for minute knowledge in a crowded field. He started laughing. He said, ‘This is why my wife reorganized my closet for me. I used to not be able to find anything.’ ’’

That team, which has been an AIM client for seven years and has won a Super Bowl during that period, had the AIM team test their entire offensive line that first year.

“It helped them with questions like, can this guard and tackle zone-block or are they better just blocking man to man,’’ Goldman said. “Based on their cognitive abilities, how do we work them together?

“At the end of the season, that team came back to us and said that discussion was really integral and valuable in the success of their season.’’

AIM’s list of clients has grown each year as more and more teams see the value of the AIQ test as an evaluation tool.

“The teams that take the time to talk to us and learn what the test can do are, at the very least, intrigued, and understand how they can use it,’’ Auerbach said.

AIM doesn’t list the names of the teams that contract with them on their website, but the GM for one AIM client said the AIQ test has been very helpful.

“I’m not going to sit here and tell you it’s going to help us find the next Tom Brady,’’ the GM said. “But it’s another piece of information, another piece of the puzzle. The more information you have on someone, the better you’re able to make a well-informed decision.

“It’s been a remarkably good indicator of how quickly a guy is going to be able to mentally make the transition to our game. If a guy hasn’t scored as high, we also have been able to use the information with respect to coaching strategy and how best to present things to him.’’