You have to hand it to Firoz Dattu. The former Paul Weiss lawyer and founder of AdvanceLaw sure knows how to whip up a crowd. Earlier this year, with no small amount of media fanfare, Dattu unleashed something he calls the GC Thought Leaders Experiment. Haven’t heard of Firoz, or AdvanceLaw, or the Thought Leaders Experiment? No worries. Dattu trots out a nifty marital metaphor to describe the project.

“[H]oneymoons don’t last forever,” he says. “But is that fade inevitable? Whether we’re talking lawyers or married couples, some relationships manage to stay strong over the long run. What’s the secret? The GC Thought Leaders Experiment tackles that question.”

And how, pray tell, will AdvanceLaw, whose core business is vetting outside counsel for GCs, turn that trick? In a word: data.

Dattu telegraphed the answer in a 2013 Harvard Business Review article, “Why Law Firm Pedigree May Be a Thing of the Past.” Again revealing a flair for showmanship, he and co-author Dina Wang, a former McKinsey consultant who advises companies on leadership, opened with a saucy quote riffing on the old saw that you can never get fired for hiring IBM – or, in the corporate legal world, Cravath.

“I would absolutely fire anyone on my team who hired Cravath,” the article quotes an unnamed GC as saying.

“While tongue in cheek and surely subject to exceptions,” Dattu and Wang explain, “it reflects the reality that there is a growing body of legal work that simply won’t be sent to the most pedigreed law firms, most typically because general counsel are laser focused on value, namely quality and efficiency.”

Dattu calls it the “Yelpification of law” in another Harvard Business Review article, “Consulting on the Cusp of Disruption,” this one co-authored by Wang and the Master of Disruption himself, Clayton M. Christensen, whose seminal work, “The Innovator’s Dilemma,” was named by The Economist as one of the six most important business books ever written.

Cut to early 2017. The GC Thought Leaders Experiment launched with a group of law departments agreeing to share data on their matters and their assessments of outside counsel. Twenty-five GCs from companies such as Nike, Panasonic, Mastercard and Molson Coors raised the curtain with an open letter explaining the initiative.

“This is a real-time experiment testing which in-house practices (e.g., convergence, value billing, competitive bids) and law firm attributes (e.g., firm size and structure, legal project management) tend to produce the strongest relationships, satisfaction, and results,” the GCs wrote. “Through a large data set, across our companies, we are moving beyond the anecdotal to measure what really works.”

It is an ambitious project that Dattu is careful to explain is not about billing or cost analyses. Eight months into the 18-month experiment, on September 17, he hosted a webinar to show just what the project is about. Four in-house leaders joined him: Bill Deckelman, GC of $26 billion DXC Technology (formed of a combination of DSC and a division of HP); Wanji Walcott, GC of $11 billion PayPal; Scott Offer, GC of $24 billion Flex (formerly Flextronics); and Lee Reichert, secretary and deputy GC of $15 billion Molson Coors, the largest U.S. brewer. Together they laid out preliminary findings rating performance across five areas: cost/efficiency, work quality, responsiveness, solutions focus, and outcome vs. expectations. Here are the quick takeaways, which range from mildly surprising to painfully obvious. (For a more complete summary of the webinar, see the piece by Ron Friedman on the Prism Legal website.)

Are law firm panels achieving their full potential? Not really. In the aggregate, panels are disappointing, with the quality of their performance turning on how they are constructed and managed. Hardly a surprise.

What are the most effective engagement tools? Client summits are a clear winner here in driving stronger firm performance. They are an effective engagement tool, as are secondments.

What is the value of performance evaluations? Another mild winner. Law departments that evaluate their firms and share the information have greater returns across all metrics. That makes sense, as timely feedback can arrest complacency and drive improvement.

Is there always a trade-off between cost and quality? This one, pitting big-city lawyers against those in smaller locales, seems to suggest the white-shoe lawyers in the largest U.S. cities underperform, but there is so much variability among firms within metro areas – A-team and Z-team lawyers can be found everywhere – as to render the results less than helpful beyond suggesting that “nearshoring” work – shunning Cravath to roll the dice with Billy Bob’s Law Emporium – may not be as risky as it seems.

On the whole, do flat fees hurt quality? They kind of punted on this one. The data suggests that flat fees, which heighten predictability and help control costs, do not undermine quality, but the jury is out as more data is crunched.

In the end, Dattu says, the experiment is rooted in “what the industry has not yet done with data: use it to test which client and law firm behaviors measurably improve satisfaction and lead to the best relationships.” That’s not the usual benchmarking, he emphasizes, noting that a key goal is to move the profession off its obsession with cost to focus on “quality, value, long-term relationships, and successful legal representations.”

The whole thing, which has drawn a lot of attention, has, he says, become a bit of a Rorschach test for the profession. “We’re all looking at the same inkblot,” Dattu says, “but we’re seeing (sometimes entirely) different things.”

Seems like another way of saying that value is in the eye of the beholder.

Stay tuned.