By Rees Morrison / Altman Weil, Inc.
Many law departments use a request for proposal (RFP) to choose outside counsel to represent them in a major matter, for a portfolio of matters likely to start over a period of time or on a panel. Typical RFPs describe the bidding protocol, expectations of the department and background facts for the firms. These facts generally include work that will likely need to be done, the annual number of similar matters in the past, the distribution of law firms used, fees paid, outcomes achieved, etc. RFPs are a comfortable, familiar tool, often involving the procurement function, but they are quite often heavy on text.
They would be improved if law departments incorporated more data in them and prepared better analyses or visualizations of that data. Based on my own experience as a consultant guiding RFPs for law departments as well as submissions from law firms, I see three phases where data analytics can improve processes.
1. Presenting the Law Department’s Data
Law firms will ask fewer questions prior to submission and craft better responses if the RFP presents data that they need in a format they readily understand. Tables can be adequate, but RFPs should rely on graphs to convey data that law firms can absorb and apply. For example, assuming that the law department is looking for a panel or a firm to handle projected matters over the next year or two, the RFP might:
- Summarize statistics of how many of those kinds of matters have been handled for each of the past two or three years. A table compiles such data acceptably, such as the number of matters opened and closed in 2014, in 2015 and in 2016, but a scatterplot can tell more about each matter and the matters as a group. The reason is that scatterplots can take advantage of shapes, colors and sizes of points to convey aspects of the matters.
- Include histograms that depict information with vertical bars whose height tells the frequency of the number of matters by some measure. For example, each bar could stand for a $50,000 range of fees so that the histogram displays how many matters were handled in each fee range.
- Create “stacked” bars that break out how many matters each incumbent firm has handled (without naming the firms). The widths of the bars and text annotations can provide information very efficiently.
- Describe how much it has paid out per year, not in actual dollars but rather in indexed dollars. So, if the starting year was 2013, the index would be set at 100 for that year’s fees. The years 2014, 2015 and 2016 would be shown above or below 100 by some percentage to indicate the change in amount spent from the 2014 index to those years. Alternatively, each bar could show the percentage change from the prior year. This gives firms some sense of variability in fees but does not frame their offers by telling them the dollar amount of the fees.
- Create contingency tables that show for each internal client how many in-house counsel handled the recent matters that are being put out to bid for future matters.
- Incorporate a pie chart of the types of matters that are included in what is being put out to bid. The pie slices might be 10-Ks, 8-Ks and other SEC filings, for example. Or, a pie chart by the geographic location of matters.
Each of these analyses of data and visualizations lets law firms understand better what they are bidding on. The better the firms’ grasp of what they might be asked to handle by the law department, the fewer assumptions they’ll make. And law firm assumptions are always risk-averse and conservative (read: costly). Eliminating them helps reduce fixed-fee bids.
But there is even more value to be gained by law departments that gather and present data in RFPs. Having collected data for the RFP, they can do things with it, such as predicting future values or classifying matters. They can either hold those revelations for their own use or share them with law firms.
2. Collecting the Law Firms’ Data
A savvy law department should prepare its request for proposal so that law firms provide at least some data in a format that the department can efficiently collate and analyze.
- For example, the RFP might require data to be returned in Word tables or a spreadsheet where the data is the number of matters the law firm has handled in the past three years that are similar to the matters being proposed on. Spreadsheets can build in drop-down menus so that responses are consistent.
- In addition to providing resumes of core team members, law firms could be asked to provide data on their team’s years of legal experience, years with the law firm and the number of matters they have worked on. Each of those components of staff talent lets a law department gauge relative experience better and do so in part analytically.
- As a third illustration, it will help a law department analyze bids if information on representative billing rates comes from the firms in a tabular format. The department might ask for partner, senior associate and junior associate rates. It’s easier to plot and calculate using a consistent data frame.
3. Evaluating the Responses of Firms
Many law departments fall back on very loose methods to select firms from those that submit proposals. They allow scattered personal impressions to unduly influence discussions, they cede decision-making to the ranking lawyer, they don’t assess the relative importance of different categories of answers, they fall prey to many of the cognitive imperfections that behavioral economists have publicized – and they definitely make little use of data analytics.
Once the law firms have submitted their offers, data analytics can help the law department choose the best proposals. The qualitative assessments of the proposals count for more, to be sure, but metrics can supplement those judgments by corroborating them, challenging them or enriching them.
It should not violate anyone’s sensibilities that those who review the proposals of firms convert their assessment into a score. Typically, the scores would be on the basis of one being a poor proposal on up to seven being an excellent proposal, or for components that are being reviewed, such as “depth of experience.” This scale is known as a “Likert scale,” and ample software tools can decipher and visualize the results.
No one would argue that a law department should choose outside counsel for a panel or for a portfolio of matters solely or even principally on the basis of metrics. Numbers cannot convey the complexity of subjective judgments that combine experience and quality of thinking about representation. But often the preconceived impressions that lawyers accumulate about law firms are dated, unrepresentative or based on one or two encounters. Numbers can mitigate those impressions and the other decision-making weaknesses outlined above.
To the degree that numbers gathered from the law firms are analyzed and the qualitative evaluations by the law department’s lawyers are stated as evaluation scores, a law department can go a long way to sort out stronger from weaker proposals. Numbers that derive from thoughtful RFP questions, if they are combined and presented with rankings or scaled values, contribute greatly to a more balanced assessment, and one that can be explained to others in the company and the disappointed bidders.
But analysis can go farther. Another technique would be to assign different weights to different elements of the valuation. Geographic location of the firm’s offices may deserve a lower weighting in the final analysis than experience, for example. Other techniques include scaling data, ranking and creating categories (e.g., “top quartile” or “bottom third”). Interviews of firms can provide other sets of data points to add to the mix.
Given that RFPs sent to law firms and other service providers play such a large role in legal department procurement, it behooves general counsel to strengthen that tool. And effective use of data and data visualization also adds potency to law firm proposals. So paying more attention can be a win-win for law departments and their want-to-be outside counsel.
Rees Morrison, a partner at Altman Weil, Inc., consults to law firms and law departments with a focus on data analytics. His new book on how graphics can help legal managers make decisions will be available soon.