Running Legal Like A Business - Ch. 11 - Legal Department Metrics - What to Measure and Why It Matters?

Chapter 11 of Running Legal Like A Business by Connie Brenton and Susan Lambreth, PLI Press, 2021, returns to the theme of metrics.

In chapter 7,  author Peter Elihaur offered a data analytics 101.  In chapter 11, authors Mick Sheehy, PwC, and Pratik Patel of Elevate lay down the gauntlet as follows, "Every day, legal departments make decisions....While there is greater maturity across the legal market in using data for financial and external resource management decisions, many other decisions are made based on judgement, experience, intuition or qualitative  information - essentially a gut feel.  This is ironic considering that law is generally an evidence-based sport."

They then focus on some useful metric type definitions and the why, what and how of legal metrics. Their hypothesis is that legal departments have been under less pressure than other internal service departments to rely on metrics due to relatively small size, tendency to "punch above their weight", and a genuine belief that the value lawyers contribute is difficult to quantify or the effort involved is not justified by the benefits.  In counterweight, Sheehy and Patel point out that (consistent with my experience), legal departments tend to have a uniquely wholistic view of the entire enterprise that makes thinking through the metrics well worth the while.

Sheehy and Patel note that a metric provides context for a data point that supports decision making, whereas a measurement is simply a data point at a single moment in time.  For example, the number of contracts executed in a year may be of passing interest (lagging indicator), but one needs a contextual benchmark, such as how contract volume trended month by month over the year, or how the number compares to the number of contracts executed last year, or how the volume corresponds to revenue, to yield a useful action item or prediction for the future (leading indicator). 

The authors start from the point of why, then back out from there to what and how:

Why?  The authors point out that investing time in measurement should only be done where the effort supports a key decision (such as resourcing, budgets, risk mitigation) or material narrative (story of which stakeholders need to be informed).  Measurement just because the data is available is a waste of time unless the information is put to useful purpose. 

What?  Sheehy and Patel refer back to a PwC Australia report with a 21-page catalog of metrics, "Legal department metrics: Understanding and Expanding Your Impact" (June 2020) that provides excellent food for thought. They state that benchmark data is not widely distributed nor frequently published. While I agree that a number of good datasets have limited availability and the coverage of metrics could be broader, there are nonetheless good resources available from HBR, CLOC, Citi Advisory, Blickstein and ACC, among others. As always, informal networks and conversations can also help in this regard.

How? Sheehy and Patel categorize data by how difficult it is to gather -

• Easy: Data readily available in an existing system dashboard or reporting tool, for example.
• Medium: "Data that is not readily available but can be obtained through a relatively straight-forward gathering exercise." In this category it is good to strategize carefully on a forward going plan for  maintenance, as there can be a tendency for legal ops to be asked to take on the task of repeatedly gathering and reporting on the data in the same manual way, rather than arriving at a more streamlined or automated process. 
• Hard: Data that can only be obtained from mapping, analysis, including clean-up and normalization, and problem-solving exercises. One of the most intriguing comments the authors make, to my mind, is that there are now tools to help create structured outputs from unstructured data (words in documents). I imagine the authors may be referring to contracts analytics tools, but would welcome further discussion on that topic and particularly any applications beyond the more common contracts analytics metrics. Our team has been in a position more than once of trying to extract data from our document management system based on creation dates, edits dates, authors and editors, matter folders and key words; it is highly manual to enrich the output and hard to apply rules with consistency.

The authors also produce useful definitions in categorizing or assessing data: Structured vs. unstructured, lagging (output) vs. leading (predictive), and determinative (aid decision-making directly) v. indicative (suggest areas for deep dive investigation). 

Read together, chapters 7 and 11 provide a disciplined framework for approaching metrics.

Note: Ops in a Box, Legal Edition includes an executive dashboard that I have used at two companies for the GC to provide a snapshot of performance in meetings with the CEO and also for reference during annual shareholders meetings. The executive dashboard reviews overall spend for the past 3 years by practice area, as well as spend, category analysis, and trend lines related to intellectual property and litigation.