PLI Legal Ops Institute 2024 - People, Process & Technology

by An Trotter October 29, 2024

The 2024 PLI Legal Ops Institute 2024 featured an all-star faculty that Kate Orr and I had the privilege of co-chairing: Adam Becker, Jamal Brown, Erin P. Buckelew, Kunoor Chopra, Aaron Crews, Elizabeth Gutay, Stacy Lettie, Mirra Levitt, Frances Pomposo, and Tommie Tavaras-Ferreira.

In a jam-packed half-day session, we covered 3 topics: technology, process optimization and data analytics. On a Friday afternoon at the end of a busy work week, we were very pleased to be joined by an audience of 700+, albeit mostly online.

While the topics organized the conversation, the cross-cutting theme was "people, process and technology," rendered here as a pointillist painting made up of data points; information we are bringing together on a single canvas as both art and science.

Technology

Stacy Lettie noted, "Technology is 3rd in the phrase 'people, process & technology' for a reason." That reason is that technology has to solve a defined problem and fit into your team's work mode. Technology is there to alleviate pain points in our work process (not to force humans to adapt to its features).

The technology panel emphasized the value of POCs (Proof of Concept testing). On theme for the season, Aaron Crewes talked about the "hand from the grave" as the stakeholder who raises late-in-the-game objections. The panel talked about how you can leverage your biggest naysayer as your "blind spot spotter." Aaron advocated for making that person a lead in the POC by asking them "What would it take to prove that this is the right approach?" then putting them in charge of designing the POC.

With respect measuring ROI, Tommie Tavaras-Ferreira talked about how you can leverage a hypothesis to come up with some potential measurements or formula for ROI. Jamal Brown advocated for factoring in the risk posture of the new technology, as well as labor cost impact and employee experience.

With respect to Generative AI, the panelists, who represent a wide range of company size, noted efforts to generate potential applications within legal and then engaging with internal developer teams to prioritize and build applications. Because the technology is evolving so quickly the audience was cautioned to keep contractual commitments short-term to retain agility to evolve.

Process Improvement

Kicking off the segment, panelists noted that process improvement is the aspect of their job they love most, because it focuses on people and removing the obstacles that makes their jobs harder than they need to be.

Frances Pomposo acknowledged that you have to be thoughtful in your approach to reassure colleagues you are not there to audit efficiency. Always focus on what's in it for them, and make it as low lift as possible on their end. Kate Orr added it is important to lead with empathy and to actively reflect back what you are hearing for validation. Elizabeth Gutay used an ergonomic analogy of putting yourself in the seat of the client to make sure any solution is comfortable for them.

Adam Becker shared how helpful it can be to meet with attorneys new to the company to understand what worked well at their last job and to surface pain points in the new role. For all team members he likes the question, "If you could change one thing in this department, what would it be?" (One of mine is "Why hasn't this problem already been fixed?"). He also shared that phrases indicating a deeper dive is merited include "everyone knows how it works" and "we've always done it this way."

The panel noted meeting 1:1 can encourage candidness and build rapport. On the other hand, team meetings often spark ideas and uncover group dynamics. The panelists shared a number of frameworks they find helpful include "rose, thorn, bud" (things that work well, are pain points and opportunities), process maps, process map inventories, journey maps and value stream maps. Echoing Frances' earlier advice to make it as easy as possible for the client, Adam advocated for maintaining tool impartiality, and to adjust to client preferences. As part of the mapping process, Elizabeth emphasized the importance of identifying who owns the process, the decision maker, and all stakeholders who touch the process.

Additional tips included:

Remember to also focus on taking out work that does not need to be done (or can be done elsewhere).
Make sure not to define the process too narrowly and to think through downstream impacts and inter-dependencies.
Complete a root cause analysis and do not jump to solution prematurely.
Be sure to put metrics in place to measure impact.

Data Analytics

From the technology panel we learned how tools can enable us to more easily collect actionable data to refine our performance and also to assess the tool's effectiveness. From the process optimization panel, we saw that process mapping is a data collection effort and recognizing a sub-optimal state allows us to create target metrics against which to measure progress. This set the context for discussing the "why" for data and metrics. As Kunoor Chopra likes to say, "legal is an evidence-based sport." We should be looking at data to drive decision-making from department operations' resourcing and efficiency goals to effectiveness of substantive legal work, and tying both back to company business goals.

Kunoor kicked us off with ways in which we can categorize data including:

Purpose: Lagging (output) vs. leading (predictive); determinative (directly aids decision making) vs. indicative (suggest areas for deeper investigation)
Difficulty level: Easy (readily available in tool), medium (straight-forward data gathering) to hard (requires mapping and normalization as with AI tools)
Subject area: Operations (financial, productivity, adoption, compliance) vs. Substantive legal work (selection, positioning, risk mitigation and outcomes).

As an easy way to dive in, Kunoor suggested asking the following, “What are you trying to achieve? What are two ways to describe that outcome? and What are two measures to let you know you are achieving that outcome?”

Erin Buckelew then took the ball to explain how the Legal Data Intelligence framework was developed to help legal teams identify their use case and receive in return both process steps and data points to measure. The model helps practitioners focus in on Sensitive, Useful, and Necessary (SUN) data and set aside or purge Redundant, Obsolete, and Trivial (ROT) data. She encouraged participants to put the goal or desired outcome first, then think through what success looks like in the short, medium and long term. Collaborate with you leader to prioritize, evolve and delete metrics. She cautioned to be careful to avoid data for data's sake.

Mirra Levitt spoke to bringing together substantive legal and talent pool data from multiple sources. In Priori experience, at the outset the process was highly manual and involved technical teams, but further down the road, they found that some data already existed in formats that could be re-purposed to avoid heavy lifting. They brought in a GenAI solution tied to an LLM on the back end to help with data structuring and normalization and were able to reduce effort by 50%. She also spoke to the process starting with thinking about the data ecosystem and understanding contributor stakeholder needs, as well as understanding what the client wants to see and how they want to see it.

The panel concurred that it has been game changing to have tools that extract meta data in a consistent way and to combine data in ways we have not done before. In addition to facilitating data gathering, it also enables deeper analysis. Several audience questions related to normalizing data. The panel acknowledged that when you are combining data sets it is hard to match data on an apples-to-apples basis. In addition to leveraging your business partners and new tools, you will need to use old fashioned legwork to look at the citation and make sure the data is pulling from where you want it to pull. Make sure you query and align your field definitions. Evaluate when a fuzzy match is sufficient and when it must be exact. Ongoing data governance is needed as the data sets evolve to make sure the definitions do not drift.

In terms of people resources, you do not always need data scientists, though being an Excel jockey can help. People who work successfully with data love to tell stories about it. One your greatest resources is people who are willing to say, "This was my problem, here are the metrics I gathered around it and here is what I learned." You need people who lead with a love for data and curiosity, rather than a specific skill set. Some may lean into surfacing the insights and others may convey the story to the stakeholders. The latter is about being accessible, not too overwhelming and understanding your audience.

To cultivate stakeholder involvement, keep in mind that people have different tolerances for how much data they can interact with and how it is delivered. As Kunoor noted, most stakeholders prefer simplicity. Start with a few metrics that can help the stakeholder gauge progress health towards their top 3 business goals. Ask them what it is important to them. Erin added that where you activate excitement don't be afraid to make the stakeholder the star of the show and empower them to help others think through how they can collaborate with you. Once you roll out, don’t just check it and set it. Always check back in. Goals and purpose will change. Revisit regularly to make sure the data still fits to purpose.

When working with traditionally siloed data, it can be very helpful to go the system administrator for the data set you need to speak with them about the problem you are trying to solve. You can foster a relationship and talk about how sharing information can help you mutually solve problems and refine your working process with one another. Erin suggested going to your leader and getting some counsel on how you may approach the conversation.

Closing advice included:

Just start. Don't let yourself get overwhelmed by the universe of the possible.
People feel like they need dashboards. Sometimes dashboards can be overwhelming. You can start with a sentence or bullet.
If you don't understand the graph, ask the question. Someone else will have the same question.
Be careful of data for data's sake. Start with why and keep anchoring back to that with your stakeholders.