Old ways of accessing knowledge will coexist with Generative AI

Even though it's been around a year since generative AI became a real point of interest to law firms, people are still incredibly excited about this technology. However, as we get into 2024, we need to move away from being excited about the technology itself and start moving towards applying it in practical circumstances for clients.

This often involves identifying a use case. One of the things I've observed in the market is that people mean different things when they talk about use cases.

At one end of the spectrum, you have people saying things such as, “a good use case for AI is contract drafting”. This kind of statement assumes that a use case is essentially the same thing as a process. The problem is that AI isn't going to replace processes such as contract drafting, although it might help you redesign those processes. AI is a tool in the toolbox, and it will always be something that affects a particular point or task in a process. Conflating a use case with a process renders the analysis too high-level.

On the other end of the spectrum, you have people talking about key capabilities of AI, such as “summarisation”. The problem here is that you are discussing tasks within a process without identifying the process that the task is part of. Lawyers do not, for example, summarise for the sake of summarisation; they always summarise things for a particular purpose. Losing sight of the wider purpose can cause problems when you end up building an application lawyers have to interact with.

This kind of thinking is commonplace when people are thinking about where AI might influence knowledge management. For example, some have suggested that knowledge management is no longer needed because we have AI.

Knowledge management

This way of thinking often skips the exercise of mapping the strengths and weaknesses of AI with the goals of knowledge management. For example, it is well documented that generative AI capabilities are not good at factual recall. They also produce inconsistent responses. These two things are baked into the design of large language models; without these things, large language models would not work as well as they do.

Yet consistency and accuracy are key goals of any knowledge management activity. For example, a firm may produce a series of approved templates lawyers can use for a particular contract. These templates might contain drafting notes that are used to manage risk for the firm. Alternatively, a knowledge management system might contain a series of survival guides to help lawyers understand how to do a particular process. The key goal of these things is to make sure lawyers are giving legal advice accurately and consistently.

AI is never, on its own, going to be the answer. Instead, we need to look at whether AI provides new routes to access knowledge, and what the impact of these might be on how knowledge is currently accessed.

The Q&A interface

AI presents some intriguing opportunities for knowledge management. The concept of a Q&A interface, as an alternative to the conventional approach of in-depth document review and note-taking, has captured the imagination of tech experts. This interface promises a more direct and interactive way to access legal knowledge, but it is not without its limitations.

Considering the purpose of legal research and what clients seek from research notes or comprehensive legal work is key. While simple questions may only require brief answers, complex legal issues often demand a deep dive into research, a process that sometimes holds more value than the resulting document itself.

Clients tend to value lawyers who have a thorough understanding of their situation of the law and who are available to address queries as they arise, instead of relying solely on a static research memo. It is through the examination of source materials that a profound comprehension of a legal area is usually achieved.

The primary role for a Q&A interface in a knowledge management system should be to make it easier to get up to speed on unfamiliar legal areas. Instead of starting from scratch reading numerous cases, a summarised overview provided by a Q&A system could highlight potential issues to be aware of when delving into the details. This utility is where the true strength of a Q&A interface lies, yet it doesn't negate the necessity for engaging with the original legal texts.

Fuzzy searching

AI's capability to enhance the discovery of knowledge materials is another area ripe for discussion. Lawyers who struggle to find specific documents might find 'fuzzy searching' particularly advantageous. This type of searching allows for the entry of a search term and returns relevant results, even if those results do not directly include the entered term.

Fuzzy searching is most effective when lawyers have only vague information about the document they are seeking. However, this is not always the case. There are times when a lawyer knows precisely which document they need. In such scenarios, fuzzy searching may be less helpful, as it tends to generate a broader array of results compared to a keyword search. When lawyers desire a specific, narrow set of results based on a particular keyword they have input, such as a document number or the title of a template document they intend to use, keyword searching remains the superior method.

In essence, fuzzy searching offers a complementary method for lawyers to locate information—it doesn't replace the precision of keyword searches, which are still preferable in certain situations.

Labelling and categorising

One of the significant challenges in developing a robust knowledge base within a law firm is the efficient organisation of information, often restricted by limited resources for labelling and categorising. AI can be instrumental here, potentially automating the population of metadata, which in turn assists users in navigating or performing detailed searches to find what they need.

While the capabilities of AI in this domain are somewhat untested, what we do know is that AI can only label based on the data that has been provided to it. Often, the most crucial context needed for knowledge management isn't directly in the document itself. For instance, knowing the industry context of a document's use or whether it was drafted under urgent conditions could significantly influence its future utility.

This new ability to label and categorise supplements traditional methods of knowledge access, like browsing and searching. It introduces capabilities previously unavailable, enhancing the overall efficiency and effectiveness of knowledge management systems.

The interplay of AI and quality content in knowledge management

The commonality among these use cases for AI in knowledge management is their reliance on the existence of high-quality knowledge content. We are still some distance from an era where AI can independently generate valuable knowledge content without a foundation of carefully curated material.

What AI offers in the interim are novel methods of accessing knowledge that complement rather than supplant the established modes. The introduction of a new method does not necessarily mean it will replace all preceding ones. The value of AI in this context lies in its ability to enrich and diversify the knowledge management landscape, not to render obsolete the systems and processes that have served well up to this point.

Jack Shepherd, Principal Business Consultant at iManage