Success with Gen AI requires law firms to restructure their IA

Law firms in a rush to adopt Gen AI, take note: This new technology is not a ‘silver bullet,’ and it will not deliver magical results if the underlying information architecture (IA) is forgotten about.

For best possible results with Gen AI, law firms should put time and effort into restructuring their IA. Unglamorous as it may seem, these hard yards will pay dividends when it comes to successfully leveraging this latest form of AI, and what comes next.

Good data fuels Gen AI success

What firms are starting to realize when it comes to generative AI is that it relies on having access to good, well-structured information. Gen AI needs to be grounded in quality information sources: clean data, curated collections of know-how and templates, and so on.

The firms that already have those assets are actually in pretty good shape for getting started with Gen AI. The ones that don't, realize very quickly that they need it if they want their AI tools to provide useful and accurate answers.

Speaking of accuracy: Without good data, Gen AI can very quickly lose any sense of context. Take a concept like jurisdiction as an example. The AI might view England, Wales, Scotland, Ireland, and the United States as all ‘basically the same’ – and suddenly, it starts merging laws from different countries together and creating answers that are a blend of different jurisdictions, which is a very dangerous output to have Gen AI spitting out to legal professionals.

AI is not going to self-filter the data to figure out what’s relevant information and what isn’t when it’s creating generative content. That’s precisely why firms need a very strong set of practices around their information architecture: everything from collection of data to careful application of tags and metadata, to proper storage of data in designated locations, and process changes, like what jurisdiction is this information for.

Critical first steps

As a first step to methodically restructuring their IA, law firms should identify where within the organization the trusted data sets are. Where does that data live?

Firms that already have a document management system (DMS) already have a bit of a foundation to work off of here, as having a DMS means they have implemented some key matter-centric factors like practice areas or regions. Additional effort, however, will be required to find ‘the good stuff.’ Identify where the practice teams are stockpiling the shared knowledge/best examples, and discreetly mark these as unique.

Filtering is an essential next step. Providing the large language model (LLM) that underlies Gen AI with access to all the files within the DMS can be overwhelming for the model – it’s simply too much ‘noise’ and not enough ‘signal.’

It is far better to give the LLM a subset of data within the DMS to work with. For instance, the final approved versions of documents are more useful than every draft that led to that final product. Also, it’s good idea to limit content to a certain time range, stretching back a specified number of years; given how much laws and regulations can shift over time, there’s no need to feed Gen AI examples from decades ago unless there’s a very compelling reason to do so.

This is where humans come in. It’s critical to have an internal knowledge curation processes that can determine what ‘good content’ is and what kind of filters to employ when determining which data will be used with Gen AI.

A balancing act around security

Security presents another area to carefully navigate as law firms are getting their information architecture in order.

Depending on how ‘open’ or ‘closed’ of a security model is in place at the organization, some matters and files within the DMS might be locked down and thus inaccessible to certain lawyers. This raises the possibility that generative AI will produce variable responses depending on who’s using the system – with the result that lawyer #1 sat next to lawyer #2 comes up with entirely different results for the same question.

Part of the curation process is about making a decision about which documents can be more open. For instance, the back and forth conversations about how to do handle a deal or matter – including what strategy to take – are very confidential. But some of the end documents – the final product, particularly if it centers on a deal that was then public – could be more open.

Firms might also want to think about adopting a more open security posture for knowledge assets and best practices content to ensure more uniform response. Once again, this reveals that the curation process isn't just about things like metadata – it’s about humans making decisions around that IA to achieve the right balance.

Harness process automation

While humans and technology have a key role to play in shoring up the firm’s information architecture, there also needs to be accompanying change around process.

When it’s tuned appropriately, process can serve as a way of ensuring that documents and data are flowing the right way, at the right time, to the right place, helping to build up the information foundation that Gen AI needs to provide the best results.

This is an area where workflow automations can be tremendously useful. Think of all the valuable data in a typical set of closing books. Process can provide a way for the documents in that closing set to get saved into a specified location in the document management system instead of accidentally not being filed at all (or being filed in the wrong location) because people are stretched in a dozen different directions and have already moved on to the next deal or matter.

A similar type of process automation can help ensure that important attachments on emails don’t disappear into a black hole when they’re filed into the DMS – that the actual documents themselves in that attachment are properly saved and filed so that Gen AI can tap into it.

A hint here: When it comes to process and curation, try to do it in a simple way that humans don't want to resist or try to find a workaround. Make it easy for people. Process only works if there’s a great user experience – otherwise people won’t follow the process, and information will wind up ‘stranded’ or ‘abandoned’ rather than getting where it needs to go.

Laying the groundwork prepares you for future success

It’s helpful to envision information architecture a bit like its real-world infrastructure counterpart. If you’re building a new home, you first need to lay down a concrete foundation, all the water and electrical pipes, and other crucial elements of that infrastructure.

Once you have that in place, though, you can build any type of house you want – and you can make that house as sophisticated as you want.

This is essentially the situation with IA. Once law firms have a solid information architecture in place, they can layer sophisticated tools and technology like Gen AI on top of it. But that foundation has to be there.

While many law firms don’t have that foundation, restructuring their IA can put them on the right path – and even the recognition that Gen AI won’t work properly without proper underlying data is a step in the right direction. With a strong IA foundation, law firms can build their future as sophisticated and innovative as they envision and set themselves up for ongoing success with Gen AI.