Document management and discovery: Tips, traps and best practices (Webinar)

Jul 18, 2018

Document management and discovery is one of the most time-consuming and costly aspects of the litigation process. That’s why it’s critical for in-house legal teams to understand the intricacies of every step of document discovery so they can adopt the most efficient and cost-effective approach to complying with their production obligations.

In the third webinar of the Best practices for the legal department of tomorrow series developed by Osler’s Litigation and Dispute Resolution team, Sarah Millar, a partner in the firm’s Litigation Group and head of Osler Works – Discovery, and Alexandre Fallon, a litigation associate in the firm’s Montréal office, offer practical guidance on the best ways to handle the costs and burdens associated with document management. Sarah and Alex also provide useful takeaways that in-house counsel can rely on in their day-to-day practice.

---

Video transcript

EVAN BARZ: Good afternoon, and welcome to Osler's webinar series Best Practices for the Legal Department of Tomorrow. I'm Evan Barz. I'm an associate in Osler's litigation department in Toronto. This is the third in a series of webinars that Osler is presenting on how your in-house legal team can position itself for success in this ever-changing legal landscape.

In today's webinar, we will be discussing everything in-house counsel needs to know to be prepared for the documents management and discovery process of any litigation claim. I'm joined today by Sarah Millar, head of Osler's discovery management group, who works with our clients and litigators to design customized discovery solutions for our clients, and Alex Fallon, a member of Osler's litigation department, with considerable experience managing large-scale documentary review and production in complex commercial cases. If you have any questions during the webinar, please email us or type them into the area provided on your screen, and we will respond to them as time permits. With that, I'm going to turn it over to Sarah and Alex.

SARAH MILLAR: Thanks, Evan. Thank you for being here today, or listening, I suppose. I'm Sarah Millar, and I'm a partner at Osler, and I practice exclusively in the area of document discovery, data discovery, and evidence management. And I've been doing that for about eight years.

And as people have heard me say before, it's not the most illustrious career, but it is definitely an important piece of the litigation process, as the documents piece is often 50% of the litigation spend. And Alex, who I will let introduce himself in a couple minutes, will talk to you a little bit about how we can minimize the cost and burden of discovery. That's the goal of the presentation today. And so I'll turn it over to Alex to introduce himself.

ALEX FALLON: Thank you very much, Sarah. So hello, everyone, and thanks for joining us. My name is Alex Fallon. I work in our litigation department in our Montreal office and have had, on several occasions, the benefit of seeing Sarah's team at work and the efficiencies that having a specialized document management team within the law firm has benefited our clients. And so I am very happy to share some of what we've learned through this practice with you today.

SARAH MILLAR: OK, perfect. Thanks, Alex. So slide two. We're going to talk a little bit about the e-discovery process. And for those of you who are aware of this, I apologize if this is repetitive, but I thought it might be good to anchor our discussion of sort of pitfalls and tips or tips and traps within the context of this model because the model really is a description of the e-discovery process, your standard sort of project, e-discovery project. And we call it sort of the bible of e-discovery.

But anyways, on the left-hand side, you have information governance. And that's before litigation or a regulatory review comes knocking on your door and you need to produce your documents. So information governance, it really deals with how an organization manages, stores, and disposes of its data prior to litigation.

Then you move on. Litigation has now dropped, or the regulators come knocking, and you have to produce documents to the identification phase. And with the identification phase, this is where you work to identify sort of sources of relevant documents, documents that the other side expects you to produce, that the regulators asked you about.

Moving on from identification, once you've identified your documents, you have a legal obligation to preserve them. And we're going to get into document preservation a little bit. Alex is going to talk about his experience with legal holds.

Once you've preserved documents, then you typically collect them. I mean, it's a fairly simple process, but you actually take the documents from your records, your IT architecture, and you give it to your lawyer or a vendor. After that, you process the documents or data. So what processing is is it's taking the horde or mass of data that you've collected, if you can imagine hundreds of thousands of emails, Excel files, PDFs, what have you, and processing is a process whereby you convert it into something that's manageable, reviewable, taggable, searchable, and most importantly, from my perspective, something that you can apply analytics to so that you can do some data analytics to really get into what's in the data and take a look to find the relevant stuff.

And then at this phase two, there's a legal review. That's where lawyers go through the documents, try to figure out what's important and what's not, what's privileged. When that's done and you have a set of documents you deem to be both relevant to either the litigation or the regulator's requests, you produce those documents out.

And then subsequent to that is what we call the presentation phase where you're actually using the evidence at trial or normal discovery at arbitrations, mediations, what have you. The idea is as you move left to right, the volume of documents that you're handling reduces. So obviously, you're going to identify more potential sources of relevant data, but what you're actually going to collect and review and handle is hopefully going to be reduced as the process goes through using technology. And the relevance is going to increase.

And you know, Alex and I have joked about this before, the old adage that you collect a million records, but really, 100 matter for trial. That's really true in document discovery. And I think that's why there's so much been-- been such emphasis on cost containment in this particular area as well because the costs of dealing with a million documents can be utterly explosive, but really what matters at the end is so little. So your counsel's job is to make sure you get from the beginning of the model to the end of the model with the least amount of burden and the least amount of cost possible. And I hope that we're going to come up with some practical tips today for that.

OK, information governance. And I've said this before, but I don't-- as lawyers, neither Alex nor I are really confident to give you much advice as an organization that's going to be very unique, so I'm not going to give you a bunch of tips about how you should manage the information within your organization. But I will say that should think about it.

I would say that one of the biggest issues, I think, in litigation preparedness and the costs of e-discovery is that most organizations have not turned their mind to the volume of data they have, where their data is, whether it's been disposed of, who has it. And information and governance is important for so many things other than just litigation preparedness. Like, frankly, litigation preparedness is probably the last on the list of 100 things that matter when it comes to your information.

Obviously, understanding where your information is and being able to use it is important for just basically running your business and business intelligence, but it's also important for things like data security and privacy. There's so many, like, ripped from the headlines cases where we're seeing data breaches, and frankly, the more data you maintain and the fact that you don't know where it is, really it lends itself to that type of situation where you're having data breaches and what have you. So I would encourage all of my clients to be interested in information governance, how can you create, organize, store, dispose information in a way that promotes the behaviors you want within your organization.

Now, specifically, information governance for litigation preparedness, I'd focus on two things in this, data mapping and data retention policies and processes. Data mapping is really just a fancy-schmancy way of saying you, as in-house counsel, it would be a wise idea in my mind to simply sit down with your IT group and ask them questions about how their IT, the IT architecture works, and how data is created, stored, deleted in your organization. Very simple things like how does our email work? Do we keep email forever? Do we delete email after a certain time frame?

Is deletion of email up to users? Do users use other forms of chats, like chat functions, like internal office Skype, what have you? Do they use texting on their iPhones? That type of thing.

What happens to employee data when the employee leaves? You know. Is there a computer wipe? Do we save a copy of their email? Who has it? Where do we save loose electronic files?

Do we use SharePoint? Do we use Dropbox? Do we just save it on the network?

All stuff like that is just very basic questions to understand where data is kept, who keeps it, and how it's housed and whether it's disposed of. Those are things that we, at Osler, we keep a list of questions that we would recommend you asking your IT group. We'd be happy to furnish that to any client. It's really just sitting down with the IT group for 45 minutes, maybe once a year, just to find out where this stuff-- what's happening with this type of thing, and I think it's really, really important.

It's really great when we start a litigation project with a client, and in-house counsel knows where this stuff is. It actually really focuses the conversation, and it eliminates often several hours of meetings with IT or in-house counsel or what have you in order to sort of map this out while we're in real time trying to deal with litigation. I see Alex. You seem like you have something to say.

ALEX FALLON: Yeah. No, it just makes me-- it made me think of a case I had not too long ago where there had not been this discussion about what kind of data are we keeping, what kind of tools do we have in-house that are generating data, and it was a wrongful dismissal case. And we had a really great case. And we had done the--

SARAH MILLAR: Or so you thought.

ALEX FALLON: Or so we thought. We had done the document review email. Everything was fantastic. All our ducks were in the row. And then somebody lifted their hand and said, what about the chat function? And it became apparent that not everyone within the organization were aware of the fact that there was this chat function--

SARAH MILLAR: I'm not laughing at the poor client. [INAUDIBLE].

ALEX FALLON: Yes. And so it turns out that there was a lot of relevant data on the chat thread that many people were not aware of, and that was not so good, and we ended up having to settle that case on unfavorable terms.

SARAH MILLAR: So you went through all this constant expense, and really, the crux of the case, you could have gotten to it if you'd gotten to the chat function first.

ALEX FALLON: Exactly. Exactly.

SARAH MILLAR: And I think, and you and I have talked about this before, like, I oftentimes will-- the client says, well, I've got some Bloomberg chats. Again, like in an insider trading case, nobody thinks to ask about the Bloomberg chats. Everyone's going through the email. So it's important just to know this because it really can change the face of a case once new evidence pops up.

ALEX FALLON: Yeah. And it raises the question, if you have multiple tools like that for communication, are you an organization that needs a chat function? And if the answer to that question is no, well then why is it accessible within your organization? Maybe it's a time to think about shutting it down because it is a source of risk.

SARAH MILLAR: And especially since people tend, as you and I talked about this, people tend to speak informally on chat. They don't use worth language, per se. They think of it as maybe sending attacks personally are offline. And but a lot of times, it's captured within an organization's records. And I think this segues really nicely into the next topic that we're going to talk-- actually, can you back up, kiddo, so-- the next topic.

Data retention policies and processes. So Alex made a great point. Does your organization need that chat function?

Some people might say yes, absolutely. It's how our guys work. You know, we're, like, a tech shop, and we're inventing apps all the time, and that's how our guys speak. Email's for old folks.

But I think it's very difficult for us as lawyers to say to you what types of data you need to keep or not keep. That's really something that an organization needs to make a decision about. I will say, unequivocally though, that companies that have well-defined data retention processes and procedures, e-discovery is generally cheaper and less burdensome because there's just simply less data there. So I would recommend really sitting down as an organization and deciding whether or not you need to have the vast amounts of information, whether you need to keep it, whether it be on a file server, whether it be in a chat function, whether it be email or not. And again, that's just something that an organization is going to need to decide, not a lawyer.

But definitely, my clients who keep less email, who keep less loose electronic files, it's just cheaper for them to engage in discovery. Now, that being said, you know, you have people who say, well, what if we destroyed exculpatory emails from 10 years ago that showed that what we were doing was fine? And you know, that's a risk, absolutely. The content of what you're keeping or destroying is hard for us to comment on it in a vacuum. But if you're looking at straight up burden and costs, I would say that absolutely, data retention policies are something to look into.

I would caution people to avoid, like, written ones, like, on a paper that they save on the internet that say save email for seven years, save this for-- like, nobody reads those. No one abides by them. I would encourage retention policies that are less user-driven or that your employees control and more that the IT group would control.

So limits on how much email you can maintain in your Outlook account and maybe directions on where it should be saved, or maybe it gets deleted after a two-year process, two-year time frame if someone hasn't saved it purposefully. You know, there's always that you can control the explosion of data in your organization that take it out of the hands of users and put it in the hands of IT, where it's recorded, where everyone knows what's going on. OK. I think we can switch now, Alex. Thanks.

ALEX FALLON: All right. So the theme of the presentation, really, why are we here? We're talking about saving costs. Obviously, there's no way that we can make the cost of document discovery go away, but there are things that we can do to make sure that it doesn't cost more than it should.

And really, it really boils down, at the end of the day, to making sure that you have a plan and that you have thought about how you are going to undertake this process. And that can go a long way in saving costs. And making sure you adopt a focused approach.

At the end of the day, when a piece of litigation comes through the door, it's about certain facts, and it's engaging at all levels of the organization to make sure that you understand who has handles on those facts, and those are the people who should be the subject of the collection and not other people. And the identity of those people can change over time. You may identify 20 people, but the time frames for each of those persons may be completely different.

And so we really need to, at the front end, think about making a plan and identifying to reduce-- in the aim of reducing the scope of the collection at the end of the day. And the last thing is the importance of really thinking about this up front because we see this all the time. Things that are not anticipated can have an outsized effect. Very small things can have an outsized effect on the ultimate cost of the production. I think Sarah is well-placed to give us some insights on the types of mistakes that can lead to costs coming out of control.

SARAH MILLAR: Absolutely. I mean, I think Alex's point, if you collect information from three more custodians than you need to, maybe you're collecting it from one custodian who's a data hoarder and keeps everything. You know what I mean? And maybe this guy's marginal. But all of a sudden, you're contending with 200 gigs of data that's costing you $20,000 to process or what have you.

So I think Alex is right. It's like this is the type of area where it's little technical things that can lead to the spiral of costs, and I think that's why it really is important to have counsel who understands the cost and locations of various decisions you might make that seem relatively harmless or light-hearted, but are actually quite significant or expensive. And I think Alex is right. The best clients are the clients who care about this process and who are engaged up front.

I recognize that clients have competing interests, and in-house counsel absolutely have competing interests, and-- I'm sorry, competing things that grab at their attention and that this might not be the priority for everybody. But if you really are concerned about the costs, in-house's counsel attention to this is really what will drive costs down because it will be working together to formulate a plan, and they'll be the liaison between the fact people at the client who know what happened and the time frames that matter and what have you, and that can be distilled down to us. Too often, I see counsel who's not really interested in this process, and as a result, we end up kind of feeling our way in the dark and guessing, and that always leads to increased costs. So yes, Alex is absolutely on the mark with that.

ALEX FALLON: Yeah. So really, the easiest thing to do, of course, is to sort of say take it out of my hands. Here are the custodians. Here's a list of people. I'm giving you all their data for the time frame.

That's absolutely the wrong approach in the sense that it demonstrates that you haven't really gone through what we had found on this slide as the key steps for preservation identification. So you really have to make sure that you've focused the sources of the data and that you've done a good exercise to make sure that-- identifying custodians is not enough. A custodian may have involvement for a very small period of time. You need to make sure that you're focusing your collection on that period of time only and not just on a custodian basis.

Then you need, of course, to think about the databases. Where do those custodians have documents that may be relevant? And the one thing that is often neglected is talk to the custodians. Don't take a sort of top-down approach to this.

Engage your business people. Have the discussions. Explain to them what the case is about, and get their input on where they think that relevant documentation may lie. And that will greatly assist in focusing the collection effort.

And of course, you want to make sure-- there's two sides to the story. You want to be focused and have a plan so that it costs less and you spend less time doing it, but you also want to make sure that you're focused and having a plan so that you don't miss things because of course, you can do one approach and have a whole host of custodians that may be of limited relevance, in which case it's going to cost you a lot, and it's going to take a lot of time, and it's going to distract the organization. You don't want that. But at the same time, you don't want to just identify a custodian, don't have the discussion with the business and say, you know, I think that this is the person. There shouldn't be any others.

If you don't have the discussion with that particular custodian who is able to identify others, well then you're missing data. And that, from a legal perspective-- from a cost perspective, that's great because you're not processing as much stuff, but from a legal perspective and your ultimate liability, that's a significant issue because it can lead to a spoliation claim. So you want to balance those two competing interests. But really, having a plan and thinking about the collection is a way of mitigating both risks on both sides. So at the end of the day, get the organization engaged, and make sure that-- don't adopt the process of just handing it to your outside counsel and saying, take care of it.

SARAH MILLAR: Yeah, that's a bad idea.

ALEX FALLON: One thing that comes up quite a bit, clients are always curious about litigation holds. So I want to take a few moments to go through that. And the one thing that the bad practice, if I will, that you hear about, is a client will approach you once you've been retained, the litigation is already-- the proceeding has already been served, and for the first time, we're having a discussion at that point about whether or not it would be a good idea to implement a litigation hold. And at that point, it's already too late.

And so really you need to think as in-house counsel about what kind of processes you have in place to make sure that litigation holds are going out at the time where they should be. And what the case law tells us is that that is as soon as litigation is contemplated or threatened. That is when the obligation to take good faith step to preserve relevant data arises.

And the best way to demonstrate that you've made those efforts is by having adequate and efficient litigation hold processes and litigation hold processes that actually result in the issuance of litigation holds to relevant custodians at that time. So that requires you to think about this collection point that we're talking about, starting the identification of relevant custodians at the time where litigation is contemplated or threatened, making sure that IT is involved in the litigation hold process so that whatever back end processes that they are running with respect to your file destruction policies, that they are aware of the impact of litigation hold being sent to a custodian, i.e. that those processes have to be suspended with respect to that custodian at that time when those are issued. So it's crucial that they be involved and that they understand the effect of this mechanism that is litigation hold.

SARAH MILLAR: Agree with you, Alex. Too many times I see, like, really poorly-- well, it's getting better, but poorly drafted litigation rules that are all legalize that, like, don't clearly explain what's relevant, that use highfalutin legal language to explain very basic concepts to people, both make sure you keep this and here's what you need to keep. That does a disservice to the people receiving a hold.

They're not going to understand it. They're not going to enjoy reading it. In fact, they're going to be bored reading it. So I agree with you. Clearly drafted holds are key to send around.

ALEX FALLON: Yeah. So on that point, it's plain language. You're issuing a set of instructions to the people within your business about-- give them the facts in the litigation. Here's what's happening. Here's why this is being issued. Here is the allegations that are being made.

But don't talk about the legal basis, right? Talk about what the facts are being alleged and why they're getting that hold that they have that involvement, and then give them instructions, right? Tell them exactly what you're looking for, what they need to do, and make sure that IT, like I said, is involved in making sure that that's implemented as well on whatever back office systems are being run.

And once you've identified certain custodians, and it goes-- you know, it's still part of that overall collection process. Talk to those people. So you send a litigation hold. Go and engage with those persons to make sure that the litigation hold is, first of all, understood, and that they understand their obligations with respect to implementing it. But then have a discussion with them about the case, and make sure that you have identified all of the relevant custodians.

And so, you know, the person will tell you, yes, I'm fully aware, but you know, I was only involved in this for a short period during three months. I took over from so-and-so. You may not know that. And after the three months, I handed the brief over to so-and-so. And that's information that you didn't have, and there you go.

You already, through this process, identified other relevant custodians. And of course, well, that results in you issuing further litigation holds. You have to-- don't sit on the information. Act on it. And even if you have an initial litigation hold, it's important that those be refreshed.

People forget this is obviously-- it's not something that stays on the mind of people for more than a few days, so you really do need to remind people that they are the subject of a litigation hold with respect to certain facts. And so those need to be sent out. And the case law says it's a benchmark at least every six months. And make sure that you also think about new people within the organization.

I do want to touch briefly on the situation in Quebec, given that I'm coming from our Montreal office, to reassure you in some respects that discovery is actually not as bad in Quebec as in other jurisdictions. And really, the idea there is that there's a conceptual difference in Quebec versus common law jurisdictions with respect to discovery. The onus is not on a party to produce all relevant documentation.

For example, through an affidavit of documents, the onus is actually on the opposing party to ask questions and to make requests for specific productions. And so we're, in Quebec, responding to specific productions, and until the request is made, there's arguably not much of an obligation of general preservation of things that could potentially be relevant to a particular litigation. So to counter that, because that obviously can give rise to risks on both sides, it is a very good practice to send out litigation hold notices to the other party.

SARAH MILLAR: Is that becoming more common, do you find, Alex, in the Quebec bar?

ALEX FALLON: Yeah, absolutely. I think it's not something that we see universally. I think it's definitely a sign, though, of a good external counsel. I'm always pleased when I see the other side doing that in the sense that we have a serious opponent on the other side. It's something that if your counsel is not doing it currently, make sure that that practice changes because really, you are setting yourself up for a much better argument before the court if ultimately say, we don't have anything that's responsive to this request.

If you know that that's not the case, but you didn't make the request in a timely manner, well, you're not in as good a position if you had sent a specific hold notice at the very outset of the litigation to inform here is the type of document that we consider relevant. Be advised that we're putting you on notice that you should preserve. And if you don't preserve and don't ultimately produce when the requests are made through the discovery process, which comes later, well, that you will make arguments, for example, based on abuse of process. And that can be a significant tactical advantage in Quebec. So think about that, and insist that your counsel get on board with that strategy, if it's not already the case.

SARAH MILLAR: Thanks, Alex. OK. So we talked about identification. We talked about preservation. We're moving on now to collection and processing.

So collection, and as I said, this is sort of where the rubber hits the road and the money starts flowing. This is where costs can start to mount if not done sort of carefully or with consideration. So I get a lot of questions from clients about OK, so who collects the data? What now?

We know where it is, and we know what matters. What do we do now? And I would say that for the vast majority of cases we deal with at a firm like Osler where it's civil litigation, it's quasi criminal and kind of relatively regular regulatory reviews, it's absolutely fine for your IT department to affect collection, assuming you have a reasonable amount of confidence in them, and they're able to sort of follow instruction and log what they're doing. It's absolutely fine.

I don't think that clients need to spend the money to hire collection vendors to frenziedly image laptops and make copies of servers and what have you. It's fine to use your own resources internally in order to affect collection. And so as long as you guys are preserving the integrity of the data, which we have directions to help, as long as you're able to facilitate the sort of the safe transition of the material to Osler or another vendor, it's absolutely fine for you guys to affect collection, and I would actually highly recommend it.

OK. A word about doing stuff yourself or retaining vendors outside of the relationship with your counsel. So I have a lot of clients who want to be more proactive in the e-discovery process. I think that stems from the fact that this has been historically so expensive and probably historically poorly managed by external counsel. And so you have a lot of clients who are sophisticated in this area who want to bring in, create their own in-house group who does e-discovery, bring in technology.

And I think that's-- I think it's great, and I think that that's just something that external counsel needs to learn how to work with and appreciate. I think the problem is, though, that some of this is sometimes fraught with issues. First and foremost, I think it's problematic often when clients attempt to bring in their own e-discovery software because a lot of clients don't invest in keeping that software up, doing the virtually quarterly upgrades you need to do on the software or investing in people who know how to run the software. So what you end up having is old technology that isn't state of the art, that doesn't have the analytics built in, and that's really, really slow.

Another thing I see that drives me a bit-- that I find-- I was going to say drives me crazy, but I find a bit more difficult is when clients already have vendor-- like, have already aligned themselves with a vendor and require us to use that particular vendor. A lot of times, I find that those are vendors that have gotten in front of clients and sold them on the notion that they have much cheaper rates and that they're going to be-- they're overall bill is going to be really, really less expensive than if you go with sort of more-- sort of the bigger names in the industry. And I've often found that that isn't the case. It often tends to be-- the situation often tends to be not that great. I'll just give some examples.

So we'll work with vendors that I'm not aware of. I've never vetted before. I don't know what they do in the background. It's situations where a vendor that is more familiar in the industry will take an evening to do something, where

I've worked with vendors that take a week and a half to do the same thing. I've had a situation where we did legal review in a particularly slow tool that a vendor had that cost $80,000 more because of the slowness of the tool. When you have lawyers going through documents in that going from document to document takes a minute as opposed to six seconds, like, that has significant and material impact on the ability of lawyers to do their job.

So I would be very wary of vendors that come calling that try to sell you on cheaper products or cheaper services. I think that you tend to pay what you get-- that you pay for what you get in this industry. So just be aware of that.

Just a couple of tips, though, if you are going to go-- if you want to bring some of this in-house, and if you want to take the reins of this a little bit more. I would recommend hiring somebody who's a point person, who's your e-discovery expert in-house, and they can vet vendors, and they can consider strategic relationships with vendors and vet technology. And I would recommend having a few vendors on menu so that your in-house counsel-- or so your external counsel, if they're comfortable working with one over the other, if one has a better reputation, that there's a menu of vendors to choose from. And again, if you do want to acquire technology, I would just recommend really vetting that technology. And I would recommend tending to go with what is industry standard rather than something that's less well-known because honestly, most of your counsel are going to know how to use the industry standard technology better than something that's a little more obscure.

ALEX FALLON: So if we move now to the step of the process that is legal review, and that, of course, as you can suspect, is the most expensive part of the process because now we're talking about human reviews that are billed at a certain way that is not the same way that technology is billed, so can get very expensive if it's not planned properly. So really, the first thing that you need to focus on is you need to engage with your external counsel about defining what their plan is. You have to ask questions because there are a growing number of tools that can streamline this process, make it much more efficient, and save you significant amounts of money. But you need to make sure that your outside counsel is aware of these tools, is familiar with these tools, and has people within the organization that are able to engage with these tools. Because if not, then you're losing much of the benefit there.

So ask questions. If the plan turns out, oh, we're going to have 21st-year associates do a line-by-line review, and it'll be comprehensive, and you know, that way, we're making sure that you don't have any risks, that's not the good answer anymore. And if you're getting that answer from external counsel, well, you need to perhaps think about what the impact of that is because you don't need that anymore today. There are technology tools that will allow you to save significant amounts on that.

And one of the things that Sarah can tell us about a little bit more is this question of predictive coding. But before we go there, just there are simple tools already that really reduce the process. For example, you have the emails that-- all these threads and attachments. Those can be organized already by existing tools in a relatively simple fashion. Saves an immense amount of time.

And the other point that I would do is taking out of a collection set documents that we know because of the nature of the business are absolutely irrelevant and don't need to be reviewed. I did a document review a few years ago where this had not been organized as such and spent a lot of time reviewing account flight reservations because it was a company where the executive traveled a lot, and that was a complete waste of time. But you have tools that we can use today to make sure that that doesn't happen. And so I'll pass it over back to Sarah to talk about predictive coding, which is probably the more exciting element in technology.

SARAH MILLAR: If there's a sexy part of it, this is it. So I think that some-- I think some of the most exciting use of artificial intelligence and, like, bona fide AI, Artificial Intelligence, is in the world of discovery. I'm sure it'll change-- you know, I'm sure this will spread to the larger legal industry, but right now, I think discovery is really where most of the technology is happening. And I think that a few years ago, the idea of having a machine determine what was relevant or not in a data set was very foreign to people. And even up to probably a couple years ago, not many people were using it in the industry.

But I would say now that predictive coding, other ideas of TAR where basically humans teach a machine through examples of relevant and not relevant documents, how to find similar documents in a larger data set, that has been fully embraced by the industry in Canada. That's fully embraced by most of the bar, I would say, that practices in this area. And I think we can make very good arguments as to why any court or any regulator should accept the results as well. So I think this is really exciting.

We certainly use TAR on almost every big project that we've had in the last year-and-a-half to two years to really great-- like, I want to say really great results. Better results than I was expecting because of course, I was a bit suspicious of this being a human and being done doing a lot of legal review in my more junior years. But just to give a very brief example, we do a lot of construction litigation here, and this is cases in which you have millions and millions of documents. And we were able to review-- we had a million documents, and we were able to get it down, through predictive coding process, to about 250,000 to 300,000 documents.

The cost to do that, the predictive coding piece was $80,000. But the return on investment was actually quite astronomical because if you consider even $1 a document for the million, we would have had to review almost a million documents to the tune of almost $1 million. Instead, their review cost us probably around just over $300,000. And when we actually went into the data after he had done the predictive coding of the stuff that the computer had deemed relevant, most of it was actually pretty darn relevant as opposed to using search terms in the past. We're seeing, like, relevance rates of 50% and even more, whereas historically, the search terms, you know, you'd see 10%, 15%, 20% relevance if you were doing your search terms well.

So I just think this is a really great tool. I think it's going to be used on every single large-scale project going forward. Some projects it's not suited for. We were talking about how if you only have 80,000 documents and you're looking at about $1.25 a document for review, are you really going to spend $80,000 worth of TAR to get it? You're going to look at the documents probably or some portion there.

I was just saying what I think what's really going to happen is probably we're going to evolve to a situation where clients are going to get comfortable with machines making determinations about relevance with the help of humans. Obviously, humans are going to guide that determination. But machines are really going to be the ones sifting through the data for relevance. And then perhaps once we have a relevant set, humans we'll do more of a privilege review or do a review for the stuff that we're particularly concerned about that might be higher risk in terms of the litigation.

But that ultimately, there's going to be a bunch of stuff going out the door that is client data that no one's had eyes on. And I think that if you're really serious about reducing litigation costs, that's going to be the way to go. And I suspect within three or four years that most files will be handled this way.

That's my prediction. Because that's really how costs are going to be-- like, get a machine to do it for you. You don't have to pay them $85 an hour to do it.

ALEX FALLON: And it goes back to what we said earlier about having the discussion with your team when you're going through the collection process. You know, people know where significant documents and risky documents are. So if you have those-- you get into a bit of a more difficult situation where you have people who've left the organization. But if you're in a situation where you're able to rely on your people to guide you through it, well, that can be very useful in leveraging technology and making these calls. At the end of the day, though, human review will always remain in some fashion.

So the one takeaway, though, really is to engage with your outside counsel. Make sure they have a staffing plan that makes sense. Document review is often something that's sort of relegated to the back burner, you know, from strategic litigation perspective, but it is really important, and it needs to be taken seriously, and it needs to be taken seriously by your outside counsel.

And they need to devote resources to it in a manner that makes sense, you know, a specific team be formed that it's not sort of something that's done when people have time and you have a rotating cast of people who go through your documents. You want to make sure that you have a focused team who is doing it efficiently, that you have sort of a line of command to make more touchy determinations, and that's sort of clearly set out. And of course, it will vary depending on the scope of the litigation, but you want to make sure fundamentally that your outside counsel does have a plan.

SARAH MILLAR: Exactly. Exactly. Ask them what they plan to do. Ask them how they plan to use technology. OK. Production beyond. We'll go through this relatively quickly.

There's a couple key takeaways from here. So the middle point is about a callback agreement. I would say in virtually all cases, if you can get the other side to agree that if you produce privileged documents inadvertently, you can ask for them back, and there's not going to be any issues, that is key because I think that increasingly with the volume of data we are processing, reviewing, and pushing out the door, especially if we stop looking eyes on every document, we need to be able to pull documents back that were inadvertently produced that are privileged. Alex, I think you agreed that this was, like, a key-- you know, this is a very important thing.

ALEX FALLON: And we see it more and more, and the reality is, I think it's going to become something that's quite standard because people want to have production quickly. Especially the larger the production, the quicker they want it because they know that they're going to have to spend time processing it on the other side. And so my experience has shown me that opposing counsel are very reasonable. The bar is very reasonable.

SARAH MILLAR: Show me where there's parity in the discovery obligations. Like, it's in both side's interest to just come together and agree on this. And I even find regulators are getting better-- people who have less risk of producing per the documents are even getting better about acknowledging the risk that the other side's going through with this massive data, and I think that in almost every case I'm on, we have a callback agreement. So that would be my recommendation with big data cases. Try to get a callback.

The other two points I wanted to make here are sort of interrelated. One would be, too often, people don't engage the other side meaningfully on limiting production and how to make production inexpensively. There's a rule that we need a discovery plan in Ontario. It's not often followed, and a lot of people hate that rule, and I think there's actually a movement afoot to abolish it. I think that's sad, but regardless of whether you have to, I just think you should engage the other side.

You'd be surprised at how many concessions you can get and you might be willing to give in order to make discovery much more efficient and less painful. And I find that sometimes if you just maybe let the underlings or the tech people, like, people like me just go and maybe negotiate with the underling on the other side who's dealing with this discovery, you'd be surprised that if the big guns aren't negotiating, you can often-- like, because I have less of a horse in the race, that we can often come to an agreement about things, and there's often more emphasis on conciliatory behavior, I think, in the e-discovery bar in Ontario, particularly, where we're interested in making trade-offs so that we can get the job done. And I also find that, increasingly, regulators are better about this. I would say that we should put the pressure on regulators to understand the demands that we have when we we're asked for everything under the sun in a two-week period.

Regulators do not want a data dump of 200,000 documents. They just want the juice to the extent exists. So if you can work with them to narrow their production request, to narrow their demands, it actually works out really well, and I think we've seen that successfully happen on a few cases we've been involved in where we know what we're talking about, and we give them options, and we give them examples of what the issues are with what they're asking. And they usually are, I find, overall pretty good about agreeing to restrict their demands or ask for different things. So I would say never be shy to engage the other side on discovery, the format, and the substance of it, how you can reduce that. For sure, for sure, for sure.

ALEX FALLON: Yeah. And I just want to chime in on the point you made, Sarah, about where the connection point has to be on discussions about these issues, and I think that that's absolutely right that you have to have people on two sides that are actually involved in it and who will be affected by the decisions that are made, right? So you want to have the people who actually have to review the documents having a larger production or a more circumscribed production. If the people who actually have to do it are involved in the discussion, you will find you get a more reasonable outcome than if you encourage your counsel to engage at a higher level by the people who don't really have a stake in-- they're not going be spending the time going through the documents.

SARAH MILLAR: A senior associate-- or sorry, the senior counsel, like, senior partners, whatever, should probably not-- I mean, they're awesome for other things. They probably shouldn't be the ones leading the charge on discovery.

ALEX FALLON: Exactly.

SARAH MILLAR: And most of them don't want to, right? So it is-- I mean, I find that people don't know a lot of the language to use, and so you might agree to-- like, I've had situations where people agree to produce all TIF documents, which means you're TIFFING 500,000 documents, which you've already increased your hosting cost by about $5,000 a month. Wouldn't you agree to that?

ALEX FALLON: And you had no idea.

SARAH MILLAR: And so a lot of time, we had to go back and be like, can we please step back from what this person agreed to? And a lot of times, the other side's like, yes. We don't want to do it either.

But I do find that it's a little bit of a fun practice to sort of maybe push this down to people who-- like I said, you can kind of couch it. Like, oh, we're just talking tech. Then you can actually get a lot of substantive concessions, especially from regulators.

Oh, we're just the tech people. But do we really have to do this? Do we really have to go into this part of the computer?

Aw, no. That doesn't matter. And then you've wiped out, like, 50,000 records to look at. It's really great if you can speak with people who are the decision-makers who know what matters and doesn't matter.

ALEX FALLON: Yeah. Absolutely. Couldn't agree more.

SARAH MILLAR: OK. So we'll go through this relatively quickly because we've been talking at you a lot, and a lot of this, we've dealt with. So dos and don'ts. OK. As we said, do you understand where your data is, who has it, how long you keep it? That's just basic-- we would highly recommend that. That's a starting point and easy to do.

Just call up your IT guys and set up a meeting. Take them out for lunch. They'll explain to you where stuff is.

Consider whether you need all the data or all the programs that house data in your organization. Again, that's hard for us to speak to as your lawyers. You might absolutely need that chat function Alex spoke of, but in the end, it might not matter. And consider whether or not you want to implement policies and procedures on the back end that manage data, that take it out of the hands of the users and put it into the hands of an automated process that runs every single time and that's documented and that you know, from that, where data exists or doesn't exist.

Have a plan for your legal holds in advance. So Alex talked about how it's, like, kind of crummy when six weeks into litigation, you discovered no one sent out a hold. I would say even before you are hit with litigation, understand how you would disseminate a hold, how you would re-disseminate it every six months. How would you get acknowledgments back from people that they've read it? And how, in fact, on the back end you might even implement hold processes in place that hold data even without users being able to have any control over that.

Build up your in-house capacity. Absolutely do it. I think the best way to do this is have a person specialized in e-discovery rather than buying a bunch of fancy tools or aligning with a particular vendor. But there's different ways of doing it, and just caring enough about e-discovery is a good thing to start.

Ask for deals from your technology vendor. So rather than agreeing and locking into, like, a four-year deal with a subpar technology vendor, have a few technology vendors that you've developed relationships with, and then go to the well every time you have a new file with them and ask them for deals. I would say ask them for deals, especially on data hosting. There's a huge markup on almost all technology fees by technology vendors. This is where they really make their money.

And I think that they'd be willing to give you deals because they could still make a good amount of profit even if they are reducing what they're charging you. Less so on legal review. Technology is really where you can squeeze your vendors a little bit.

Ask counsel specifically what their plans are for legal review. And as Alex said, I would ask them specifically, what are your plans for using modern day analytics? What are you going to do? Are you going to chain emails? Are you going to suppress emails?

Are you going to use TAR? How much is it going to cost me? How much do I expect to not pay for legal review as a result of using this technology? What would be my return on investment?

And I would say try to get the other side engaged in production and reducing the scope of production, especially regulatory authorities. I'm finding that's increasingly happening more, and it's a good thing. And callback agreements. Always, always try to get a callback agreement in place, if you can.

And I think the don'ts are a little bit of the other side of the coin of the dos, but we'll just grab a couple of them. And like I said, don't be afraid to use TAR, predictive coding analytics. It's the wave of the future. Everyone's doing it, as they like to say, and it's definitely going to be what is on all big files in the next couple years.

Don't assume you'll save money by locking into an agreement with a cheaper vendor. I really firmly believe that most of the time you don't. There's other ways in which the cost savings get eaten up, mostly because oftentimes, their service isn't as good, or your lawyers don't know how to use the technology as efficiently.

And again, don't not engage with the other side when it comes to discovery. You'd be surprised at concessions you can earn and that are helpful to both you and them. And maybe you will walk away both happy over the process, maybe not. But you know, you gotta try, I would say.

So yeah. So those are the dos and don'ts. Alex, I don't know. You're in the-- as I said, you're in the trenches more so than I am.

And I'm lucky. I just get to manage the data. So do you have any thoughts based on your practice? Any takeaways? Any closing--

ALEX FALLON: The only thing I would say is obviously, nobody enjoys this process. And we all wish it didn't have to happen at all, but it does.

SARAH MILLAR: It's like going to the dentist.

ALEX FALLON: Yes. And so the one thing from a practical perspective, just don't close your eyes and hope that it happens. It will. If you send it to your outside counsel and say take care of it, we'll take care of it, but you're not going to like the result from a cost perspective and the impact that that had on your organization.

So be involved. Ask questions. Get your business people engaged, and you know, together, we can achieve significant cost savings that make it much less painful than otherwise it would be.

SARAH MILLAR: I agree. Thank you, Alex.

EVAN BARZ: All right. Thank you very much, Sarah and Alex. We've already had a couple of questions pour in to you both while this has been going on, so I'll put those to you now.

The first one relates to the involvement of company's IT department and their participation in the document collection process. Question relates to Osler's practices, and does Osler's discovery management group work with the company's IT department to identify relevant documentation, to collect those documents? And if that process is one in which Osler's involved, what does that process actually look like on the ground?

SARAH MILLAR: OK. So as I said, my preference, unless it's, like, some crazy intense, like, criminal file that we needed to get a forensic investigator in to do a forensic copy of a particular device or something, my preference is for clients' IT groups to do the collection. They're more than capable. It's a cost savings on the front part of the client. We would provide you with what we call a data inventory of all the sources of documentation and data that need to be collected. They would be Filepass and a list of custodians for email to pull.

But really, the IT group goes away and does it, and that's perfect. I mean, that has worked out perfectly in the past. We don't like to send Osler people in to touch our clients' IT systems. I don't think our clients want that either for security reasons.

So we're more than happy to work with the IT group to pull the data. We will provide the listing of what needs to be pulled in a way that's intelligible to the IT group, and we will provide them with a suggested mechanism which transfer the data to us as well as some steps on how to preserve the integrity of the data. But really, they're more than capable of doing it. It's very simple. We do it fairly regularly, and we have our IT guys that talk to your IT guys, and that's really how it works.

EVAN BARZ: That's great. So they both speak the language.

SARAH MILLAR: Yes.

EVAN BARZ: It makes it easy, yeah?

SARAH MILLAR: Yes. Yes.

EVAN BARZ: Fabulous. Well, the second question relates to technology. And you both mentioned that technology can be very helpful, but it's important to select the right technology to fit with in-house counsel. So for in-house counsel specifically, are there specific products that you would recommend that they use in terms of technology, that they implement for their own discovery management needs?

SARAH MILLAR: So I think a lot of clients-- I mean, most clients don't implement full-scale e-discovery resources. It's astronomically expensive to buy, like, a Relativity or the latest tool in the market. It's hundreds of thousands of dollars, and you have to have people who know how to run it and who service it and servers to put it on and stuff like that. Only our clients who are the most sophisticated in this particular area have stuff like that.

I would say what might be good is looking into legal hold technology, which could be an add-on to your email system or your servers. You might want to look into technology about disseminating a legal hold and getting acknowledgments back. Other than that, I mean, I don't think there's very much technology that in-house counsel and/or an organization needs to answer their e-discovery requirements. There's a lot of technology out there that once the data is collected, we put in so that we can perform analytics and do the review.

But stuff in-house, you really don't need all those fancy bells and whistles because most of the time, you just need to know where to pull your data from. And most of the time, an email dissemination of the hold and an email back acknowledgment is absolutely fine. But some clients like to have the extra hold technology so they know that stuff is being preserved. That's what I would recommend people look into is maybe legal hold technology, but beyond that, I wouldn't buy a lot of fancy stuff.

And anything that you have that you can search your own records, some clients want to search their own records before giving it to us, and they want to just give us the search results. The problem is, a lot of their searching isn't what would be considered 100% valid because it doesn't search all pieces of the file. It might not search PDFs that haven't been made readable. It might not search metadata.

So you know, if clients want to use that, we just explained the risks, and if they accept it, then we're happy to use it. But really, once you cross the threshold into searching in analytics, it's mostly external tools that do a better job, like the Relativitys or the Ringtails of the world. And that's very expensive to acquire.

EVAN BARZ: Fair enough.

SARAH MILLAR: Yeah.

EVAN BARZ: All right. Well, thank you very much, Sarah.

SARAH MILLAR: Oh, it's our pleasure.

EVAN BARZ: On behalf of both Sarah and Alex, thank you very much, everyone, for joining us for this webinar. We hope it has given you some useful insights. Make sure you join us for the remainder of the webinars in this series. More details will be posted on osler.com in the near future. Thank you again for attending this webinar, and have a lovely day.

ALEX FALLON: Thank you, everyone.

---