How could a machine understand Natural Legal Language?

The digital revolution has, for a little more than a decade, generated profound changes  in our society and in our habits. If we have tendency to describe the changes it has introduced in many professions, it is also important to remember that these changes are consequences of the exponential proliferation of the sources and the quantity of data or information that this same revolution has put at our disposal.

And this state of affairs, in spite of its many advantages, has nevertheless
made the search and extraction of reliable information more complex.  However , these  tasks are very important  for any company or person, because they allow to understand and to have at one's disposal data that are essential for effective and efficient decisions.

Particularly in the legal field and as the THEOLEX tool shows, they are the basis for the analysis of of regulatory decisions and negotiated settlements.

They help companies to be informed about the financial and legal risks that they incur,to protect themselves, to call for a new decision; investors to invest or not.
More simply, these are tasks that are the pillars of a guarantee of security and
efficiency.

The considerable amount of research in artificial intelligence (AI) and more exactly in NLP around these more exactly in NLP, justifies their importance. We have thus witnessed the development of many models of information extraction.

One of these models, particularly ingenious and still at the heart of current One of these models is the Question Answering (QA) model. The systems QA systems aim at finding and providing accurate information available in a database from questions asked literally in natural language by a user . As an example: we could directly give him a document summarizing the sentence of a financial crime case and ask the question "What monetary penalty has been imposed? "

Figure 1: Illustration of the operating principle of QA systems
These systems are simple to use, efficient and easily adaptable. They are based today on mechanisms that have the ability to better  understand the meaning of words and the interdependence between them, unlike previous systems that used convolutional or recurrent neural networks. This aspect makes efficient the
mechanisms used in NLP . Mechanisms that have allowed the machines to
the GLUE benchmark to surpass human performance. On the other hand, these
mechanisms require less time to learn how to perform these specific tasks and specific tasks and above all require less labelled data.

This allows companies to free themselves from a major problem. Finally, their increasing accessibility allows to focus less on technical aspects but on business specificities. In many cases of use then, we could note their economic and commercial impact.

During my internship as an engineering assistant at Theolex, I was able to take advantage of these qualities to optimize the startup's information extraction process. More precisely, I worked to improve the extraction accuracy of the decision date, monetary sanction and defendant fields. Fields selected for their importance and the weakness of their extraction precision. If mentally the same model has been used for all these fields, different improvements have been proposed in order to increase the performances of the model itself. And these improvements differ according to the field. The table below summarizes the improvements made

For the defendant field, the lack of responses from the currently deployed model made it difficult to assess improvements. But the proposed version has been compared to the labels in two ways. First, by checking the exact presence and then the partial presence of the correct answer in the answer of the new version. The first method resulted in a success rate of 34.4%. A score which, in spite of everything, shows the prowess of the current model compared to the old method used. A result which is predictable according to the article "DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter".

The second method shows a score of 93% of correct answers. An encouraging result overall, easily exportable to other fields or even other jurisdictions, and which can be improved upon as the methods used are flexible.