Super-EURLEX dataset containing legal documents from multiple languages.
The datasets are build/scrapped from the EURLEX Website [
https://eur-lex.europa.eu/homepage.html]
With one split per language and sector, because the available features (metadata) differs for each
sector. Therefore, each sample contains the content of a full legal document in up to 3 different
formats. Those are raw HTML and cleaned HTML (if the HTML format was available on the EURLEX website
during the scrapping process) and cleaned text.
The cleaned text should be available for each sample and was extracted from HTML or PDF.
'Cleaned' HTML stands here for minor cleaning that was done to preserve to a large extent the necessary
HTML information like table structures while removing unnecessary complexity which was introduced to the
original documents due to actions like writing each sentence into a new object.
Additionally, each sample contains metadata which was scrapped on the fly, this implies the following
2 things. First, not every sector contains the same metadata. Second, most metadata might be
irrelevant for most use cases.
In our minds the most interesting metadata is the celex-id which is used to identify the legal
document at hand, but also contains a lot of information about the document
see [
https://eur-lex.europa.eu/content/tools/eur-lex-celex-infographic-A3.pdf] as well as eurovoc-
concepts, which are labels that define the content of the documents.
Eurovoc-Concepts are, for example, only available for the sectors 1, 2, 3, 4, 5, 6, 9, C, and E.
The Naming of most metadata is kept like it was on the eurlex website, except for converting
it to lower case and replacing whitespaces with '_'.
