Views
No views yet
fastino/gliner2-large-v1 trained to extract datasets, data mentions, and their relations from academic papers, research, and reports (with a focus on World Bank/UNHCR documents).fastino/gliner2-large-v1ai4data/datause-extraction-v2ai4data Python library wrapper. The library automatically handles:1git clone <repository-url>
2cd monitoring_of_datause
3uv sync1from ai4data import extract_from_text, extract_from_document
2
3text = """Our analysis uses the 2022 Demographic and Health Survey (DHS) conducted by
4the National Statistics Office. We complement this with administrative systems, but
5only the DHS is used in the empirical models."""
6
7# Extract from raw text
8results = extract_from_text(text)
9print(results["datasets"])
10
11# Extract from a PDF document
12pdf_results = extract_from_document("report.pdf", pages=[0, 1, 2])
13print(pdf_results)ai4data, each extracted entity in the "datasets" list has the following structure:1{
2 "mention_name": {
3 "text": "Demographic and Health Survey",
4 "confidence": 0.9999,
5 "start": 23,
6 "end": 52
7 },
8 "specificity_tag": {
9 "text": "named",
10 "confidence": 0.9999,
11 "start": 23,
12 "end": 52
13 },
14 "usage_context": {
15 "text": "primary",
16 "confidence": 0.9999,
17 "start": 23,
18 "end": 52
19 },
20 "typology_tag": {
21 "text": "survey",
22 "confidence": 0.9999,
23 "start": 23,
24 "end": 52
25 },
26 "acronym": {
27 "text": "DHS",
28 "confidence": 0.9996,
29 "start": 54,
30 "end": 57
31 },
32 "producer": {
33 "text": "National Statistics Office",
34 "confidence": 0.9999,
35 "start": 72,
36 "end": 98
37 },
38 "reference_year": {
39 "text": "2022",
40 "confidence": 0.9999,
41 "start": 18,
42 "end": 22
43 },
44 "is_used": {
45 "text": "True",
46 "confidence": 0.9999,
47 "start": 23,
48 "end": 52
49 },
50 "geography": {
51 "text": "",
52 "confidence": 0.9999,
53 "start": 23,
54 "end": 52
55 }
56}named: A specific, citable dataset (e.g., "DHS 2020", "World Development Indicators", "Ghana Living Standards Survey (GLSS)")descriptive: A general category of data, not a specific named dataset (e.g., "household survey data", "administrative records", "panel data on firms")vague: An indirect or ambiguous reference (e.g., "available data", "our dataset", "the data used in this study")primary: Core data driving the main analysis in the report.supporting: Secondary data used to validate, calibrate, or provide robustness checks.background: Mentioned in passing, in a literature review, or as historical context.