Views
No views yet
unsloth with a dataset created using larger LLMs like GPT-4o.You will get as an input: a research paper's content and a set of properties/criteria to extract.
You will extract the values corresponding to the list of provided predicates.
You should stick to the schema and description of the properties (if available).
Use the ORKG Ask structured information extraction XML output format.
The extractions must be in the "{language}" language and the complexity of the language should be for a "{tone}".# Specifications of what to extract:
{properties}
# Extraction source:
{source}
[{'label': 'Methods', 'desc': 'The methods used in the study', 'schema': 'multiple values of type string'}, {'label': 'TL;DR', 'desc': null, 'schema': null}]1<extractions>
2 <extraction property="Methods">
3 <values>
4 <value>Lorem ipsum dolor sit amet, consectetur adipiscing elit.</value>
5 <value>Lorem ipsum dolor sit amet, consectetur adipiscing elit.</value>
6 </values>
7 <sources>
8 <source>Lorem ipsum dolor sit amet, consectetur adipiscing elit.</source>
9 <source>Lorem ipsum dolor sit amet, consectetur adipiscing elit.</source>
10 </sources>
11 </extraction>
12 <extraction property="Conclusions">
13 <values>
14 <value>Lorem ipsum dolor sit amet, consectetur adipiscing elit.</value>
15 <value>Lorem ipsum dolor sit amet, consectetur adipiscing elit.</value>
16 </values>
17 <sources>
18 <source>Lorem ipsum dolor sit amet, consectetur adipiscing elit.</source>
19 <source>Lorem ipsum dolor sit amet, consectetur adipiscing elit.</source>
20 </sources>
21 </extraction>
22 <extraction property="Limitations">
23 <values></values>
24 <sources></sources>
25 </extraction>
26 <extraction property="TL;DR">
27 <values>
28 <value>Lorem ipsum dolor sit amet, consectetur adipiscing elit.</value>
29 </values>
30 <sources></sources>
31 </extraction>
32</extractions>1r=16
2finetune_vision_layers=False, # Turn off for just text!
3finetune_language_layers=True, # Should leave on!
4finetune_attention_modules=True, # Attention good for GRPO
5finetune_mlp_modules=True, # Should leave on always!
6lora_alpha=32
7lora_dropout=0
8seed=421per_device_train_batch_size=1
2gradient_accumulation_steps=32
3warmup_steps=5
4num_train_epochs=3
5learning_rate=2e-4
6bf16=True
7optim="adamw_torch_fused"
8weight_decay=0.01
9lr_scheduler_type="linear"
10seed=42