Based on Gemma-2b-it, the model was developed through customized fine-tuning for summarizing tasks of different paper data. Based on existing natural language processing (NLP) capabilities, it is designed to effectively summarize the paper text. The model can learn a large-scale paper dataset to deliver the core content of the paper concisely and accurately, and can be applied to papers in various disciplines.
Intended uses : Paper summary, information retrieval and document processing, automated reviews
After loading the json file, the dataset was organized in a chat format to extract only the 'original_text' and 'summary_text' required for learning and enter them into the model.
It take 17 hours for training.
train code :
https://colab.research.google.com/drive/1z8ER-AfVcccDXFWsRzuD-m2LxTAjuaTR1messages = [
2 {
3 "role": "user",
4 "content": "다음 글을 요약해주세요:\n\n{}".format(doc)
5 }
6]
7prompt = pipe_finetuned.tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
8
9outputs = pipe_finetuned(
10 prompt,
11 do_sample=True,
12 temperature=0.2,
13 top_k=50,
14 top_p=0.95,
15 add_special_tokens=True
16)
17print(outputs[0]["generated_text"][len(prompt):])
