ArttuPakarinen/

FIN-ai-chat-summarization.Q4_K_M.gguf

Full fine-tune on LumiOpen's viking 7B, 4bit kvantization - for testing.

Around 3-4 million tokens full fine tune to make the model produce summarization.

Around 2-6 hours of compute on A100 GPU.

Partial dataset provided on the model card - other datasets used were Diagsum and Samsum, which were autotranslated to finnish.

Model works fine for it's purpose, even in this low-parameter and low kvantization mode.

Model has been trained to start "first prompt" with "Ihminen aloittaa keskustelun AI-botin kanssa\n\n". I call this 'First summanry'. Without adding the first summanry, some weird behaviour may occur.

The idea is to summarize the chat context, and to be able to continue chatting, by using the produced summarization in the begigning of the next prompt.

In theory, the model can be used to produce AI responses, but the model will always produce

###TIIVISTELMÄ###

<summarization>

This means, if you want to produce AI answer, it is completely possible by ending the prompt with "\nAI-botti: ", but you need to parse the summarization part away from the response.