webbigdata/

gemma-2-2b-jpn-it-translate-gguf

pip install -U transformers
pip install requests

import transformers
import requests
import json
from transformers import AutoTokenizer

system_prompt = "You are a highly skilled professional Japanese-English and English-Japanese translator. Translate the given text accurately, taking into account the context and specific instructions provided. Steps may include hints enclosed in square brackets [] with the key and value separated by a colon:. Only when the subject is specified in the Japanese sentence, the subject will be added when translating into English. If no additional instructions or context are provided, use your expertise to consider what the most appropriate context is and provide a natural translation that aligns with that context. When translating, strive to faithfully reflect the meaning and tone of the original text, pay attention to cultural nuances and differences in language usage, and ensure that the translation is grammatically correct and easy to read. After completing the translation, review it once more to check for errors or unnatural expressions. For technical terms and proper nouns, either leave them in the original language or use appropriate translations as necessary. Take a deep breath, calm down, and start translating.\n\n"
instruct = "Translate Japanese to English.\nWhen translating, please use the following hints:\n[writing_style: casual]"

initial_messages  = [
    {"role": "user", "content": system_prompt + instruct},
    {"role": "assistant", "content": "OK"}
]

message_list = [
 "and I was a little bit nervous, too, speaking to a Japanese audience really for the first time, certainly since I left the White House. ",
 "And I had a very good interpreter, and if you have ever made a speech in Japan in English, it takes a lot longer to say it in Japanese. ",
 "I decided I would break the ice by telling the shortest joke that I knew.",
 "It was not the best joke I knew, but it was the shortest joke I knew, left over from my governor's campaign years before.",
 "So I told my joke, the interpreter told the joke, and the audience just collapsed in laughter. ",
 "I never got a better response from any audience in my life. ",
 "So I could not wait to get through the speech and talk to the interpreter and ask him,"
 "\"How did you tell my joke?\"",
 "He was very evasive. He would not tell me how he told it.",
 "I insisted, and he finally ducked his head and said,",
 "\"I told the audience, 'President Carter told a funny story. Everybody, laugh.'\""
]

tokenizer = AutoTokenizer.from_pretrained("webbigdata/gemma-2-2b-jpn-it-translate")

if __name__ == "__main__":
    messages = initial_messages.copy()
    for i in range(len(message_list)):
        messages.append({"role": "user",  "content": message_list[i]})
        print("user: " + message_list[i])


        # Transformersのトークナイザーを使いたくない場合は、Colabのサンプルが手でプロンプトテンプレートを書いてるのでそちらを参考してください
        # If you don’t want to use the Transformers tokenizer, you can use the Colab example to manually write the prompt template.  
        prompt = tokenizer.apply_chat_template(
            messages,
            add_generation_prompt=True,
            tokenize=False
        )

        payload = {
            "prompt": prompt,
            "n_predict": 1200
        }
    
        # Define the URL and headers for the POST request
        url = "http://localhost:8080/completion"
        headers = {
            "Content-Type": "application/json"
        }

        # Send the POST request and capture the response
        response = requests.post(url, headers=headers, data=json.dumps(payload))
        # print(response)
        # print( response.json() )


        # Check if the request was successful
        if response.status_code != 200:
            print(f"Error: {response.text}")

        # Parse the response JSON
        response_data = response.json()

        # Extract the 'content' field from the response
        response_content = response_data.get('content', '').strip()

        print("assistant: " + response_content)
        messages.append({"role": "assistant",  "content": response_content})
        
        # Max 6 message, you need more memory for more massages.
        if len(messages) > 8:  # 2 (initial) + 6 (new) = 8
            messages = initial_messages + messages[-6:]

user: and I was a little bit nervous, too, speaking to a Japanese audience really for the first time, certainly since I left the White House.
assistant: そして、私もちょっと緊張していました。白宮を離れてから、日本人の前で初めて、本当の意味で話しているというのは 、かなり初でした。
user: And I had a very good interpreter, and if you have ever made a speech in Japan in English, it takes a lot longer to say it in Japanese.
assistant: そして、私はとても優れた通訳がいて、もしあなたが英語で日本で講演をしたりしたことがあるのなら、日本語で話すの にはるかに長くなります。
user: I decided I would break the ice by telling the shortest joke that I knew.
assistant: そこで、最も短いジョークを話して、最初のうちは気を緩ませようと決めたのです。
user: It was not the best joke I knew, but it was the shortest joke I knew, left over from my governor's campaign years before.
assistant: 一番のジョークではないですが、私の知っていた短いジョーク、それまで governors の選挙でのキャンペーン中にいたものです。
user: So I told my joke, the interpreter told the joke, and the audience just collapsed in laughter.
assistant: それで、私が私のジョークを言ったら、通訳が私のジョークを、そしてオーディエンスはただただ笑い転げました。
user: I never got a better response from any audience in my life.
assistant: 私はこのジョークは、今までに、これ以上いい反応をもらったことがありませんでした。
user: So I could not wait to get through the speech and talk to the interpreter and ask him,"How did you tell my joke?"
assistant: それで、私は、その演説を終わらせて、通訳と話すのが待ちきれないで、その通訳に尋ねたいと思っていました、「どの ように私のジョークを伝えるの？」
user: He was very evasive. He would not tell me how he told it.
assistant: 彼はとても避けていました。どうしたのかと教えてくれません。
user: I insisted, and he finally ducked his head and said,
assistant: 断るまいと、彼はついに首をかしげて、「
user: "I told the audience, 'President Carter told a funny story. Everybody, laugh.'"
assistant: 「司会者から、カーター大統領が面白い話をした。皆さん、笑って。」

@misc{dahara2024imatrix,
  author       = {dahara1@webbigdata},
  title        = {gemma-2-2b-jpn-it-translate: A translation task-specific gguf model based on gemma-2-2b-jpn-it},
  year         = {2024},
  howpublished = {\url{https://huggingface.co/webbigdata/gemma-2-2b-jpn-it-translate-gguf/}},
  note         = {Accessed: 2024-10-10},
  abstract     = {This model was developed to verify how much Japanese-English and English-Japanese translation performance can be improved with the 2B gguf model.},
}