model_id = "hoang1123/llama3.1-8b-instruct-sum-trans"
from unsloth import FastLanguageModel
import torch
max_seq_length = 2048 
dtype = torch.float16
load_in_4bit = True

model, tokenizer = FastLanguageModel.from_pretrained(
  model_name = model_id,
  max_seq_length = max_seq_length,
  dtype = dtype,
  load_in_4bit = load_in_4bit,
)

system_prompt = "You are a helpful AI assistant. Please answer the following question as best you can. If you don't know the answer, just say that you don't know, don't try to make up an answer."
dialogue = """Donald John Trump (born June 14, 1946) is an American politician, media personality, and businessman who served as the 45th president of the United States from 2017 to 2021.

Trump received a Bachelor of Science in economics from the University of Pennsylvania in 1968. His father named him president of his real estate business in 1971. Trump renamed it the Trump Organization and reoriented the company toward building and renovating skyscrapers, hotels, casinos, and golf courses. After a series of business failures in the late 1990s, he launched successful side ventures, mostly licensing the Trump name. From 2004 to 2015, he co-produced and hosted the reality television series The Apprentice. He and his businesses have been plaintiffs or defendants in more than 4,000 legal actions, including six business bankruptcies.

Trump won the 2016 presidential election as the Republican Party nominee against Democratic Party nominee Hillary Clinton while losing the popular vote.[a] A special counsel investigation established that Russia had interfered in the election to favor Trump. During the campaign, his political positions were described as populist, protectionist, isolationist, and nationalist. His election and policies sparked numerous protests. He was the only U.S. president without prior military or government experience. Trump promoted conspiracy theories and made many false and misleading statements during his campaigns and presidency, to a degree unprecedented in American politics. Many of his comments and actions have been characterized as racially charged, racist, and misogynistic.

As president, Trump ordered a travel ban on citizens from several Muslim-majority countries, diverted military funding toward building a wall on the U.S.–Mexico border, and implemented a family separation policy. He rolled back more than 100 environmental policies and regulations. He signed the Tax Cuts and Jobs Act of 2017, which cut taxes and eliminated the individual health insurance mandate penalty of the Affordable Care Act. He appointed Neil Gorsuch, Brett Kavanaugh, and Amy Coney Barrett to the U.S. Supreme Court. He reacted slowly to the COVID-19 pandemic, ignored or contradicted many recommendations from health officials, used political pressure to interfere with testing efforts, and spread misinformation about unproven treatments. Trump initiated a trade war with China and withdrew the U.S. from the proposed Trans-Pacific Partnership trade agreement, the Paris Agreement on climate change, and the Iran nuclear deal. He met with North Korean leader Kim Jong Un three times but made no progress on denuclearization.

Trump is the only U.S. president to have been impeached twice, in 2019 for abuse of power and obstruction of Congress after he pressured Ukraine to investigate Joe Biden, and in 2021 for incitement of insurrection. The Senate acquitted him in both cases. Trump refused to concede after he lost the 2020 presidential election to Biden, falsely claiming widespread electoral fraud, and attempted to overturn the results. On January 6, 2021, he urged his supporters to march to the U.S. Capitol, which many of them attacked. Scholars and historians rank Trump as one of the worst presidents in American history.

Since leaving office, Trump has continued to dominate the Republican Party and is their candidate again in the 2024 presidential election. In May 2024, a jury in New York found Trump guilty on 34 felony counts of falsifying business records related to a hush-money payment to Stormy Daniels in an attempt to influence the 2016 election, making him the first former U.S. president to be convicted of a crime. He has been indicted in three other jurisdictions on 54 other felony counts related to his mishandling of classified documents and efforts to overturn the 2020 presidential election. In civil proceedings, Trump was found liable for sexual abuse and defamation in 2023, defamation in 2024, and for financial fraud in 2024. In July 2024, he survived an assassination attempt at a campaign rally in Pennsylvania."""

messages = [
    {
        "role": "system",
        "content": system_prompt,
    },
    {
        "role": "user",
        "content": f"# Please summary the following dialogue in three sentences:\n{dialogue}",
    },
]
input_ids = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    return_tensors="pt",
).to(model.device)

terminators = [128001, 128008, 128009]

FastLanguageModel.for_inference(model) # Enable native 2x faster inference
outputs = model.generate(
    input_ids,
    max_new_tokens=256,
    eos_token_id=terminators,
    do_sample=True,
    temperature=0.6,
    top_p=0.9,
)
response = outputs[0][input_ids.shape[-1]:]
print(tokenizer.decode(response, skip_special_tokens=True))

hoang1123/llama3.1-8b-instruct-sum-trans

Llama3.1-8b-Instruct-Finetuned

Datasets

Training

Inference

By Unsloth

By Huggingface

hoang1123/
llama3.1-8b-instruct-sum-trans