Views
No views yet
heack/HeackMT5-ZhSum100k, is a fine-tuned mT5 model for Chinese text summarization tasks. It was trained on a diverse set of Chinese datasets and is able to generate coherent and concise summaries for a wide range of texts.1from transformers import MT5ForConditionalGeneration, T5Tokenizer
2
3model = MT5ForConditionalGeneration.from_pretrained("heack/HeackMT5-ZhSum100k")
4tokenizer = T5Tokenizer.from_pretrained("heack/HeackMT5-ZhSum100k")
5
6chunk = """
7财联社5月22日讯，据平安包头微信公众号消息，近日，包头警方发布一起利用人工智能（AI）实施电信诈骗的典型案例，福州市某科技公司法人代表郭先生10分钟内被骗430万元。
84月20日中午，郭先生的好友突然通过微信视频联系他，自己的朋友在外地竞标，需要430万保证金，且需要公对公账户过账，想要借郭先生公司的账户走账。
9基于对好友的信任，加上已经视频聊天核实了身份，郭先生没有核实钱款是否到账，就分两笔把430万转到了好友朋友的银行卡上。郭先生拨打好友电话，才知道被骗。骗子通过智能AI换脸和拟声技术，佯装好友对他实施了诈骗。
10值得注意的是，骗子并没有使用一个仿真的好友微信添加郭先生为好友，而是直接用好友微信发起视频聊天，这也是郭先生被骗的原因之一。骗子极有可能通过技术手段盗用了郭先生好友的微信。幸运的是，接到报警后，福州、包头两地警银迅速启动止付机制，成功止付拦截336.84万元，但仍有93.16万元被转移，目前正在全力追缴中。
11"""
12inputs = tokenizer.encode("summarize: " + chunk, return_tensors='pt', max_length=512, truncation=True)
13summary_ids = model.generate(inputs, max_length=150, num_beams=4, length_penalty=1.5, no_repeat_ngram_size=2)
14summary = tokenizer.decode(summary_ids[0], skip_special_tokens=True)
15
16print(summary)
17
18包头警方发布一起利用AI实施电信诈骗典型案例:法人代表10分钟内被骗430万元1from transformers import MT5ForConditionalGeneration, T5Tokenizer
2
3model_heack = MT5ForConditionalGeneration.from_pretrained("heack/HeackMT5-ZhSum100k")
4tokenizer_heack = T5Tokenizer.from_pretrained("heack/HeackMT5-ZhSum100k")
5
6
7def _split_text(text, length):
8 chunks = []
9 start = 0
10 while start < len(text):
11 if len(text) - start > length:
12 pos_forward = start + length
13 pos_backward = start + length
14 pos = start + length
15 while (pos_forward < len(text)) and (pos_backward >= 0) and (pos_forward < 20 + pos) and (pos_backward + 20 > pos) and text[pos_forward] not in {'.', '。','，',','} and text[pos_backward] not in {'.', '。','，',','}:
16 pos_forward += 1
17 pos_backward -= 1
18 if pos_forward - pos >= 20 and pos_backward <= pos - 20:
19 pos = start + length
20 elif text[pos_backward] in {'.', '。','，',','}:
21 pos = pos_backward
22 else:
23 pos = pos_forward
24 chunks.append(text[start:pos+1])
25 start = pos + 1
26 else:
27 chunks.append(text[start:])
28 break
29 # Combine last chunk with previous one if it's too short
30 if len(chunks) > 1 and len(chunks[-1]) < 100:
31 chunks[-2] += chunks[-1]
32 chunks.pop()
33 return chunks
34
35def get_summary_heack(text, each_summary_length=150):
36 chunks = _split_text(text, 300)
37 summaries = []
38 for chunk in chunks:
39 inputs = tokenizer_heack.encode("summarize: " + chunk, return_tensors='pt', max_length=512, truncation=True)
40 summary_ids = model_heack.generate(inputs, max_length=each_summary_length, num_beams=4, length_penalty=1.5, no_repeat_ngram_size=2)
41 summary = tokenizer_heack.decode(summary_ids[0], skip_special_tokens=True)
42 summaries.append(summary)
43 return " ".join(summaries)
44
45| 企业类型 | 永久授权费（人民币元） |
|---|---|
| 初创企业或个人(年营业额100万以下) | 1,000元 |
| 中型企业(年营业额100万以上的非上市公司) | 5,000元 |
| 上市公司 | 20,000元 |
| Enterprise Type | Perpetual License Fee(CNY¥) |
|---|---|
| Startups Or Individuals(Annual Revenue < ¥1M) | 1,000 |
| Mid-sized Enterprises (Non-listed, Annual Revenue ≥ ¥1M) | 5,000 |
| Listed Companies | 20,000 |
1@misc{kongyang2023heackmt5zhsum100k,
2 title={HeackMT5-ZhSum100k: A Large-Scale Multilingual Abstractive Summarization for Chinese Texts},
3 author={Kong Yang},
4 year={2023}
5}