Views
No views yet
ct2-transformers-converter --model MBZUAI/LaMini-Flan-T5-783M --output_dir lamini-flan-t5-783m-int8_float16 --quantization int8_float16git lfs install
git clone git@hf.co:saikatkumardey/lamini-flan-t5-783m_int8_float161import ctranslate2
2import transformers
3
4
5model_dir = "lamini-flan-t5-783m_int8_float16"
6translator = ctranslate2.Translator(
7 model_dir, compute_type="auto", inter_threads=4, intra_threads=4
8)
9tokenizer = transformers.AutoTokenizer.from_pretrained("MBZUAI/LaMini-Flan-T5-783M")
10
11
12input_prompt = """
13instruction: Restrict the answer based on the context only. Be verbose.
14context: <P>This is an introduction to pandas categorical data type, including a short comparison with R’s factor.
15
16Categoricals are a pandas data type corresponding to categorical variables in statistics. A categorical variable takes on a limited, and usually fixed, number of possible values (categories; levels in R). Examples are gender, social class, blood type, country affiliation, observation time or rating via Likert scales.
17
18In contrast to statistical categorical variables, categorical data might have an order (e.g. ‘strongly agree’ vs ‘agree’ or ‘first observation’ vs. ‘second observation’), but numerical operations (additions, divisions, …) are not possible.
19
20All values of categorical data are either in categories or np.nan. Order is defined by the order of categories, not lexical order of the values. Internally, the data structure consists of a categories array and an integer array of codes which point to the real value in the categories array.
21
22The categorical data type is useful in the following cases:
23
24- A string variable consisting of only a few different values. Converting such a string variable to a categorical variable will save some memory, see here.
25- The lexical order of a variable is not the same as the logical order (“one”, “two”, “three”). By converting to a categorical and specifying an order on the categories, sorting and min/max will use the logical order instead of the lexical order, see here.
26- As a signal to other Python libraries that this column should be treated as a categorical variable (e.g. to use suitable statistical methods or plot types).</P>
27
28question: In what cases is the categorical data type useful?
29answer:
30"""
31
32input_tokens = tokenizer.convert_ids_to_tokens(tokenizer.encode(input_prompt))
33
34results = translator.translate_batch(
35 [input_tokens],
36 beam_size=1,
37 sampling_temperature=0.01,
38 max_decoding_length=1024,
39 batch_type="tokens",
40 max_batch_size=64,
41)
42
43output_tokens = results[0].hypotheses[0]
44output_text = tokenizer.decode(tokenizer.convert_tokens_to_ids(output_tokens))
45
46print(output_text)