Views
No views yet
1llama.cpp $ ./quantize models/gemma-2b-it.gguf models/gemma-2b-it-Q4_K_M.gguf Q4_K_M
2main: build = 2351 (652ca2bd)
3main: built with Android (10552028, +pgo, +bolt, +lto, -mlgo, based on r487747d) clang version 17.0.2 (https://android.googlesource.com/toolchain/llvm-project d9f89f4d16663d5012e5c09495f3b30ece3d2362) for x86_64-apple-darwin22.5.0
4main: quantizing 'models/gemma-2b-it.gguf' to 'models/gemma-2b-it-Q4_K_M.gguf' as Q4_K_M
5llama_model_loader: loaded meta data with 19 key-value pairs and 164 tensors from models/gemma-2b-it.gguf (version GGUF V3 (latest))
6llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
7llama_model_loader: - kv 0: general.architecture str = gemma
8llama_model_loader: - kv 1: general.name str = gemma-2b-it
9llama_model_loader: - kv 2: gemma.context_length u32 = 8192
10llama_model_loader: - kv 3: gemma.block_count u32 = 18
11llama_model_loader: - kv 4: gemma.embedding_length u32 = 2048
12llama_model_loader: - kv 5: gemma.feed_forward_length u32 = 16384
13llama_model_loader: - kv 6: gemma.attention.head_count u32 = 8
14llama_model_loader: - kv 7: gemma.attention.head_count_kv u32 = 1
15llama_model_loader: - kv 8: gemma.attention.key_length u32 = 256
16llama_model_loader: - kv 9: gemma.attention.value_length u32 = 256
17llama_model_loader: - kv 10: gemma.attention.layer_norm_rms_epsilon f32 = 0.000001
18llama_model_loader: - kv 11: tokenizer.ggml.model str = llama
19llama_model_loader: - kv 12: tokenizer.ggml.bos_token_id u32 = 2
20llama_model_loader: - kv 13: tokenizer.ggml.eos_token_id u32 = 1
21llama_model_loader: - kv 14: tokenizer.ggml.padding_token_id u32 = 0
22llama_model_loader: - kv 15: tokenizer.ggml.unknown_token_id u32 = 3
23llama_model_loader: - kv 16: tokenizer.ggml.tokens arr[str,256128] = ["<pad>", "<eos>", "<bos>", "<unk>", ...
24llama_model_loader: - kv 17: tokenizer.ggml.scores arr[f32,256128] = [0.000000, 0.000000, 0.000000, 0.0000...
25llama_model_loader: - kv 18: tokenizer.ggml.token_type arr[i32,256128] = [3, 3, 3, 2, 1, 1, 1, 1, 1, 1, 1, 1, ...
26llama_model_loader: - type f32: 164 tensors
27llama_model_quantize_internal: meta size = 6042528 bytes
28[..snip..]
29llama_model_quantize_internal: model size = 9561.29 MB
30llama_model_quantize_internal: quant size = 1549.19 MB
31
32main: quantize time = 27285.22 ms
33main: total time = 27285.22 ms