Beta
Explore
Marketplace
Neural Labs
Chat
Wallet
Docs
gemma-2-9b-ru-doc-spellfix-v2 – AI Model by Kemsekov | AlphaNeural AI
You can deploy this model and start earning money today!
Kemsekov
/
gemma-2-9b-ru-doc-spellfix-v2
like
0
keras-hub
text-generation
us
Views
No views yet
Model card
Files and Versions
Community
API
Deploy
This is a
Gemma
model
uploaded using the KerasNLP library and can be used with JAX, TensorFlow, and PyTorch backends. This model is related to a
CausalLM
task.
Model config:
name:
gemma_backbone
trainable:
True
dtype:
{'module': 'keras.dtype_policies', 'class_name': 'DTypePolicyMap', 'config': {'default_policy': None, 'policy_map': {'token_embedding': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_0/ffw_linear': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_0/ffw_gating_2': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_0/ffw_gating': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_0/attention/attention_output': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_0/attention/value': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_0/attention/key': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_0/attention/query': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_1/ffw_linear': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_1/ffw_gating_2': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_1/ffw_gating': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_1/attention/attention_output': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_1/attention/value': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_1/attention/key': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_1/attention/query': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_2/ffw_linear': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_2/ffw_gating_2': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_2/ffw_gating': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_2/attention/attention_output': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_2/attention/value': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_2/attention/key': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_2/attention/query': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_3/ffw_linear': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_3/ffw_gating_2': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_3/ffw_gating': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_3/attention/attention_output': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_3/attention/value': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_3/attention/key': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_3/attention/query': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_4/ffw_linear': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_4/ffw_gating_2': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_4/ffw_gating': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_4/attention/attention_output': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_4/attention/value': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_4/attention/key': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_4/attention/query': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_5/ffw_linear': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_5/ffw_gating_2': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_5/ffw_gating': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_5/attention/attention_output': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_5/attention/value': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_5/attention/key': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_5/attention/query': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_6/ffw_linear': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_6/ffw_gating_2': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_6/ffw_gating': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_6/attention/attention_output': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_6/attention/value': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_6/attention/key': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_6/attention/query': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_7/ffw_linear': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_7/ffw_gating_2': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_7/ffw_gating': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_7/attention/attention_output': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_7/attention/value': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_7/attention/key': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_7/attention/query': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_8/ffw_linear': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_8/ffw_gating_2': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_8/ffw_gating': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_8/attention/attention_output': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_8/attention/value': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_8/attention/key': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_8/attention/query': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_9/ffw_linear': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_9/ffw_gating_2': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_9/ffw_gating': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_9/attention/attention_output': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_9/attention/value': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_9/attention/key': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_9/attention/query': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_10/ffw_linear': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_10/ffw_gating_2': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_10/ffw_gating': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_10/attention/attention_output': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_10/attention/value': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_10/attention/key': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_10/attention/query': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_11/ffw_linear': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_11/ffw_gating_2': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_11/ffw_gating': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_11/attention/attention_output': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_11/attention/value': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_11/attention/key': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_11/attention/query': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_12/ffw_linear': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_12/ffw_gating_2': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_12/ffw_gating': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_12/attention/attention_output': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_12/attention/value': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_12/attention/key': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_12/attention/query': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_13/ffw_linear': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_13/ffw_gating_2': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_13/ffw_gating': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_13/attention/attention_output': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_13/attention/value': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_13/attention/key': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_13/attention/query': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_14/ffw_linear': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_14/ffw_gating_2': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_14/ffw_gating': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_14/attention/attention_output': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_14/attention/value': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_14/attention/key': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_14/attention/query': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_15/ffw_linear': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_15/ffw_gating_2': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_15/ffw_gating': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_15/attention/attention_output': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_15/attention/value': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_15/attention/key': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_15/attention/query': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_16/ffw_linear': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_16/ffw_gating_2': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_16/ffw_gating': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_16/attention/attention_output': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_16/attention/value': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_16/attention/key': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_16/attention/query': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_17/ffw_linear': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_17/ffw_gating_2': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_17/ffw_gating': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_17/attention/attention_output': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_17/attention/value': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_17/attention/key': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_17/attention/query': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_18/ffw_linear': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_18/ffw_gating_2': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_18/ffw_gating': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_18/attention/attention_output': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_18/attention/value': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_18/attention/key': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_18/attention/query': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_19/ffw_linear': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_19/ffw_gating_2': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_19/ffw_gating': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_19/attention/attention_output': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_19/attention/value': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_19/attention/key': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_19/attention/query': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_20/ffw_linear': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_20/ffw_gating_2': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_20/ffw_gating': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_20/attention/attention_output': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_20/attention/value': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_20/attention/key': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_20/attention/query': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_21/ffw_linear': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_21/ffw_gating_2': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_21/ffw_gating': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_21/attention/attention_output': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_21/attention/value': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_21/attention/key': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_21/attention/query': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_22/ffw_linear': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_22/ffw_gating_2': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_22/ffw_gating': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_22/attention/attention_output': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_22/attention/value': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_22/attention/key': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_22/attention/query': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_23/ffw_linear': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_23/ffw_gating_2': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_23/ffw_gating': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_23/attention/attention_output': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_23/attention/value': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_23/attention/key': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_23/attention/query': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_24/ffw_linear': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_24/ffw_gating_2': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_24/ffw_gating': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_24/attention/attention_output': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_24/attention/value': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_24/attention/key': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_24/attention/query': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_25/ffw_linear': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_25/ffw_gating_2': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_25/ffw_gating': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_25/attention/attention_output': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_25/attention/value': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_25/attention/key': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_25/attention/query': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_26/ffw_linear': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_26/ffw_gating_2': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_26/ffw_gating': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_26/attention/attention_output': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_26/attention/value': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_26/attention/key': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_26/attention/query': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_27/ffw_linear': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_27/ffw_gating_2': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_27/ffw_gating': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_27/attention/attention_output': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_27/attention/value': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_27/attention/key': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_27/attention/query': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_28/ffw_linear': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_28/ffw_gating_2': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_28/ffw_gating': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_28/attention/attention_output': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_28/attention/value': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_28/attention/key': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_28/attention/query': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_29/ffw_linear': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_29/ffw_gating_2': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_29/ffw_gating': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_29/attention/attention_output': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_29/attention/value': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_29/attention/key': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_29/attention/query': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_30/ffw_linear': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_30/ffw_gating_2': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_30/ffw_gating': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_30/attention/attention_output': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_30/attention/value': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_30/attention/key': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_30/attention/query': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_31/ffw_linear': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_31/ffw_gating_2': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_31/ffw_gating': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_31/attention/attention_output': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_31/attention/value': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_31/attention/key': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_31/attention/query': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_32/ffw_linear': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_32/ffw_gating_2': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_32/ffw_gating': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_32/attention/attention_output': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_32/attention/value': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_32/attention/key': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_32/attention/query': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_33/ffw_linear': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_33/ffw_gating_2': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_33/ffw_gating': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_33/attention/attention_output': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_33/attention/value': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_33/attention/key': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_33/attention/query': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_34/ffw_linear': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_34/ffw_gating_2': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_34/ffw_gating': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_34/attention/attention_output': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_34/attention/value': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_34/attention/key': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_34/attention/query': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_35/ffw_linear': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_35/ffw_gating_2': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_35/ffw_gating': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_35/attention/attention_output': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_35/attention/value': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_35/attention/key': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_35/attention/query': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_36/ffw_linear': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_36/ffw_gating_2': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_36/ffw_gating': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_36/attention/attention_output': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_36/attention/value': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_36/attention/key': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_36/attention/query': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_37/ffw_linear': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_37/ffw_gating_2': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_37/ffw_gating': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_37/attention/attention_output': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_37/attention/value': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_37/attention/key': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_37/attention/query': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_38/ffw_linear': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_38/ffw_gating_2': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_38/ffw_gating': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_38/attention/attention_output': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_38/attention/value': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_38/attention/key': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_38/attention/query': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_39/ffw_linear': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_39/ffw_gating_2': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_39/ffw_gating': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_39/attention/attention_output': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_39/attention/value': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_39/attention/key': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_39/attention/query': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_40/ffw_linear': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_40/ffw_gating_2': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_40/ffw_gating': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_40/attention/attention_output': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_40/attention/value': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_40/attention/key': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_40/attention/query': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_41/ffw_linear': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_41/ffw_gating_2': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_41/ffw_gating': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_41/attention/attention_output': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_41/attention/value': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_41/attention/key': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}, 'decoder_block_41/attention/query': {'module': 'keras.dtype_policies', 'class_name': 'QuantizedDTypePolicy', 'config': {'mode': 'int8', 'source_name': None}, 'registered_name': None}}}, 'registered_name': None}
vocabulary_size:
256000
num_layers:
42
num_query_heads:
16
num_key_value_heads:
8
hidden_dim:
3584
intermediate_dim:
28672
head_dim:
256
layer_norm_epsilon:
1e-06
dropout:
0
query_head_dim_normalize:
True
use_post_ffw_norm:
True
use_post_attention_norm:
True
final_logit_soft_cap:
30
attention_logit_soft_cap:
50
sliding_window_size:
4096
use_sliding_window_attention:
True
This model card has been generated automatically and should be completed by the model author. See
Model Cards documentation
for more information.
