This is a Phi3 model uploaded using the KerasNLP library and can be used with JAX, TensorFlow, and PyTorch backends.
This model is related to a CausalLM task.
Model config:
name: phi3_backbone_1
trainable: True
vocabulary_size: 32064
num_layers: 32
num_query_heads: 32
hidden_dim: 3072
intermediate_dim: 8192
num_key_value_heads: 32
layer_norm_epsilon: 1e-05
dropout: 0.0
max_sequence_length: 4096
pretraining_sequence_length: 4096
rope_max_wavelength: 10000.0
rope_scaling_type: None
rope_scaling_short_factor: None
rope_scaling_long_factor: None
This model card has been generated automatically and should be completed by the model author. See Model Cards documentation for more information.
