Phi-2 is a Transformer with
2.7 billion parameters. It was trained using the same data sources as
Phi-1.5, augmented with a new data source that consists of various NLP synthetic texts and filtered websites (for safety and educational value). When assessed against benchmarks testing common sense, language understanding, and logical reasoning, Phi-2 showcased a nearly state-of-the-art performance among models with less than 13 billion parameters.
Given the nature of the training data, the Phi-2 model is best suited for prompts using the QA format, the chat format, and the code format. Phi-2 is intended for QA, chat, and code purposes; the model-generated text/code should be treated as a starting point rather than a definitive solution for potential use cases.
Phi-2 may generate inaccurate code snippets and facts; its scope for code is limited (the majority of its training data is Python-based); it may struggle to adhere to intricate or nuanced instructions as it has not undergone instruction fine-tuning; it is primarily designed to understand standard English; it is not entirely free from societal biases; it can still produce harmful content if explicitly prompted; and as a base model it often produces irrelevant or extra text following its first answer.
This model is licensed under the MIT license.
