Phi-3.5 - Microsoft's next-generation AI model, a hybrid of mini, MoE, and vision models.

What is Phi-3.5?

Phi-3.5 is Microsoft's next-generation AI model series, comprising three versions: Phi-3.5-mini-instruct, Phi-3.5-MoE-instruct, and Phi-3.5-vision-instruct, designed for lightweight inference, hybrid expert systems, and multimodal tasks, respectively. Phi-3.5 is licensed under the MIT open-source license, offers different parameter scales, supports 128k context lengths, and features optimized multilingual processing and multi-turn dialogue capabilities. In benchmark tests, its performance surpasses that of similar models such as GPT4o, Llama 3.1, and Gemini Flash.

Performance evaluation and features of the Phi-3.5 series models

• Phi-3.5-mini-instruct 
  • ParametersThe Phi-3.5-mini-instruct has approximately 3.82 billion parameters.
  • Design PurposeThis model is designed to follow instructions and supports fast reasoning tasks.
  • Context supportIt supports context lengths of up to 128k tokens, making it suitable for processing long text data.
  • Applicable ScenariosSuitable for environments with limited memory or computing resources, it can perform tasks such as code generation, mathematical problem solving, and logic-based reasoning.
  • performanceIt performs well in multilingual and multi-turn dialogue tasks, and in the RepoQA benchmark, its performance in "long context code understanding" surpasses that of other models of similar size, such as Llama-3.1-8B-instruct and Mistral-7B-instruct.
  • Training detailsUsing 512 H100-80G GPUs, 3.4 trillion tokens were trained in 10 days.

• Phi-3.5-MoE-instruct
  • ParametersThe Phi-3.5-MoE-instruct has approximately 41.9 billion parameters.
  • Architectural featuresThis model employs a hybrid expert architecture, combining multiple different types of models into one, with each model specializing in handling different tasks.
  • Context supportIt supports context lengths of up to 128k tokens, making it suitable for handling complex multi-language and multi-tasking scenarios.
  • PerformanceIt excels in code, math, and multilingual understanding, and typically outperforms large models in specific benchmarks, including outstanding performance in the RepoQA benchmark.
  • Multitasking capabilityIn the 5-shot MMLU (Massive Multitask Language Understanding) benchmark test, it outperformed the GPT-40 mini at different levels across multiple disciplines, including STEM, humanities, and social sciences.
  • Training detailsIt used 512 H100-80G GPUs and trained 4.9 trillion tokens in 23 days.

• Phi-3.5-vision-instruct
  • ParametersThe Phi-3.5-vision-instruct has approximately 4.15 billion parameters.
  • Function integrationThis model integrates text and image processing capabilities, enabling it to handle multimodal data.
  • Applicable tasksIt is particularly suitable for tasks such as general image understanding, optical character recognition (OCR), chart and table understanding, and video summarization.
  • Context supportIt supports context lengths of up to 128k tokens, allowing models to manage complex multi-frame vision tasks.
  • Training dataThe model is trained using synthetic datasets and selected public datasets, with a focus on high-quality, inference-intensive data.
  • Training detailsIt used 256 A100-80G GPUs to train 500 billion tokens in 6 days.

Project address for Phi-3.5

• GitHub repository:https://github.com/microsoft/Phi-3CookBook
• Phi-3.5-mini-instruct model address:https://huggingface.co/microsoft/Phi-3.5-mini-instruct
• Phi-3.5-MoE-instruct model address:https://huggingface.co/microsoft/Phi-3.5-MoE-instruct
• Phi-3.5-vision-instruct model address:https://huggingface.co/microsoft/Phi-3.5-vision-instruct

How to use Phi-3.5

• Environmental preparationEnsure that the development environment meets the hardware and software requirements for the model to run, such as the Python environment, necessary libraries, and frameworks.
• Get the modelAccess the Hugging Face model library for the Phi-3.5 model and download the model code.
• Install dependenciesInstall the required dependencies, such as the Transformers library, PyTorch, or TensorFlow, according to the model's documentation.
• Loading Model: Load Phi-3.5 models using API or code snippets. For example, if using Hugging Face's Transformers library, you can load models by name or path.
• Data processingPrepare the input data and preprocess it according to the model's requirements, such as word segmentation and encoding.
• Model ConfigurationConfigure model parameters according to the application scenario, such as setting the context length and selecting specific task configurations.
• Execute the taskUse the model to perform the required tasks, such as text generation, question answering, and text classification.

Application scenarios of Phi-3.5

• Phi-3.5-mini-instructSmall and efficient AI models suitable for rapid text processing and code generation in embedded systems and mobile applications.
• Phi-3.5-MoE-instruct: Expert hybrid models provide deep reasoning for data analysis and multilingual text, suitable for interdisciplinary research and professional fields.
• Phi-3.5-vision-instructAdvanced multimodal processing capabilities, suitable for automatic image annotation, video surveillance, and in-depth analysis of complex visual data.