AB
AB
AiBoss

All Categories

project

AMD-135M - AMD's first small language model

The AMD-135M is AMD's first Small Language Model (SLM), offering a balance between performance and resource consumption for specific use cases. Based on the LLaMA2 model architecture, it is trained on the AMD Instinct MI250 accelerator, based on 67 billion...

What is AMD-135M?

AMD-135M is AMD's first Small Language Model (SLM), offering a balance between performance and resource consumption for specific use cases. Based on the LLaMA2 model architecture and trained on the AMD Instinct MI250 accelerator, it utilizes 67 billion tokens. AMD-135M comes in two versions: AMD-Llama-135M for general text and AMD-Llama-135M-code for code. AMD-135M employs speculative decoding, generating multiple tokens in each forward pass, improving inference speed and reducing memory consumption. In terms of performance, AMD-135M performs comparably to other similar models on the market across various natural language processing tasks. For example, it achieves a pass rate of approximately 32.31% on the Humaneval dataset using the MI250 GPU, a strong performance metric.

Main features of AMD-135M

  • Text generationIt can generate coherent text and supports applications such as chatbots and content creation.
  • Code generationBased on a slightly modified version of AMD-Llama-135M-code, it supports code generation and assisted programming.
  • Natural Language UnderstandingIt understands the intent and context of the input text, and is used in question-answering systems, summary generation, etc.
  • Multi-platform compatibilityIt runs on different hardware platforms, including AMD GPUs and CPUs.

AMD-135M Technical Principles

  • Speculation DecodingCandidate tokens are generated using a small draft model and then verified by a large target model, improving inference efficiency.
  • Autoregressive methodsThe traditional generation method generates a token with each forward pass, but it is optimized through speculative decoding technology.
  • Multi-head attention mechanismMulti-head attention mechanism is used to improve the model's ability to capture the relationships between different parts of the text.
  • Location codingRelative Position Encoding (RoPE) is used to preserve the order information of words in the sequence.
  • Activation functionUse the Swiglu activation function, which is an activation function designed specifically for language models.
  • Layer Norm: Use RMSNorm (Root Mean Square Normalization) to stabilize the training process.
  • Model ArchitectureBased on the LLaMA-2 model architecture, it has 12 layers, 768 hidden units, and other technical specifications.

Project address for AMD-135M

Application scenarios of AMD-135M

  • ChatbotAs the backend of a chatbot, it provides natural language understanding and generation capabilities to enable interactive dialogue with users.
  • Content creation: Assists in writing by generating drafts of articles, stories, or other text content.
  • Programming aidsBased on the AMD-Llama-135M-code version, it helps developers generate code snippets, provides programming suggestions, and offers auto-completion.
  • Language translationThe model is primarily designed for English, but the architecture is adapted to other languages for machine translation tasks.
  • Text SummaryAutomatically generates text summaries for quick overviews of news, articles, or reports.
Back