AB
AB
AiBoss

All Categories

project

TeleChat2-115B - Open-source Starry Sky Semantic Model launched by China Telecom AI Research Institute

TeleChat2-115B is a large-scale language model developed by the China Telecom Artificial Intelligence Research Institute (TeleAI), belonging to the Xingchen Semantic Large Model series. It is trained using domestically produced computing power and employs a high-quality Chinese and English language dataset with 10 trillion tokens...

What is TeleChat2-115B?

TeleChat2-115B is a large-scale language model developed by the TeleAI Research Institute of China Telecom, belonging to the Xingchen Semantic Large Model series. It is trained using domestically produced computing power and employs a high-quality Chinese and English corpus of 10 trillion tokens. Compared to its predecessor, TeleChat2-115B shows significant performance improvements in general question answering, knowledge-based, code-based, and mathematical tasks. The model has achieved excellent results on multiple benchmarks, such as ranking first in the C-Eval Open Access Model comprehensive leaderboard. The open-sourcing of TeleChat2-115B marks a significant advancement in domestic large-scale model training technology and will help promote innovation and industrial applications of large-scale model technology.

Main functions of TeleChat2-115B

  • Text generationIt can generate high-quality Chinese and English text.
  • Multilingual supportIt supports training on high-quality Chinese and English corpora and processes texts in both languages.
  • Multi-format and multi-platformIt provides weight files in multiple formats and on multiple platforms, making it easy to deploy and use in different environments.
  • High-performance inferenceSupports single-card and multi-card inference, as well as optimization for long-text inference.
  • API and Web DeploymentIt provides API and web deployment options, and supports streaming generation and multi-turn dialogues.

Technical Principles of TeleChat2-115B

  • Decoder-only structureDesigned using a standard Decoder-only architecture, it is used for text generation tasks.
  • Rotary EmbeddingThe positional encoding method using Rotary Embedding helps the model better capture relative positional information in sequence data.
  • SwiGLU activation functionThe SwiGLU activation function is used to replace the traditional GELU activation function to improve the performance of the model.
  • RMSNorm Pre-NormalizationLayer normalization based on RMSnorm pre-Normalization helps stabilize model training.
  • Separate word embeddings and output layer parametersSeparating the parameters of the word embedding layer and the output layer (lm head) helps to enhance training stability and convergence.
  • GQA optimizationChoosing GQA (Grouped Query Attention) saves on the number of parameters and computation in the attention part, improving training and inference speed.

TeleChat2-115B Project Address

Application scenarios of TeleChat2-115B

  • Intelligent Customer ServiceAs a chatbot, it provides customer consultation services and answers user questions.
  • Content creationIt assists in writing by generating text content such as articles, stories, and poems.
  • Language translationTo achieve high-quality Chinese-English translation.
  • Educational guidanceIt provides language learning and homework tutoring to help students understand complex concepts.
  • Programming aidsGenerate code snippets to help developers solve programming problems.
  • Data AnalysisProcessing and analyzing text data to extract useful information.
  • Intelligent SearchEnhance the search engine to provide more accurate search results.
Back