XVERSE-Ent - A large-scale, open-source, pan-entertainment bilingual (Chinese and English) base model.

What is XVERSE-Ent?

XVERSE-Ent is an open-source, bilingual (Chinese and English) foundational model designed specifically for the pan-entertainment industry by Metaverse Technology. It includes the Chinese model XVERSE-Ent-A4.2B and the English model XVERSE-Ent-A5.7B. The model excels in character consistency, long-term story comprehension, and adaptation to diverse contexts. It can maintain the character design and style of virtual characters over a long period, accurately grasp complex storylines, and provide stylized expressions according to different themes. Technological innovations include MoE hot-start technology and a multi-stage training strategy, trained on nearly a trillion tokens of data with an 8K context window. The model parameters are 25B and 36B respectively, supporting high-concurrency, low-cost deployment, suitable for single-card cloud operation, and significantly lowering the deployment threshold. XVERSE-Ent performs exceptionally well in core tasks such as novel and dialogue generation while retaining over 98% of its general capabilities, making it a highly promising AI model in the pan-entertainment field.

XVERSE-Ent's main functions

• Enhanced character consistencyIt can maintain the character design, memory, and speaking style of virtual characters for a long time, effectively avoiding the problems of "forgetting" or "breaking out of the role", and providing a stable experience for role-playing and virtual interaction.
• Accurate understanding of long-running storylinesIt can accurately grasp complex storylines, character relationships, and foreshadowing, generating logically sound and coherent plot content, suitable for novel writing, screenwriting, and other scenarios.
• Multi-context adaptationIt possesses rich stylistic expression and background knowledge for different themes (such as ancient style, science fiction, urban, etc.), and can flexibly adjust language style and content output according to specific scenarios.
• Bilingual support in Chinese and EnglishIt includes the Chinese model XVERSE-Ent-A4.2B and the English model XVERSE-Ent-A5.7B, which have been optimized for Chinese and English contexts respectively to meet the creation and interaction needs of users of different languages.
• High-concurrency, low-cost deploymentIt supports single-card deployment in the cloud, achieving a balance between inference cost and processing efficiency, significantly reducing the deployment threshold and operation and maintenance costs for developers, and facilitating efficient operation on various devices.

XVERSE-Ent's Technical Principles

• MoE Hot Start TechnologyBy transforming a Dense model into a MoE model, the FFN part is finely split into multiple sub-networks as experts, and the expert sub-networks are copied according to the memory size during inference, so as to achieve efficient model expansion and performance improvement.
• Multi-stage training strategyThe training process employs three phases: S0 capability reconstruction, S1 language bias, and S2 domain enhancement. This phase gradually builds specialized models applicable to specific languages and domains, ensuring the model's professionalism and versatility in the pan-entertainment field.
• Large model architecture and parameter optimizationThe Chinese model XVERSE-Ent-A4.2B and the English model XVERSE-Ent-A5.7B have 25B and 36B parameters respectively. They have been trained on nearly a trillion tokens of data and have powerful language generation and understanding capabilities.
• 8K Context WindowIt supports 8K context windows, enabling it to handle longer text input and output, making it suitable for complex plot generation and long dialogue tasks.
• Low-cost and efficient deploymentIt supports high-concurrency, low-cost deployment solutions, making it particularly suitable for cloud-based single-card deployments. It optimizes the balance between inference costs and processing efficiency, lowering the deployment threshold for developers.

XVERSE-Ent's project address

• Github repository: https://github.com/xverse-ai/XVERSE-Ent
• HuggingFace model libraryhttps://huggingface.co/collections/xverse/xverse-ent

XVERSE-Ent application scenarios

• Virtual character interactionIt gives virtual characters a consistent and coherent persona and dialogue style, supports a long-term stable interactive experience, and is suitable for scenarios such as virtual idols and virtual customer service.
• Novel and screenplay writingIt can generate coherent and creative long stories and scripts, support complex plot lines and character relationships, and help creators quickly produce high-quality content.
• Game story generationIt provides dynamically generated storylines and dialogues for games, enhancing immersion and playability, and is suitable for role-playing games (RPGs) and interactive narrative games.
• Cross-cultural communicationIt supports bilingual generation in Chinese and English, which can help users from different language backgrounds to engage in creative writing and interactive dialogue, and promote cross-cultural content creation and communication.
• Social media content creationGenerate copy, stories, or dialogues that fit a specific style and theme, helping creators quickly generate engaging social media content.
• Education and TrainingBy generating interactive dialogues and scenario simulations, it can be used in educational scenarios such as language learning and vocational training to provide a more vivid learning experience.