CDial-GPT - A large-scale Chinese short text dialogue dataset and dialogue generation model launched by Tsinghua University.

What is CDial-GPT?

CDial-GPT is a large-scale Chinese short text dialogue dataset, LCCC, and a pre-trained dialogue generation model based on it, developed by a research team at Tsinghua University. The LCCC dataset undergoes a rigorous cleaning process and includes a basic version (LCCC-base) and an extended version (LCCC-large) to improve the quality of dialogue models. The research team provides a GPT model pre-trained on the LCCC dataset. This model was first pre-trained on a Chinese novel dataset and then further trained on top of that, generating more natural and fluent dialogues. The CDial-GPT model is of great significance to research in the field of Chinese natural language processing and will help advance the development of Chinese dialogue systems.

Main functions of CDial-GPT

• Provides large-scale Chinese dialogue datasetsCDial-GPT has released two versions of the Chinese dialogue dataset (LCCC-base and LCCC-large). The datasets have been rigorously cleaned and are intended for research and development of Chinese dialogue systems.
• Pre-trained dialogue generation modelBased on the LCCC dataset, CDial-GPT provides a pre-trained dialogue generation model. The model learns from a large amount of Chinese dialogue data and can generate more natural and appropriate dialogue responses.
• Supports fine-tuningIt provides fine-tuning capabilities for pre-trained models, allowing researchers and developers to further optimize model performance on specific dialogue tasks or domains.
• Model EvaluationEvaluate the performance of pre-trained models on standard dialogue datasets, providing results from both automated and human evaluations to help users understand the model's dialogue generation capabilities.
• Interactive DialogueUsers can interact with the model in real time via command line to generate responses, which helps to test and experience the model's conversational capabilities.

CDial-GPT Technical Principles

• Data cleaningBased on a series of rules and a machine learning-based classifier, the original dialogue data is cleaned to remove invalid or low-quality dialogue, such as dialogue containing profanity, emojis, grammatical errors, etc.
• Knowledge Graph ConstructionThe cleaned data is organized into a knowledge graph, and the nodes and edges in the graph represent the entities and relationships in the dialogue.
• Transformer architectureBased on the Transformer architecture, it is a deep learning model with a self-attention mechanism that effectively processes sequential data, such as text.
• Pre-training and fine-tuningFirst, the model is pre-trained on a large-scale Chinese novel dataset to learn the basic rules of the language. Then, it is fine-tuned on a specific dialogue dataset to better adapt the model to the dialogue generation task.
• Multimodal learningBy combining various types of data such as text and images, the model's ability to understand and generate dialogue content can be improved.

CDial-GPT project address

• GitHub repository:https://github.com/thu-coai/CDial-GPT
• arXiv technical paper:https://arxiv.org/pdf/2008.03946

Application scenarios of CDial-GPT

• Customer ServiceIn the customer service field, CDial-GPT builds chatbots to provide automated customer consultation and problem-solving services.
• Smart AssistantIn smartphones and smart home devices, it acts as a smart assistant, understanding and responding to users' natural language commands.
• Online EducationAs an automated Q&A system for online education platforms, it provides learning guidance and interaction.
• social mediaOn social media platforms, CDial-GPT helps generate interactive conversational content, increasing user engagement.
• Content creation: Assist content creators in generating articles, stories, or other text content.
• Language learningAs a language learning tool, it helps learners practice Chinese conversation and comprehension.