ChildMandarin - A Chinese speech dataset for young children jointly developed by Zhiyuan and Nankai University.
ChildMandarin is a Mandarin speech dataset for children aged 3-5, jointly developed by the Beijing Academy of Artificial Intelligence (BAAI) and the Human Language Technology Lab (HLT Lab) of the School of Computer Science at Nankai University. The dataset contains 41.25 hours of speech...
What is ChildMandarin?
ChildMandarin is a Mandarin speech dataset for children aged 3-5, jointly developed by the Beijing Academy of Artificial Intelligence (BAAI) and the Human Language Technology Lab (HLT Lab) of the School of Computer Science at Nankai University. The dataset contains 41.25 hours of speech data from 397 children across 22 provincial-level administrative regions in China, with a balanced gender distribution. The data was recorded using smartphones to ensure high-quality audio. A core feature of ChildMandarin is its natural and realistic interaction, using parent-guided dialogue to simulate natural communication scenarios. This dataset fills a gap in early childhood speech research and promotes the development of children's speech recognition, language development research, and intelligent voice interaction systems.
ChildMandarin's main functions
- Speech recognitionIt provides a large amount of natural speech data from children aged 3-5 years to improve the accuracy and robustness of children's speech recognition.
- Speaker verificationSupports speaker verification (SV) tasks, helping to identify and differentiate the voices of different children, and can be used in scenarios such as child identification.
- Language ResearchTo provide data support for research on children's language development and to help develop children's language learning tools and interactive education systems.
ChildMandarin's technical principles
- Data collectionUsing a parent-guided dialogue approach, natural communication scenarios are simulated to ensure the authenticity and naturalness of the voice data. Data collection covers 22 provincial-level administrative regions in China, ensuring the diversity of regional accents. Recordings are made using smartphones (Android and iPhone) to ensure high audio quality, with a sampling rate of 16kHz and 16-bit precision.
- Data labelingManual annotation by professional transcribers includes natural language phenomena such as children's pronunciation, pauses, and repetitions. Information such as the speaker's age, gender, place of birth, recording equipment, and accent level is also annotated.
- Model training and evaluationThe system employs various ASR models (such as Transformer, Conformer, and Paraformer) for training and evaluation, based on techniques like CTC, AED, and RNN-T. Pre-trained models (such as HuBERT and Whisper) are fine-tuned using features learned on large-scale data to improve the performance of children's speech recognition. Speaker embedding extraction models (such as x-vector, ECAPA-TDNN, and ResNet-TDNN) are used for speaker verification tasks to evaluate the model's performance on children's speech.
- Dataset DesignThe dataset is divided into training, validation, and test sets to ensure the scientific rigor and effectiveness of model training and evaluation. Diverse data collection and annotation methods ensure that the dataset covers children's speech across different age groups, genders, regions, and accents.
ChildMandarin's project address
- GitHub repository:https://github.com/flageval-baai/ChildMandarin
- HuggingFace model library:https://huggingface.co/datasets/BAAI/ChildMandarin
- arXiv technical paper:https://arxiv.org/pdf/2409.18584
Application scenarios of ChildMandarin
- Children's language learning toolsDevelop intelligent voice-assisted tools to help children learn language pronunciation, vocabulary, and grammar, thereby improving their language skills.
- Interactive Education SystemIt provides voice interaction functionality for children's educational software and interactive learning platforms, making the learning process more vivid and interesting.
- Smart toy developmentEnhance the voice recognition capabilities of smart toys to better understand children's voice commands and improve the interactive experience.
- Voice assistant optimizationImprove the recognition and response of voice assistants (such as smart speakers and mobile phone assistants) to children's voices, making them more suitable for children to use.
- Children's health managementTo monitor children's language development and health status, and to support early intervention.