Alitongyi launches PrismAudio: Audio and video synchronized, sound effects follow.

Alibaba's Tongyi Labs has launched PrismAudio, a video-to-audio generation framework focusing on ambient sound/sound effect synthesis. The model innovatively combines reinforcement learning with a thought chain approach, using a "decompositional thought chain" to allow the model to first analyze video content, timing, sound quality, and spatial location before generating audio. It also incorporates four "teachers" (semantic, temporal, aesthetic, and spatial) to score and optimize the model.