Meta has introduced AudioCraft – an open-source AI-powered platform that lets users create music and sounds solely based on text prompts. This innovative technology offers a spectrum of possibilities, from generating simple noise to crafting complex melodies, all driven by the capabilities of generative AI.
Today we're sharing details on AudioCraft, a new family of generative AI models built for generating high-quality, realistic audio & music from text. AudioCraft is a single code base that works for music, sound, compression & generation — all in the same place.
— Meta AI (@MetaAI) August 2, 2023
More details ⬇️
AudioCraft consists of three models:
- MusicGen: This model enables the creation of melodies based on textual prompts. It was trained on 20,000 hours of music owned by Meta or licensed specifically for this purpose.
- AudioGen: Designed to simulate specific sounds from text inputs, AudioGen reproduces a range of auditory experiences, from a dog's bark to human footsteps. It draws on public sound effects for its training.
- EnCodec: By processing sounds and reducing artifacts, EnCodec ensures high-quality audio output, minimizing any unwanted distortions.
Kay Lee
The company allowed media representatives to preview sample audio clips produced by the AI platform. Impressively, sounds like whistling, sirens, and ambient noise emerged with remarkable authenticity. However, nuances such as the timbre of guitar strings still bore a slight artificial quality, as observed by experts.
