Google has introduced the Magenta RealTime neural network, which generates melodies not in their entirety, but in separate fragments lasting two seconds. This allows the AI model to run on weak hardware and provides several other advantages.
The neural network with 800 million parameters was trained on 190,000 hours of compositions from open sources. It generates the result in stages, so the user has the opportunity to control the creation of the track in real time and direct the process. Magenta RealTime takes into account not only prompts, but also the content of the previous 10 seconds of the already generated track, so the melody is consistent.
Another advantage of this type of generation is the ability to work even on relatively weak hardware. Thus, when running in Google Colab on a free plan, one two-second segment with a sampling frequency of 48 kHz is created in just 1.25 seconds.
The training dataset was based on compositions without vocals, so the neural network can generate only such tracks, and the range of genres is limited to Western ones. The model has already been published on Hugging Face , and its source code is on GitHub . The demo version can be tried on Google Colab .