Futureverse, an AI and metaverse tech company with one of the largest digital collectible communities in the space, published its first research paper introducing a new infrastructure model to democratize high-end music quality by fusing musical compositions with artificial intelligence (AI).
The research paper, JEN 1: TEXT-GUIDED UNIVERSAL MUSIC GENERATION WITH OMNIDIRECTIONAL DIFFUSION MODELS,” is authored by members of Futureverse’s Altered State Machine innovation team, and dives into the intricacies of current audio generation.
JEN 1, according to the press release shared with nft now, is a “high-fidelity” sound synthesis model for “text-to-music generation” – combining musical compositions and free-form text prompts to produce higher quality sound output.
The advancement of AI, as we’ve experienced it so far in 2023, has already made substantial advancements with imagery and text. However, when it comes to producing authentic and realistic music – that isn’t stealing the image and likeness of an artist’s voice, lyrics, or melodies – there are still several challenges, which Futureverse’s research paper touches upon.
The research underscores the complexity of sound generation within musical compositions. Unlike speech, music requires higher sampling rates to capture its nuances, requiring a more refined training process. With AI, enhanced data processing during training requires high-quality audio that also considers the interplay of multiple instruments and intricate melodic structures.
Meet JEN 1
“Jen is spelled J-E-N because she’s designed to be your friend who goes into the studio with you. She’s a tool,” says Shara Senderoff, co-founder of Futureverse in an exclusive interview and first-look with Billboard.
Predicted to release in early 2024, JEN 1 can form up-to three-minute songs and help producers with half-written songs by offering ‘continuation’ and ‘in-painting’. It can perform text-guided music generation, music inpainting, and continuation while generating high-fidelity 48kHz stereo audio – and maintaining computational efficiency.
During the initial testing phases, JEN 1, according to the company, underwent rigorous evaluation against state-of-the-art models, with both computational metrics and human assessments conducted by an esteemed advisory board comprising music industry experts.
The results?
JEN 1 is superior in producing higher quality music at an average score of 85.7/100, compared to the current industry methods of 83.8/100. The company has also released early audio demos showcasing the capabilities of JEN 1.
“Human sensitivity to musical dissonance demands high precision in music generation,” said Senderoff and co-founder Aaron McDonald, who have been working deeply in the space for the last two years.
Senderoff, before launching Futureverse, co-founded the music tech venture fund, Raised In Space, with music industry titan Scooter Braun, helping push for innovation spanning music copyrights, royalties, and revenue streams by tapping into blockchain technology and Web3.
McDonald, who has spent 20 years in tech as an engineer, product developer, and investor with portfolios with over $1 billion, is focused on empowering developers and users to create and engage with interoperable content that is specifically built within metaverse applications.
This research paper seems to be the first step in actually applying AI tastefully to an industry-wide hindrance.
Launching ‘Born Ready,’ a $50 Million Venture Fund & Studio
Last week, Futureverse announced its new $50 million venture fund and studio, Born Ready, following its recently announced $54 million Series A funding round, led by 10T Holdings, including contributions from Ripple Labs. It has already integrated 11 metaverse infrastructure and content companies into one collaborative ecosystem, delivering the essential components needed to construct a metaverse application.
Over the past year, the company has secured strategic global partnerships with Wimbledon, FIFA, Authentic Brands Group (ABG), Mastercard, Death Row Records, Snoop Dogg, Keanu Reeves, Timbaland, and more. Its most recent partnership with the Cool Cats NFT collection fuses the brand’s IP with Futureverse’s cutting-edge technology, while both Senderoff and McDonald are joining Cool Cats’ Board of Directors.
