AI Can Help Anyone Become a Beatbox Champion

To help up-and-coming musicians create the best beats for their song, developers from a Japanese-based AI startup developed a deep learning system called Neural…

NVIDIA logo
NVIDIA
•
Nefi Alarcon
•2 min read•beginner•
--
•View Original
KerasTensorFlow

Overview

The article discusses the development of a deep learning system called Neural Beatboxer by a Japanese AI startup, which allows users to convert everyday sounds into custom drum kits. It highlights the technology behind the system, including the use of NVIDIA GPUs and various deep learning frameworks.

What You'll Learn

1

How to use the Neural Beatboxer to create custom drum kits from everyday sounds

2

Why deep learning frameworks like Keras and TensorFlow.js are essential for audio classification and rhythm generation

3

When to apply convolutional and recurrent neural networks for audio processing tasks

Prerequisites & Requirements

  • Basic understanding of deep learning concepts(optional)
  • Familiarity with Keras and TensorFlow.js(optional)

Key Questions Answered

How does the Neural Beatboxer convert sounds into drum kits?
The Neural Beatboxer uses a convolutional neural network trained on a drum machine sound dataset to classify audio based on its spectrogram. Users can input sounds, and the system generates a custom drum kit that can last for hours.
What technologies are used in the development of Neural Beatboxer?
The Neural Beatboxer utilizes NVIDIA Quadro GP100 GPUs, cuDNN-accelerated Keras for deep learning, and a web front-end built with TensorFlow.js, magenta.js, and p5.js for user interaction.
What is the role of the Magenta project in this system?
The Magenta project provided a pre-trained recurrent neural network that was used for rhythm generation in the Neural Beatboxer, enhancing its ability to create novel music and rhythms.
What inspired Nao Tokui to create the Neural Beatboxer?
Nao Tokui was inspired to create the Neural Beatboxer while working as a DJ in Japan, feeling frustrated with the limitations of current AI music generation research and aiming to produce interesting and novel rhythms.

Technologies & Tools

Some links below are affiliate links. We may earn a commission if you make a purchase.

Hardware
Nvidia Quadro Gp100
Used for GPU acceleration in the deep learning processes.
Framework
Keras
Deep learning framework used for building the convolutional neural network.
Frontend
Tensorflow.js
JavaScript library used for building the web front-end of the Neural Beatboxer.
Frontend
Magenta.js
Library used in conjunction with TensorFlow.js for music generation.
Frontend
P5.js
JavaScript library used for creative coding in the web front-end.

Key Actionable Insights

1
Experiment with the Neural Beatboxer to create unique drum kits from everyday sounds, which can enhance your music production.
This tool allows musicians to explore new rhythmic possibilities that may not be achievable through traditional methods.
2
Leverage deep learning frameworks like Keras and TensorFlow.js to build your own audio processing applications.
Understanding these frameworks can empower you to create innovative solutions in the field of music and audio technology.
3
Consider the use of pre-trained models from projects like Magenta to accelerate your development process.
Utilizing existing models can save time and resources, allowing you to focus on refining your unique contributions to audio generation.

Common Pitfalls

1
Relying solely on AI-generated music without understanding the underlying technology can lead to uninspired results.
It's important to engage with the technology and understand its capabilities to create truly innovative music.