Artificial intelligence is quietly transforming the world as we know it - and AI in music is no exception. From song-identifying apps to composition tools, all the way to lyric generators and production platforms, AI is reshaping the way we create, share, and even discover music. In this episode of The Mix, Cari, Silvia, and Stella share their professional experience and research into the topic of AI in music, as well as chat with AI music expert Valerio Velardo about his thoughts on these exciting innovations and predictions for the evolution of the industry. Join us as we explore the world of AI in music to see how it is already being used and what’s in store for the future.
Useful links:
Valerio Velardo- website
Valerio Velardo-The Sound of AI Youtube Channel
Musixmatch Pro Beta
Thanks for tuning in! This Mix is an industry innovation podcast powered by Musixmatch Pro. Our mission is to demystify complex new trends in the music industry while gathering top tips to help musical artists better monetise, distribute, and advertise their music.
Find us on: Spotify, Amazon Music, Google Podcasts and don’t forget to hit subscribe to never miss an episode.
Cari Quoyeser: Hello and welcome! You're listening to The Mix, a Musixmatch PRO podcast hosted by yours truly, Cari Quoyeser, in Artist Community and Services.
Silvia Olivieri: Silvia Olivieri, in Publishing.
Stella Tavella: and Stella Tavella, in AI.
Cari Quoyeser: We’re here to bring you the scoop on what's new in music innovation to help you navigate the modern music industry. Enjoy!
Silvia Olivieri: In today's episode, we will talk about the application of Artificial Intelligence in music composition. AI has become an increasingly popular mode of co-creation.
Stella Tavella: Cool, right?
Cari Quoyeser: But what does that actually mean?
Stella Tavella: The three of us will break down the most common questions about AI in songwriting.
Cari Quoyeser: Questions like, will AI compositions replace human compositions altogether?
Silvia Olivieri: How do artists feel about AI and how are they currently using it?
Cari Quoyeser: What are the implications of using this kind of technology…
Silvia Olivieri: …and how do we even begin to establish corporate ownership?
Stella Tavella: Sound like a lot?
Cari Quoyeser: Don't worry. There's a lot to untangle, but along with our guest AI expert Valerio Velardo, we're here to help guide you through every step, starting with the most basic question.
Cari Quoyeser: So, what is artificial intelligence, AI?
Silvia Olivieri: Let's start off easy and define what AI means. If you're not an expert or familiar with AI, you might think about robots taking over the world. It's not really like that.
Stella Tavella: Artificial intelligence is the simulation of human intelligence processes by computer systems. It refers to the ability of the machine to learn and keep improving its performance without the necessity of humans explicitly programming it for a particular outcome.
Artificial intelligence comprises a set of different techniques. A subset of AI is the field of machine learning, by which computers make use of statistics and logic for the learning process.
Another subset of AI and machine learning is deep learning, where with the use of a large amount of data and sophisticated algorithms, the machine is able to autonomously train itself without the need of having specific engineered features as input.
Silvia Olivieri: So, AI has different applications in music. It helps enhance the consumer experience, but also it gives machines the ability to become music co-creators. Today, we want to discuss the latter.
Silvia Olivieri: While writing music is an expression of human creativity, music has a deeply mathematical composition. The rhythm, scales, intervals, and harmonies all are based on mathematical structures. The first studies on AI and music date back to 1980, but the recent developments have been possible thanks to the advances in both technology and computation.
In conjunction to this, the AI now has easy access to larger datasets and sound sources that it can study and from which it can create new music. This is called data mining, and it's the extraction of contents from a database. In simple terms, where the AI is taking the data to study from.
Now, we could open a whole new chapter on this and try to analyze what happens if AI uses copyrighted data to develop. However, today's episode is not about that as we want to concentrate on how AI is serving artists right now.
Cari Quoyeser: So as our tech and AI expert, Stella, you would probably be the best to explain how this is happening.
Stella Tavella: Well, actually, there is some really interesting software on the market currently. Some of them, for instance, can translate vocals into any instrument. So maybe if someone has tons of ideas, but not the classical training or time to write out the whole score, they'd still be able to capture it and express them.
Cari Quoyeser: Yeah, I actually saw an advertisement for a phone app that you can hum the tune and it generates the music, which is so crazy.
Stella Tavella: Additionally, there is other software that maybe doesn’t auto-generate the whole tune, but that are able to play piano or assist in the creative process in other ways.
Cari Quoyeser: Yeah, it's wild to think about the possibilities that are arising with this new technology, but I also think it's really important to recognize for songwriters or producers or anyone in that sector that most of us are using AI softwares already, right now, and have been for a long time without realizing that it was AI.
So in the studio, things like auto-tune, auto-leveling, auto-mixing and mastering, or any instance really where the computer learns where a note, pitch, or level should be relative to the others, and implements that learning autonomously, is AI. Applying this general definition, there are a ton of other super practical uses of AI.
I for instance (with Stella's help of course) have used an AI software that intelligently separates individual instrument tracks from a master copy of one of my songs. This is really useful if perhaps you've lost access to the stems and still want to edit or remix your own track. Far from robbing me of my own creative process, softwares like this save hours of time, they make our jobs easier.
Cari Quoyeser: I think we can argue that AI is serving a number of different types of needs. A great example of this however, is Amper Music and Jukedeck who were created with the purpose of offering low budget compositions for background music and vlogs and videos to help people that perhaps can't afford music licenses of manmade compositions.
Additionally, to pull it back to creatives, AI compositions can serve as inspiration for songwriters and through human collaboration can be crafted into songs indistinguishable from the ones created through the typical process.
Silvia Olivieri: How do we feel about AI being used to write music?
Cari Quoyeser: As we mentioned before, there's a big difference between using AI to level your music or quickly tweak a sharp note, than it is to use it to generate new ideas or write full compositions entirely, for that matter. For the latter application, AI can be kind of a controversial subject.
Stella Tavella: In fact, even Cari and I have different perspectives on this matter as two very different types of creators. Cari, you're more of a, shall we say analog musician, and I'm a producer and computer scientist that makes electronic experimental music, right?
Cari Quoyeser: Absolutely. I definitely see from my end of things, a pushback from musicians that make music in the more traditional way, like myself, but I think comfort levels vary. For instance, personally, I don't mind AI creating harmonies, rifts, or accompaniment, but I get a little bit weird about AI writing lyrics as lyrics are really important to me specifically as a songwriter.
All of this being said, I know some artists like you Stella that use AI to create your art. And as you've explained it to me, AI is something that you collaborate with, rather than something that writes for you. That's not to say however, that there is an AI software out there doing just that and doing it successfully.
Stella Tavella: So from my perspective, in the world of experimental music and production, I can see that there is a lot of excitement about AI being used for music creation. There have been a lot of well known artists that have been making use of this technology for quite a while now.
Artists such as Holly Herdon or Ash Koosha for example, since this is a very experimental field of music, I can also see that producers who aren't really familiar with these tools yet are curious to learn more and excited to try this type of software for themselves.
Silvia Olivieri: Will technical musical skills become obsolete. To what degree?
Cari Quoyeser: It's hard to say to what extent AI will progress. Currently, it's used most widely in making our jobs of creating and producing easier, but I don't think AI could ever replace technical jobs altogether. For example, I know very well how to record, mix and master myself but I'll still pay for an engineer, a producer to help me do it.
Having that extra body in the room, their trained ear, their impression, and their emotional reaction to a take is invaluable to making sure the track is the best it can be. AI can tell you what's technically correct, but sometimes having a trained person there can pull the best performances out of you. At least, that's been my experience.
Stella Tavella: Like with every technology that has ever existed, the initial concern is that humanity will stop learning and become too limited by tech. I say, this is like when humans invented writing, they needed it in order to avoid remembering everything in their minds.
Writing is a powerful tool that allowed all humanity to be informed and educated faster. It did not make people less smart just because they could read things instead of remembering them.
Cari Quoyeser: When AI helps you write the song, who gets the publishing rights?
Silvia Olivieri: So before starting to navigate such a complex topic, it's necessary to understand better what the scope of copyright production is. We will try to make it simple, I promise!
First, as the work is considered to be the result of the author’s intent and labor, copyright ensures the protection of the work by granting the author both moral and economic rights. Moral right is the right to be considered as the author of the work, as well as the right to object to any alteration of such work.
The economic rights, instead, ensure exclusive rights, such as copying, distributing, and licensing the work. Getting the money, let's say. Because copyright protects both moral and economic rights, the inclusion of AI music leads to two main problems. The first one is ethical. Can the AI entity be considered an author and consequently, be ensured moral protection?
The second is economical. A machine doesn't have any economic interest, thus copyright wouldn't be able to incentivize the creation and development of AI music. And how do we pay it? If we assign the rights to a machine, then the machine needs to be considered a fully fledged theoretical entity and be able to take decisions and responsibility for the work.
Does this mean we will need to consider AI the same level as humans? This has a huge ethical implication too. Now, we don't want to make things more complicated than they already are. And we want to use this space to educate and push whoever is listening to research and be more at ease with new technologies. We don't want to scare anyone here.
Cari Quoyeser: In keeping all of this in mind, what's the answer to the question?
Silvia Olivieri: At the current state of the technology, considering the possibility of a machine author seems to be possible only on a hypothetical level. Granting no protection at all is also impossible. Why? Because denying copyright protection would mean denying AI the protection it needs to keep improving. Indeed, without any protection, there would be no incentive for the AI music to be further developed.
A lot of people may think, ah, well, we have so many musicians, we don't need AI. That's true. But it's also true that you can't stop technological development, so it's better if we learn to live with it and exploit it in the best way possible. Also, given what we have discussed so far, we are not yet at the level where the AI machine is doing everything.
Instead, we are in an exciting moment where artists use AI as a collaborative tool, inventing new sounds and challenging their creativity. So, what has been recently discussed in both EU and US legislation is that AI is a tool, like a camera for a photographer. You wouldn't grant copyright protection to the camera.
Cari Quoyeser: In the next and final chapter, our AI manager Stella and I were lucky enough to sit down with AI music expert Valerio Velardo. Besides having 10 years of experience in the field of AI as an engineer and consultant, Valerio is a classically trained musician with a PhD in AI music. Valerio founded Melodrive, a company that creates AI music for video games.
He has two prominent podcasts, The Sound of AI, that educates his listeners on the hard and soft skills necessary to create AI music for themselves, and The Sound of AI: Tech and Society that explores the impact of AI on society. If that's not enough, Valerio also founded and currently runs the largest AI audio and music community in the world.
Here he is: Valerio Velardo.
Valerio, thank you so much again for being here. We're just gonna jump right in, if that's okay with you. The first question we have is: in your opinion, what is the potential of AI to be used in composition and how will this help musicians and songwriters in their creative process?
Valerio Velardo: Yeah, I think there is some big potential there for AI music composition. And up until now, we've only scratched the surface I believe. And that's because we are mainly concentrating on, in a sense, trying to replace musicians, trying to create some applications that can generate music, and trying to replace what composers can do right now.
But I think the real hidden gem there is the possibility of expanding what's possible for musicians, namely, the possibility of creating a sort of infinite version of the very same piece of music. That's because potentially you could use an AI creative music system to pass it a music script that can then be implemented in a number of different musical implementations that potentially respond to different inputs and external variables. Like for example, the weather, your emotions, or some sort of emotional setting, for example, in a video game. So I see that as the most interesting application of AI music generation in the future.
Cari Quoyeser: I can see why. Personally, the gears are turning in my head when I start thinking about all the different ways we could create art with these kinds of variables. To change gears a bit, I was wondering, in your opinion, why do you think AI is more commonly used in certain genres over others? For instance, electronica or you know, something like that.
Valerio Velardo: Well, I think right now, the way AI is used in music is mainly for music production. Just like automatic mastering - we have a few companies that do that and do that reasonably well, like for example, Lander, right?
And I wouldn't necessarily say that AI is mainly used for certain types of musical music genres, but rather, that some of these (genres) lend themselves better to AI. That's because, for example, for a synthesizer or synthesizer-based music, it's easier to have an AI that can help you in a number of different things.
Like the production part of the composition, if you will, as well as the possibility of generating new sounds that are completely unheard of. And that's a little bit more difficult if you're talking about some music that's a little bit more traditional, like some sort of art music, contemporary music, because you don't need that type of extra help where AI really shines, (generating new sounds) because you’re using an acoustic instrument.
But nonetheless, I believe that there are many possibilities in that aspect as well. I think we have to make a distinction here between using AI for purely composition purposes, in other words, in order to create music or help you create some music. And on the other hand, AI, helping you with the production part of things, like mastering, mixing, or creating new sounds and these sort of things. For the latter situation, I believe that electronic music can have more applications.
Cari Quoyeser: You kind of already answered this a little bit, but how do you expect AI to be positioned in the music industry in the future as a product?
Valerio Velardo: I think right now, the biggest obstacle that we have in AI-generated music is the fact that we don't have a market yet for that. And there are many reasons for that, number one probably being that the technology is still not up to that level. It's not as good as a human being would be actually composing some music.
But there's also another aspect which is a lot of inertia from users and especially bigger companies. I used to work in generating music for video games and it is really difficult to go to video gaming companies and tell them, “Hey, let's just switch the way that you create music from your traditional way to this completely crazy and new way of doing music, helped through artificial intelligence.” So, I believe that the main problem moving there would be to have people, the stakeholders, actually using this technology. And for that to happen, we have to create products which make sense and are not just some really cool technologies out there.
In that respect, I believe that probably the most interesting work that will open the way for AI music generation is the work that people are doing within social media, where it will be possible for normal users (like TikTok users or Instagram users) to add some sort of customized music, very simple music, but quite effective and very personal, which I think is the main thing. So having the illusion of having something that's completely geared for you and that you had a say in, in terms of the composition process.
And for me, that would be probably the most interesting near-term application. So you have a big social network company that actually adopts music generation AI to enable their users to create music that they could actually couple with the content that they create on that platform. I think that this is an extremely interesting aspect and I see a real market potential for this. Then there are also other applications, but these would be for more expert musicians.
There, I see the possibility of some sort of enhanced doors, for example, or enhanced music scoring systems, which will help you create melodies that could potentially understand the way you create music and streamline your music creation process by offering you new melodies that are in your style or perhaps different from your style. And you can just pick up those ideas and use what works best. So in that way, you could have a sort of sparring partner, where AI is just helping you, providing you language, like snippets of music that you can actually use. So this is also something really interesting.
And then there's the third aspect that I believe is the one that I'm the most fascinated by, and that's the idea of taking my musical scripts, and then extending them within, for example, an interactive piece of media, like a video game or a virtual reality application. I would create a musical script and then the AI could extend it and implement that into different emotional settings, depending on what the end user is actually experiencing. And that for me is the most interesting part, and I think it's gonna happen probably, but not right now.
Cari Quoyeser: Yeah, I can certainly see that interest towards AI and AI technology is growing, and as long as those communities are strengthening and growing in numbers, that it's going to be something we're going to see more of in the future anyway.
There seems to be a general feeling in the artist community, in the songwriting community, that AI is going to replace the need for skill, the development of skill. And I'm trying to change my own mind about this, because I have some aversion to AI writing lyrics, personally.
That being said, I have to acknowledge the fact that there's a whole world of opportunity that's open to me. For example, I studied notes very briefly in university, but it's not something I retained necessarily. And having a technology that could take my song and show me what it could be really widens the variety of music I'm able to create.
Valerio Velardo: Yeah, absolutely. And I think also for education purposes as well, right? It’s a very powerful system that you can potentially utilize to make people more aware of the theory that we have behind songs or compositions in general.
And then again, there could be a continuous feedback loop between the stage of learning from the AI, learning from the theory behind the music, as well as inputting new ideas that the AI will pick up and then develop. The work of the composer (at least the traditional composer who sits on the piano and writes all the notes out on a score) sometimes can be dull, right? Because there's a lot of low level details that are not necessarily super interesting.
Cari Quoyeser: And super time-consuming..
Valerio Velardo: And super time-consuming! Imagine you are writing music for a very large orchestra, now you have to write a lot of lines of music for a lot of musicians or groups of musicians. Now, all of a sudden you could potentially focus on things that are higher level, like for example, your overall harmonic direction, the overall gestures that you want.
And then you could have the AI actually filling up all the tiny details for you. That is going to streamline your composition process. It’s going to allow you to go through the same process multiple times because you’re saving a lot of time in the end. So that is going to probably facilitate your understanding of your overall composition process, hopefully making you more creative in the sense that you can explore more alternatives.
And then of course, at the end of the day, it will be you who's in charge of deciding which direction to go in the end. But nonetheless, you would have a sort of sparring partner who could suggest things that you could accept or reject, or initially accept and then edit in order to arrive at the final idea that you had in mind.
Cari Quoyeser: So what do you think is the immediate future for AI music creation and generation versus what's maybe farther down the line?
Valerio Velardo: It’s very difficult to answer that question. I think AI music, it's so vast in a sense, because of course you have the generative part, which is the one that I'm most interested in at this point, but there are other aspects which I think are way farther ahead in their productization. Like for example, a lot of music genre classification systems or music emission classification systems, or of course, the important music recommendation systems which are already out there in services like Spotify.
So I think that is gonna grow quite a lot because we have a huge problem which is filtering music. As music listeners, we are bombarded with a lot of music information every day. So we need to filter out all of the noise and of course, an algorithmic process to that is very important because it's going to help you remove the noise and focus on the things that possibly are the most relevant for you.
So I think that is going to naturally grow and develop. In terms of the generative aspects, or more AI music in the creative sense, I think the first things that are going to be affected quite a lot are production aspects, like automatic mixing, automatic mastering. But then we'll move on to other things like the generation of new sounds. So for example, new synthesizers that use neural audio synthesis to create very crazy and unheard of sounds. So that is a very interesting aspect that I think is going to possibly be adopted relatively soon.
And then I think later on, the possibility of having actual AI music composition systems will follow, which will initially help amateurs or users of social networks. But when the systems mature a little bit, they will arrive at a level where even more experienced musicians and composers could actually benefit from them. So I think that's the sort of trajectory, I don’t know when it's going to happen, if all of these aspects are going to happen, but we'll just wait and see.
Cari Quoyeser: Thank you, very well said Valerio. I do have one final question for you if that's okay.
Valerio Velardo: Sure!
Cari Quoyeser: So, I just want to know how you address the apprehensions of music industry professionals (whether it be musicians or producers) towards AI under the belief that it would eliminate their job down the line?
Valerio Velardo: Here we are within a creative environment, and then AI can only go so far. There's always new things to explore and to discover. And the AI that we have right now, most of the time, just repeats what we know or what we can do. Sometimes decently, sometimes less decently.
But it's unheard of to have an AI that just innovates things by itself. And probably this is going to remain like this for a very, very long time. So in order to have musical innovation and all of that, we need human input. And by the way, we are in an environment that's highly creative and so innovation and new value brought to the users is highly praised and valued in that respect.
I think that good producers, good musicians are going to remain there and AI is going to be just another tool that they'll have available to streamline some of the most tedious aspects. Or just, let's say, get inspired. I always love to make this analogy with chess. So, chess was basically solved back in 1996 or 1997, when Deep Blue defeated the world chess champion Garry Kasparov. But chess is still played today, right? When we have chess software, chess AI that works, that plays at superhuman level.
Nonetheless, we're interested in watching human players playing against each other. And that's because there's an intrinsic human aspect to that endeavor. Chess is a form of science and art at the same time, and something like music that’s even more artistic than chess is always going to be brought back to the human aspects.
We don't just connect with some music. We connect with the music and with the artists who are behind the music, we want to know their stories and where their music came from. So this is something that's going to remain there, regardless of whether AI in the future is going to be able to create music that’s really, really good.
The human aspect is always gonna remain there. So I really don't think there's ever going to be a full replacement of musicians. Of course, we can think for certain types of music which are more functional, like for example, music for advertisements where the quality of the music isn't necessarily the highest, there’s a possibility that composers will use a little bit of that work.
But still the most important music productions which are the ones that have a higher artistic value and are charged with a lot of creative potential behind them, those are going to remain unchanged. AI in that respect is going to only be an amplifier, it's not gonna be a source for a full replacement.
Cari Quoyeser: Wow, that was a lot of information. We hope you feel confident that AI won't be the end of your musical career. Perhaps it can be something you look forward to using yourself.
Thanks so much for tuning into The Mix. If you like this episode or have a topic you're dying for us to cover, head on over to the music Musixmatch Artist Community slack channel and share your thoughts.
For more information on AI and links to other cool resources, like Valerio’s other projects, check out our upcoming AI article on themix.musixmatch.com.
AI is a subject we're really into here at The Mix. This episode is meant to give you a broad overview of what AI music is all about, but the truth is there's so much more to talk about, like virtual pop stars or immersive multi-sensory art projects. Cast your vote for the subject of our next AI episode on The Mix website. The Mix podcast is powered by Musixmatch Pro. Thanks for tuning in. See you next time.
