Search⌘ K
AI Features
Home
Courses
Vector Databases: From Embeddings to Applications

Generating Embeddings for Audio Files and Metadata of Songs

Explore how to generate embeddings for both audio files and song metadata using BERT and OpenL3 deep learning models. This lesson guides you through converting song attributes to text, applying special tokens, and extracting audio embeddings, preparing you to build a vector database for music recommendations.

We'll cover the following...

Metadata embeddings with BERT embedding model

To use BERT for generating metadata embedding, we need to convert the tabular metadata information (attributes and values) of a song into a string, which we call the textual_description of the song. For example:

"The song Infinity Edge has a danceability of 0.528, energy of 0.847, loudness of -4.741, speechiness of 0.0307, acousticness of 0.00674, instrumentalness of 0.814, liveness of 0.12, valence of 0.389, tempo of 143.997."

To do this, we read the CSV file containing metadata about songs and create descriptive text for each song using its attributes and values. We add this text to the DataFrame and then save the updated DataFrame back to the CSV file.

Python 3.10.4
import pandas as pd
metadata_file_path="/content/drive/MyDrive/vector-databases-course/music-recommendation-system/dataset/reduced_80_fer2013_music_dataset_with_youtube_URLS.csv"
# Loading metadata into DataFrame
metadata_df = pd.read_csv(metadata_file_path)
# Extracting relevant numeric attribute and creating a textual description using them
metadata_df['textual_description'] = metadata_df.apply(
    lambda row: f"The song {row['song_name']} has a danceability of {row['danceability']}, "
                f"energy of {row['energy']}, "
                f"loudness of {row['loudness']}, "
                f"speechiness of {row['speechiness']}, "
                f"acousticness of {row['acousticness']}, "
                f"instrumentalness of {row['instrumentalness']}, "
                f"liveness of {row['liveness']}, "
                f"valence of {row['valence']}, "
                f"tempo of {row['tempo']}",
    axis=1
)
# Saving the updated DataFrame with the textual descriptions for each song back to the CSV file
metadata_df.to_csv(metadata_file_path, index=False)
print("The 'textual_description' column has been added to the CSV file.")

The textual_description column added to the metadata_song.csv file is shown in the illustration below:

textual_description column added to the metadata_songs.csv file
textual_description column added to the metadata_songs.csv file
...