Calliope - antisocial music recommendations

When I spotted ‘antisocial music recommendations’ in the roadmap I was excited. I have been slowly laying foundations to do exactly that in a research project I call Calliope. Progress is slow but steady. I haven’t shared the project anywhere yet but it ties in with Beets, so let me try and fix that by giving an overview of the design.

Goals

The main goal of Calliope is to generate interesting playlists. It doesn’t do much of that yet.

The other goal of Calliope is to be a sustainable project, currently meaning it can be developed and maintained in small blocks of maybe 2 hours a week. (You might also think of it as small technology).

Design

Concretely, Calliope is a suite of commandline tools which operate on playlists. It’s written in Python so you can work with it in a Python shell like iPython, or in any UNIX-style shell.

The best playlist format is XSPF, but there aren’t good UNIX shell tools for working with XML. Inspired by jq, I combined XSPF with JSON Lines to define the ‘Calliope playlist format’. There’s a small example in the linked documentation.

The general operation of a recommender is this:

  1. Get source data
  2. Process the data, based on some configurable parameters
  3. Output a playlist

Calliope provides tools for each of those things. For getting source data, you can cpe import an existing playlist, use cpe lastfm-history to pull from last.fm, cpe spotify to get various things from Spotify, etc. (Note that cpe spotify requires you to register a Spotify API key). I’m sure you can think of more.

The magic happens in stage two. A simple playlist processing example is cpe shuffle which shuffles its input. Let’s say you want to listen to random tracks from your Beets library – once cpe beets is ready, you could do this:

cpe beets tracks | cpe shuffle | cpe export > playlist.xspf

Then you would open playlist.xspf in a media player and away you go. (Try to ignore the fact that your media player already has this functionality built in).

Plans

I’m looking at a more interesting use case of reminding you about music you didn’t listen to for a while. The cpe lastfm-history module fetches your listen history from last.fm (a slow process) and stores it in an SQLite database. This lets us do interesting queries. My idea is to score the artists out of 10 on different axes, for example when you first listened, how much you have listened, when you last listened, when they last released music, and how popular they are overall. This will find its way to a new cpe remind command which you might call like this:

# Music I've forgotten about but other people haven't
cpe remind --forgotten 7.0 --popular 10

# Music I discovered a long time ago
cpe remind --fresh 1.0

etc.

You can see a list of currently existing commands at: https://calliope-music.readthedocs.io/en/latest/reference.html

The code itself is here: https://gitlab.com/samthursfield/calliope/

I have more ideas for things we could do, more ideas than time to try them all as is normal :). In particular I like Spotify’s ‘artist radio’ and ‘track radio’ feature and I think Calliope could do something similar. I’m sure you have more ideas as well, so please let me know… here, or I’m also in the #beets IRC room as ssam2.

3 Likes

Sounds intriguing. I’ve always wished beets could generate coherent playlists. For now, I’m using Plexamp for this, but a FOSS solution would be most welcome. Looking forward to the results of your project.

Right up my alley. Is the beets IRC in use still? I thought it transitioned to Gitter, which has now been deprecated for this + a private dev platform.

There are 89 of us in there at the time of writing… but you’re correct that it’s not used for core dev discussion.

Wow! This is ridiculously cool! I’m super excited about this effort and will definitely keep an eye on it.

While the JSON Lines format looks quite practical, I can’t help but also point out that there is also “JSPF,” a direct translation of XSPF into JSON:

I would also be interested in whether ListenBrainz ever becomes a viable alternative to Last.fm:

1 Like

Glad you like the project! I’ll post updates here if and when I make progress.

It’d be nice to use that, the problem is that if we have a very big playlist we have to load everything into RAM and then serialize it as one big JSON object. We can import/export to JSPF of course.

You’re right about Listenbrainz, we can never be sure when Last.fm might make it more difficult to get our own data. I should switch over myself…

1 Like

As a Last.fm donor - their entire site is stagnant. Listenbrainz posts cool stuff to their twitter all the time and has a growing dev community. I wish it didn’t require licensing your plays as public domain, but I’ll probably switch.


Question: how does this tie into beets and how do you see it tying into beets in the future? my main concern is hitching myself to a program that doesn’t integrate into beets as much as i want. I have reasonable Python knowledge and some knowledge of beets so I might be able to bolt something together if it comes down to it.

specifically, I want to algorithmically-generate “playlists” that I can use as a “pick list” for my phone. I need to be able to copy a subset of my library to my phone, most likely compressing along the way with a plugin like convert or alternatives.

Question: how does this tie into beets and how do you see it tying into beets in the future?

Each service is a separate command, so all Beets integration lives in the cpe beets command. I just merged an initial version of this. Currently it can output tracks and artists from Beets as a Calliope playlist. It basically wraps beet export plugin, so…

specifically, I want to algorithmically-generate “playlists” that I can use as a “pick list” for my phone.

The basics of this are already possible. Here’s a command that picks 20 random tracks from your Beets library, transcodes to MP3 using GStreamer, and copies to /tmp/my-phone:

cpe beets tracks|cpe shuffle -|head -n 20|cpe sync -t /tmp/my-phone --allow-formats=mp3 -

You can add --dry-run to the last command, which will print the relevant commands instead of running them. For example…

rsync --archive /home/sam/External/Music/The Bird and the Bee - Recreational Love [2015]/09 We’re Coming to You.mp3 /tmp/my-phone/09 We’re Coming to You.mp3
rsync --archive /home/sam/External/Music/La Tarrancha - Ö 3 3 [2009]/09 La palabra Llibertá.mp3 /tmp/my-phone/09 La palabra Llibertá.mp3
gst-launch-1.0 -t filesrc location="/home/sam/External/Music/Alternative Electronic Volume 1 (Music by University of Huddersfield students) [0000]/01 Richard Hearn - I Am Rubber, You Are Glue. This Is Plastic And Metal.flac" ! decodebin ! audioconvert ! lamemp3enc quality=0 ! id3mux ! filesink location="/tmp/my-phone/01 Richard Hearn - I Am Rubber, You Are Glue. This Is Plastic And Metal.mp3"
...

It’s good that you know Python, as I’m sure that you will reach the limits of what’s currently possible fairly quickly… :slight_smile:

I’ve recently been thinking more about playlist generation, and I found a couple of academic papers on the subject:

Both of these talk about using stochastic optimisation to produce playlists that satisfy a set of user-supplied constraints. This sounds like exactly what I want the core of Calliope to do – prepare a set of factors about the music, and then let the user choose which factors to prioritize. I’m playing with the Python simpleai module to see if this is enough to produce good results :slight_smile:

2 Likes

I’ve been working on this quite a lot recently. In particular the documention is a lot more polished, including various examples. I also wrote a blog post.

Has anyone tried it out yet? I’m wondering about publishing to PyPI now that the design is a bit more stable. I realise the current install method is a bit too much effort.

1 Like

Yay! If you do publish this, I would love to shout it from the rooftops on Twitter, etc.

That’s good to know… I will definitely let you know once its there :slight_smile:

Just to say, Calliope sounds very much like what I intended to write - a super powerful smartplaylist plugin.

I started on this journey a few days ago (though like you have wanted a solution to playlists for many years) and wrote a modified version of lastimport to sync my last.fm play dates info with beets as well as last.fm loved tracks: https://github.com/pruperting/lastupdate

What I really want to do is use the Acousticbrainz data as well, to make playlists based on song features (happy, sad, danceable etc) and I’ve been messing around with https://github.com/adamjakab/BeetsPluginXtractor to start to categorize the music that isn’t in Acousticbrainz.

I’ll be messing around with Calliope with interest over the next few nights and will follow it with interest.

Great! If there’s anything I can help with, let me know. I hope Calliope saves you some time in the ‘plumbing’ layers, most of what’s there so far is about getting data from different sources into a common format. An Acousticbrainz module would be welcome.

I have made a few changes to Calliope recently, in particular adding a bandcamp module and improving the way the diff command works. I wrote a blog about I use the tool to check Bandcamp for albums that I forgot to download, you can find it here!

Work progresses on this project, at a slow steady pace.

I am doing a couple of talks about Calliope. Once at GUADEC 2021, at 17.40 UTC today. You’ll be able to see it on the livestream here. And another (very similar) talk at EuroPython 2021, on Friday 30th July at 13.45UTC.

I’ll post links to recordings once they are available :slight_smile:

2 Likes

That’s super awesome! Can’t wait to see them. :smiley:

Hi @prupert,
you can use the standalone Essentia music extractor to produce the same descriptors as being used in AcousticBrainz (they are using Essentia 2.1 beta2). Let me know if you need more details.

Also, there are more up-to-date machine learning models in Essentia (same classifiers, better accuracy), if you want to go for analysis locally.

I am very interested in seeing/collaborating on smart playlists using Essentia features.

1 Like

I had success with the Beets Extractor plugin and pretty much analysed all my music. It took a week or so (running it on a small arm device). I then used the smartplaylist plugin to create playlists based on some of the fields like happy, sad, danceable, male / female vocalist, music type and key.

It is kinda hit and miss I find, especially when it comes to music type and gender. But it does result in a very varied set of playlists!

Predictions of those SVM classifiers are not very reliable as we found out after validation on external ground truth. That is, the accuracy drops when evaluating on completely new music from a dataset not used for training:

image

(source: https://arxiv.org/pdf/2003.07393.pdf)

And some of the classifiers are more usable than others. We created better classifier models since then.

1 Like