Calliope - antisocial music recommendations

There are 89 of us in there at the time of writing… but you’re correct that it’s not used for core dev discussion.

Wow! This is ridiculously cool! I’m super excited about this effort and will definitely keep an eye on it.

While the JSON Lines format looks quite practical, I can’t help but also point out that there is also “JSPF,” a direct translation of XSPF into JSON:

I would also be interested in whether ListenBrainz ever becomes a viable alternative to Last.fm:

1 Like

Glad you like the project! I’ll post updates here if and when I make progress.

It’d be nice to use that, the problem is that if we have a very big playlist we have to load everything into RAM and then serialize it as one big JSON object. We can import/export to JSPF of course.

You’re right about Listenbrainz, we can never be sure when Last.fm might make it more difficult to get our own data. I should switch over myself…

1 Like

As a Last.fm donor - their entire site is stagnant. Listenbrainz posts cool stuff to their twitter all the time and has a growing dev community. I wish it didn’t require licensing your plays as public domain, but I’ll probably switch.


Question: how does this tie into beets and how do you see it tying into beets in the future? my main concern is hitching myself to a program that doesn’t integrate into beets as much as i want. I have reasonable Python knowledge and some knowledge of beets so I might be able to bolt something together if it comes down to it.

specifically, I want to algorithmically-generate “playlists” that I can use as a “pick list” for my phone. I need to be able to copy a subset of my library to my phone, most likely compressing along the way with a plugin like convert or alternatives.

Question: how does this tie into beets and how do you see it tying into beets in the future?

Each service is a separate command, so all Beets integration lives in the cpe beets command. I just merged an initial version of this. Currently it can output tracks and artists from Beets as a Calliope playlist. It basically wraps beet export plugin, so…

  • it will work against Beets master only (needs --format=jsonlines option)
  • you need to enable the ‘export’ plugin
  • you should apply https://github.com/beetbox/beets/pull/3762 which makes beet export significantly faster

specifically, I want to algorithmically-generate “playlists” that I can use as a “pick list” for my phone.

The basics of this are already possible. Here’s a command that picks 20 random tracks from your Beets library, transcodes to MP3 using GStreamer, and copies to /tmp/my-phone:

cpe beets tracks|cpe shuffle -|head -n 20|cpe sync -t /tmp/my-phone --allow-formats=mp3 -

You can add --dry-run to the last command, which will print the relevant commands instead of running them. For example…

rsync --archive /home/sam/External/Music/The Bird and the Bee - Recreational Love [2015]/09 We’re Coming to You.mp3 /tmp/my-phone/09 We’re Coming to You.mp3
rsync --archive /home/sam/External/Music/La Tarrancha - Ö 3 3 [2009]/09 La palabra Llibertá.mp3 /tmp/my-phone/09 La palabra Llibertá.mp3
gst-launch-1.0 -t filesrc location="/home/sam/External/Music/Alternative Electronic Volume 1 (Music by University of Huddersfield students) [0000]/01 Richard Hearn - I Am Rubber, You Are Glue. This Is Plastic And Metal.flac" ! decodebin ! audioconvert ! lamemp3enc quality=0 ! id3mux ! filesink location="/tmp/my-phone/01 Richard Hearn - I Am Rubber, You Are Glue. This Is Plastic And Metal.mp3"
...

It’s good that you know Python, as I’m sure that you will reach the limits of what’s currently possible fairly quickly… :slight_smile:

I’ve recently been thinking more about playlist generation, and I found a couple of academic papers on the subject:

Both of these talk about using stochastic optimisation to produce playlists that satisfy a set of user-supplied constraints. This sounds like exactly what I want the core of Calliope to do – prepare a set of factors about the music, and then let the user choose which factors to prioritize. I’m playing with the Python simpleai module to see if this is enough to produce good results :slight_smile:

2 Likes

I’ve been working on this quite a lot recently. In particular the documention is a lot more polished, including various examples. I also wrote a blog post.

Has anyone tried it out yet? I’m wondering about publishing to PyPI now that the design is a bit more stable. I realise the current install method is a bit too much effort.

1 Like

Yay! If you do publish this, I would love to shout it from the rooftops on Twitter, etc.

That’s good to know… I will definitely let you know once its there :slight_smile:

Just to say, Calliope sounds very much like what I intended to write - a super powerful smartplaylist plugin.

I started on this journey a few days ago (though like you have wanted a solution to playlists for many years) and wrote a modified version of lastimport to sync my last.fm play dates info with beets as well as last.fm loved tracks: https://github.com/pruperting/lastupdate

What I really want to do is use the Acousticbrainz data as well, to make playlists based on song features (happy, sad, danceable etc) and I’ve been messing around with https://github.com/adamjakab/BeetsPluginXtractor to start to categorize the music that isn’t in Acousticbrainz.

I’ll be messing around with Calliope with interest over the next few nights and will follow it with interest.

Great! If there’s anything I can help with, let me know. I hope Calliope saves you some time in the ‘plumbing’ layers, most of what’s there so far is about getting data from different sources into a common format. An Acousticbrainz module would be welcome.

I have made a few changes to Calliope recently, in particular adding a bandcamp module and improving the way the diff command works. I wrote a blog about I use the tool to check Bandcamp for albums that I forgot to download, you can find it here!

Work progresses on this project, at a slow steady pace.

I am doing a couple of talks about Calliope. Once at GUADEC 2021, at 17.40 UTC today. You’ll be able to see it on the livestream here. And another (very similar) talk at EuroPython 2021, on Friday 30th July at 13.45UTC.

I’ll post links to recordings once they are available :slight_smile:

2 Likes

That’s super awesome! Can’t wait to see them. :smiley:

Hi @prupert,
you can use the standalone Essentia music extractor to produce the same descriptors as being used in AcousticBrainz (they are using Essentia 2.1 beta2). Let me know if you need more details.

Also, there are more up-to-date machine learning models in Essentia (same classifiers, better accuracy), if you want to go for analysis locally.

I am very interested in seeing/collaborating on smart playlists using Essentia features.

I had success with the Beets Extractor plugin and pretty much analysed all my music. It took a week or so (running it on a small arm device). I then used the smartplaylist plugin to create playlists based on some of the fields like happy, sad, danceable, male / female vocalist, music type and key.

It is kinda hit and miss I find, especially when it comes to music type and gender. But it does result in a very varied set of playlists!

Predictions of those SVM classifiers are not very reliable as we found out after validation on external ground truth. That is, the accuracy drops when evaluating on completely new music from a dataset not used for training:

image

(source: https://arxiv.org/pdf/2003.07393.pdf)

And some of the classifiers are more usable than others. We created better classifier models since then.

1 Like

I’ll post links to recordings once they are available

It’s been about a million years since I said this, but the EuroPython talk recording is now available! And slides are here.

I haven’t had much time for Calliope recently but it’s ticking along, and I’m hoping to get some time this month to tackle ListenBrainz integration for listens and playlists.

2 Likes

From the Listenbrainz /musicbrainz twitter, sounds like more work is being done on their end for recommendations too.

Great talk. I would’ve mentioned that Spotify pays MusicBrainz, though why I’m not sure.

https://wiki.musicbrainz.org/MetaBrainz:Sponsors

I swear they used to be a platinum sponsor like Google. Not sure what changed.

Liked this slide and the forgotten songs idea.