Calliope - antisocial music recommendations

When I spotted ‘antisocial music recommendations’ in the roadmap I was excited. I have been slowly laying foundations to do exactly that in a research project I call Calliope. Progress is slow but steady. I haven’t shared the project anywhere yet but it ties in with Beets, so let me try and fix that by giving an overview of the design.

Goals

The main goal of Calliope is to generate interesting playlists. It doesn’t do much of that yet.

The other goal of Calliope is to be a sustainable project, currently meaning it can be developed and maintained in small blocks of maybe 2 hours a week. (You might also think of it as small technology).

Design

Concretely, Calliope is a suite of commandline tools which operate on playlists. It’s written in Python so you can work with it in a Python shell like iPython, or in any UNIX-style shell.

The best playlist format is XSPF, but there aren’t good UNIX shell tools for working with XML. Inspired by jq, I combined XSPF with JSON Lines to define the ‘Calliope playlist format’. There’s a small example in the linked documentation.

The general operation of a recommender is this:

  1. Get source data
  2. Process the data, based on some configurable parameters
  3. Output a playlist

Calliope provides tools for each of those things. For getting source data, you can cpe import an existing playlist, use cpe lastfm-history to pull from last.fm, cpe spotify to get various things from Spotify, etc. (Note that cpe spotify requires you to register a Spotify API key). I’m sure you can think of more.

The magic happens in stage two. A simple playlist processing example is cpe shuffle which shuffles its input. Let’s say you want to listen to random tracks from your Beets library – once cpe beets is ready, you could do this:

cpe beets tracks | cpe shuffle | cpe export > playlist.xspf

Then you would open playlist.xspf in a media player and away you go. (Try to ignore the fact that your media player already has this functionality built in).

Plans

I’m looking at a more interesting use case of reminding you about music you didn’t listen to for a while. The cpe lastfm-history module fetches your listen history from last.fm (a slow process) and stores it in an SQLite database. This lets us do interesting queries. My idea is to score the artists out of 10 on different axes, for example when you first listened, how much you have listened, when you last listened, when they last released music, and how popular they are overall. This will find its way to a new cpe remind command which you might call like this:

# Music I've forgotten about but other people haven't
cpe remind --forgotten 7.0 --popular 10

# Music I discovered a long time ago
cpe remind --fresh 1.0

etc.

You can see a list of currently existing commands at: https://calliope-music.readthedocs.io/en/latest/reference.html

The code itself is here: https://gitlab.com/samthursfield/calliope/

I have more ideas for things we could do, more ideas than time to try them all as is normal :). In particular I like Spotify’s ‘artist radio’ and ‘track radio’ feature and I think Calliope could do something similar. I’m sure you have more ideas as well, so please let me know… here, or I’m also in the #beets IRC room as ssam2.

3 Likes

Sounds intriguing. I’ve always wished beets could generate coherent playlists. For now, I’m using Plexamp for this, but a FOSS solution would be most welcome. Looking forward to the results of your project.

Right up my alley. Is the beets IRC in use still? I thought it transitioned to Gitter, which has now been deprecated for this + a private dev platform.

There are 89 of us in there at the time of writing… but you’re correct that it’s not used for core dev discussion.

Wow! This is ridiculously cool! I’m super excited about this effort and will definitely keep an eye on it.

While the JSON Lines format looks quite practical, I can’t help but also point out that there is also “JSPF,” a direct translation of XSPF into JSON:

I would also be interested in whether ListenBrainz ever becomes a viable alternative to Last.fm:

1 Like

Glad you like the project! I’ll post updates here if and when I make progress.

It’d be nice to use that, the problem is that if we have a very big playlist we have to load everything into RAM and then serialize it as one big JSON object. We can import/export to JSPF of course.

You’re right about Listenbrainz, we can never be sure when Last.fm might make it more difficult to get our own data. I should switch over myself…

1 Like

As a Last.fm donor - their entire site is stagnant. Listenbrainz posts cool stuff to their twitter all the time and has a growing dev community. I wish it didn’t require licensing your plays as public domain, but I’ll probably switch.


Question: how does this tie into beets and how do you see it tying into beets in the future? my main concern is hitching myself to a program that doesn’t integrate into beets as much as i want. I have reasonable Python knowledge and some knowledge of beets so I might be able to bolt something together if it comes down to it.

specifically, I want to algorithmically-generate “playlists” that I can use as a “pick list” for my phone. I need to be able to copy a subset of my library to my phone, most likely compressing along the way with a plugin like convert or alternatives.

Question: how does this tie into beets and how do you see it tying into beets in the future?

Each service is a separate command, so all Beets integration lives in the cpe beets command. I just merged an initial version of this. Currently it can output tracks and artists from Beets as a Calliope playlist. It basically wraps beet export plugin, so…

  • it will work against Beets master only (needs --format=jsonlines option)
  • you need to enable the ‘export’ plugin
  • you should apply https://github.com/beetbox/beets/pull/3762 which makes beet export significantly faster

specifically, I want to algorithmically-generate “playlists” that I can use as a “pick list” for my phone.

The basics of this are already possible. Here’s a command that picks 20 random tracks from your Beets library, transcodes to MP3 using GStreamer, and copies to /tmp/my-phone:

cpe beets tracks|cpe shuffle -|head -n 20|cpe sync -t /tmp/my-phone --allow-formats=mp3 -

You can add --dry-run to the last command, which will print the relevant commands instead of running them. For example…

rsync --archive /home/sam/External/Music/The Bird and the Bee - Recreational Love [2015]/09 We’re Coming to You.mp3 /tmp/my-phone/09 We’re Coming to You.mp3
rsync --archive /home/sam/External/Music/La Tarrancha - Ö 3 3 [2009]/09 La palabra Llibertá.mp3 /tmp/my-phone/09 La palabra Llibertá.mp3
gst-launch-1.0 -t filesrc location="/home/sam/External/Music/Alternative Electronic Volume 1 (Music by University of Huddersfield students) [0000]/01 Richard Hearn - I Am Rubber, You Are Glue. This Is Plastic And Metal.flac" ! decodebin ! audioconvert ! lamemp3enc quality=0 ! id3mux ! filesink location="/tmp/my-phone/01 Richard Hearn - I Am Rubber, You Are Glue. This Is Plastic And Metal.mp3"
...

It’s good that you know Python, as I’m sure that you will reach the limits of what’s currently possible fairly quickly… :slight_smile: