I’ll post links to recordings once they are available
It’s been about a million years since I said this, but the EuroPython talk recording is now available! And slides are here.
I haven’t had much time for Calliope recently but it’s ticking along, and I’m hoping to get some time this month to tackle ListenBrainz integration for listens and playlists.
From the Listenbrainz /musicbrainz twitter, sounds like more work is being done on their end for recommendations too.
Great talk. I would’ve mentioned that Spotify pays MusicBrainz, though why I’m not sure.
https://wiki.musicbrainz.org/MetaBrainz:Sponsors
I swear they used to be a platinum sponsor like Google. Not sure what changed.
Liked this slide and the forgotten songs idea.
I just released Calliope 4.0, with initial support for Listenbrainz! 
Two new commands & modules are added:
cpe listenbrainz which can export playlists from Listenbrainz (such as your Year in Retrospective playlists)cpe listenbrainz-history which works exactly like cpe lastfm-history, only, with Listenbrainz.Just came across Calliope and am mighty impressed. Any chance you are considering adding Plex support? It would be great to export these smart playlists directly to Plexamp.
Hello! Glad you like it.
I personally use Jellyfin and there’s no special support in Calliope, I have a shell script running periodically on my media server that generates M3U playlists to a location that Jellyfin is configured to search, and from there they appear in Jellyfin.
Here’s a stripped-down version of the script:
#!/bin/bash
PLAYLISTS_PATH=/media/HD1/Playlists/
SCRIPT_DIR=$(dirname $0)
echo "Cleanup playlists more than 2 weeks old"
find /media/HD1/Playlists/ -name *.m3u -not -mtime -14 -delete
echo "Query all songs"
cpe tracker tracks > /tmp/tracks.cpe
echo "Shuffle and select"
cpe shuffle /tmp/tracks.cpe | cpe select --constraint=type:playlist-duration,vmin:1hours,vmax:1hours - > /tmp/daily-shuffle.cpe
title=$(date +'Daily shuffle %F')
output="$PLAYLISTS_PATH/$title.m3u"
echo "Write $output"
python3 $SCRIPT_DIR/cpe-relative-paths.py < /tmp/daily-shuffle.cpe | cpe export -f m3u -t "$title" - > "$output"
That generates a “Daily shuffle” playlist with a random selection of 60 minutes of music.
Unfortunately, Plex does not automatically import m3u files. But let me explore - there should be a way.
More examples because I didn’t find the OP or other examples clear. Note the -
cpe beets tracks | cpe shuffle --count 3 - | cpe export -f m3u --title asdf - >myfile.m3u
Next up: resolving these paths between my linux and windows machine.
edit: I enabled asciify_paths to reduce issues with m3u in foobar.
end to end example on alpine linux/beets docker container:
cpe beets tracks | cpe shuffle --count 3 - | cpe export -f jspf -t asdf - >/blk/black/mysys/cpe/jspf/my.jspf
./jspf_to_m3u.sh
python3 m3u_fix.py
for filename in /blk/black/mysys/cpe/jspf/*.jspf; do
mybase="${filename##*/}"
without_ext=${mybase%.jspf*} # remove suffix starting with ".jspf"
echo $without_ext
# TODO?: import the original playlist name?
cpe import $filename | cpe export -f m3u -t zxcv - >/blk/black/mysys/cpe/m3u-og/$without_ext.m3u
done
import os
from urllib.parse import unquote
# where raw exports from callipe are stored - possibly after `cpe import`ing a jspf file.
# it feels cleaner to have these stored rather than transparently deleted.
# the original jspfs are kept because they [appear to] allow full imports into cpe and mapping back to a beets library.
# that is, they preserve enough metadata to allow a kind of "session resume", which m3u does not.
m3u_path = r'/blk/black/mysys/cpe/m3u-og/'
def fix_line(line):
old = r'file:///blk/'
new = r'\\ODROIDHC4\\' # escaped single \
fixed_start = line.replace(old, new)
fixslash = fixed_start.replace('/', '\\') # escaped single \
# fix the percent encode / url encoded spaces (%20) for foobar2000 - no clue what players need what
nopercent = unquote(fixslash)
return(nopercent)
def fix_m3u(folder, myfile):
out_name = myfile
fullpath = os.path.join(folder, myfile)
out_lines = ''
with open(fullpath, 'r') as ff:
for line in ff:
# print(line)
new_line = fix_line(line)
out_lines = out_lines + new_line
print(out_lines)
out_dir = r'/blk/black/mysys/cpe/m3u-win'
out_path = os.path.join(out_dir, out_name)
with open(out_path, 'w') as ff:
ff.write(out_lines)
for root, dirs, files in os.walk(m3u_path):
for ff in files:
fix_m3u(root, ff)
for name in dirs:
pass
I’m trying to get my feet wet with “all beets albums, ordered by $added”. I think beets/plugins can do this already but my idea is to build more functionality on top of it. I got this far:
beet ls --format '$added-$album-$track-$title'
Should I be looking in constraints.py ? calliope/select/constraints.py · main · Sam Thursfield / calliope · GitLab
beets? calliope/beets/__init__.py · main · Sam Thursfield / calliope · GitLab
This looks promising: examples/special-mix/special_mix.py · main · Sam Thursfield / calliope · GitLab
Is this something cpe can do now or do I have to edit the code? Perhaps the keys that are requested from beets?
Right now I miss a sql-like query functionality.
A metaphor like this might make sense:
example:
query.sql
select
genre, artist, year, album, disc, track, plays, rating, days_since_last_play
from
beets
where
genre = 'jazz'
Then moving beyond the sql metaphor:
It could look like this:
cpe select query.sql picklist.py
picklist_helpers.py:
def prefer(candidates, criterion):
# see also: https://gitlab.com/samthursfield/calliope/-/blob/main/calliope/select/localsearch.py#L157
'''do some non-deterministic magic to rank higher by `criterion` but don't just do a simple sort.
for example, 2 is not always greater than 1 in this paradigm. (If I'm understanding the calliope docs.)'''
picklist.py (custom per ‘picklist’, I may have one for jazz and one for running and one for parties, etc.)
def picker(table):
'''Choose tracks from table based on below constraints.'''
# each line in picklist is its own distinct set of constraints, unrelated to other lines
# each line is attempted to be solved until that line is fulfilled or the overall constraint (total runtime) is met
picklist = '''n=4;rating>=9;runtime<20minutes;shuffle_albums;
n=10;n_albumartists=1;last_played>7;prefer(last_played+);shuffle_albums''' # only 1 album per artist on this line
output_playlist = []
while sum(output_playlist['runtime']) < '60minutes':
for list_item in picklist:
new_items = parse_list_item(list_item)
# allow for the while-loop to break partway through, if constraint is met
for item in new_items:
output_playlist.append(new_items)
Or maybe the picklist line format could be something like count, sort, select.
n=10; sort=shuffle(albums); select= rating>=9, runtime<20minutes
This looks like a set of notes more than questions where i can concretely help with everything, anyway its great to see someone diving into the code 
I’m trying to get my feet wet with “all beets albums, ordered by $added”.
It might indeed be possible to do this with Beets commandline.
certainly using beet ls -a added+ you can get the list of albums. If you want to avoid -a so you get a list of albums and all their tracks in a single list, it might be hard to get the right sort order.
So cpe beets albums runs in two stages, first querying the albums with ls -a, then running beet export for each album to get the tracklist.
(as a side note, this can be slow for large exports as it runs beet many times and the program can be slow to startup. adding a beet export-many command would help speed up cpe beets)
Looking at the code, cpe beets albums doesn’t have any --sort-order option, it would be nice to add that, although it should already to run cpe beets albums added+.
Right now I miss a sql-like query functionality.
Early designs of Calliope had a single giant database that would store all possible info about your music, and processing would be done using a query language like SQL. A “big SQL database” design didn’t work for me as (1) it’s good if you can design your schema up front, but its not so easy to prototype and evolve new features (2) SQL is readable for short queries, but becomes very unreadable for large complex queries.
So the key insight of this version, is to have many small tools instead of one big one. The json-lines based playlist format is designed as an interface between different components. But it’s not necessarily the best way to process data inside a component.
In many cases, importing/exporting from an SQL database in the component is a good option. For example, listenbrainz-history and lastfm-history both use an SQLite database for storage, and run SQL queries internally to generate the playlists that are output.
There isn’t currently a cpe sql command that could import a playlist into an SQL database and run SQL-like queries, but it would be valid to add one. (There is cpe tracker, which if you happen to be on Linux lets you use Tracker SPARQL database).
Another thing I am playing with myself is Nushell. It’s a shell with first-class support for tables and dataframes, which plays very nicely with the design of Calliope. you can get data out of cpe and into nu using from json --objects.
Here’s a quick nushell example to show the top lastfm artists I’ve listed to <100 times:
cpe lastfm-history --user ssam artists | \
from json --objects | \
sort-by "lastfm.playcount" -r | \
where "lastfm.playcount" < 100 | \
first 10
Worth noting that nushell is in active development and things sometimes don’t work. But it could be useful for prototyping and scripting in a way that feels natural.
This looks promising: examples/special-mix/special_mix.py · main · Sam Thursfield / calliope · GitLab
That’s what i am currently using to generate my playlists
More smart playlist work, from Rivers Cuomo (Weezer)
Some academic work on recommendations:
Some related talks in the sidebar. Not clear if Spotify still does them.
Oooh thanks for these!
Not sure if I mentioned but this 2008 paper inspired the cpe select command: https://www.researchgate.net/publication/223327847_Music_playlist_generation_by_adapted_simulated_annealing
Are any folk here local to Europe? I will be doing a short talk on Calliope in the Python room at FOSDEM 2023, could be a nice opportunity to chat about DIY music recommendation
# install dependencies to compile C code on ARM/alpine
# all but git are used for calliope and its dependency on splitstream
apk add gcc musl-dev python3-dev libc-dev
python3 -m pip install calliope-music m3u8
Can I sort tracks? Maybe with constraints? My tracks are coming back split by artist in the same album. Ex. track 18/30 comes at the end of the album because it’s “Album Artist & Other Artist”.
cpe beets tracks | cpe export -f jspf -t asdf - >full.jspf
Edit: I can’t reproduce on my latest run. No idea why it was out of order before. Is beet ls deterministic?
@samthursfield In addition to the rule-based playlists (e.g., not played in 2 weeks, etc.), is it possible to have a simple content-based recommendation? For example, we could have a playlist of songs that the user may like given their prior history or a playlist of songs based on a seed song, etc.
By “content-based recommendation”, i’m understanding that first you would need a way to analyze each song’s content. Calliope itself doesn’t wrap any tool for analyzing song content, it’s not something i’ve looked into myself. I know Acousticbrainz did some work on this but that project has been retired. Anyway, once you have this info, you would store this information in some way (in Beets, for example). Then you could develop constraints using this new metadata.
If you’re interested in looking at this, I think the first step is to work out what tools are available for analyzing the audio. (Or as a test you could use the descriptors provided by Spotify’s API - that of course means that Spotify API rate limiting applies but it could be useful for a proof of concept)