Project roadmap?

Development
1

Is there a roadmap of planned features for Beets? There is some mention of new functionality like web UI and attachments in the wiki, but it looks fairly old. I’m not seeing evidence of how work toward these features is being organized, or how these stack up vs other priorities. Perhaps this thread could serve as a conversation about current priorities and feature planning? Also, if the project doesn’t have a roadmap, maybe it should? GitHub Projects?

I’m also interested in the status of Aura, since I have a project or two I’m considering using it for. Would discussing of the Aura specification make sense here?

2 Likes
2

Hello! This is a really great question. I would love to work on a roadmap of some sort—we have never really done one of these, and I think it would be awesome to put some longer-term, strategic thought into what we want to accomplish.

For some project management context, I have for some time been looking for volunteers to help form a “core team” of contributors to guide the direction of the project:

I got a few bites, but not quite enough momentum to really set up the structure for such a team. Maybe it would be good to revisit that effort in the context of a roadmap definition problem.

As far as what should actually go on that roadmap, IMO the most immediate thing we should do is drop Python 2 support. We should release a version (1.5.0) that will be the last version to support both Python 2 & 3. Then we should ship a new version with lots of hacks removed in fairly short order.

Then there are more ambitious projects to think about. I have had three big things on my mind:

  1. AURA. I have been excited about the potential of finishing this API for some time. I think it’s especially urgent in the time of monolithic streaming services like Spotify. There are many good self-hosted music streaming projects out there, but they are all one-offs. They would be stronger if they shared a common infrastructure. Imagine being able to use beets for your library management but wire that up to a separately developed Web-based streamer, another app for desktop streaming on a Mac, and other apps just for mobile streaming on your phone. I have many more thoughts on the potential here, but the key is polishing off the AURA spec, blessing a v1.0, and implementing a full server as an official beets plugin.

  2. Antisocial music recommendation. Also on the topic of the modern era of streaming services: one of the things that everyone seems to love about them is their recommendations—i.e., the help they give the user in deciding what to listen to. Personally, however, I don’t usually like the algorithmic recommendations I get from most services. In conjunction with AURA-based players and ListenBrainz, I would love to investigate less conventional approaches to recommending music to listen to—preferably privacy-preserving ones that don’t depend on aggregating lots of data about users’ listening habits. Things that are based mostly on your own listening, like these:

    • Resurface music I have listened to a lot but not recently. Or explore your music by seeing how your habits evolved over time (what were my top albums from last month, last year, last decade, etc.).
    • Let me enqueue an album to listen to later from my phone when I think of it—even if I’m not listening to music right now.
    • Synthesize a new playlist that lasts an exact number of minutes to help with timeboxing.
    • Even just random selections from my library would be better than alphabetical-by-artist the UI most services use, which makes me listen to the A and Z artists more than anyone else!

    We could create a framework for experimenting with this kind of privacy-preserving music recommendation technique.

  3. Alternative importer interfaces. Abstract out the current terminal interface to the importer so other interfaces can do the same stuff—a GUI interface in the browser, for example.

Anyway, I’m obviously still excited to explore potential paths for the future evolution of beets! I think we’ll want to choose exciting and ambitious new directions that are not currently part of beets’s “core competency” of importing unsorted music and querying it. I’d be really interested in exploring ambitious ideas along these lines.

3 Likes
3

Ah thanks for giving me this context. I can’t say I’m surprised - unfortunately getting enthusiastic and
competent engineers to not only contribute to / manage open source projects consistently is a difficult feat, even for great projects like Beets.

Yes I strongly approve of the premise. Even the self-hosted selection of music servers / streaming music players is populated by paid products that are not built with modularity or interoperability in mind. Many attempt to be one stop shops, much like the far-more-popular hosted equivalents they are trying to emulate. This is not an approach that appeals to me - I like how Beets does one thing well, and I think that it can be built upon in a way that expands its usefulness, beginning to rival what the self-hosted music library/streaming projects are doing, but without all the bloat and baked-in assumptions.

Cool. Here are some more suggestions:

  • What about using Musicbrainz community ratings to surface suggestions? E.g. - you like this artist, so here are some other albums by that artist with a high community rating?
  • What about querying Musicbrainz artists/albums based on intersection of tags? (I haven’t worked with their API much, but I assume there is a way to do this?)
  • What about suggesting other albums from the same record label as one you clearly like?
  • You mentioned suggestions based on my play history - what about also sourcing my Musicbrainz user ratings?
  • What about suggesting songs by artists you like that are getting lots of listens on Listenbrainz, that you yourself have not listened to or rated highly? Not 100% clear if there’s a way to do this with Listenbrainz.

The idea of giving Beets a GUI makes sense, but I also wouldn’t want to see it turn into another one of these one stop shops that conflate the ideas of library management and music consumption.

I actually bring all of this up because I have written a very prototypey music server that reads my Beets db, exposing it as an API mimicking the responses of another (proprietary) self-hosted music server. With this, I’m able to stream my Beets library via existing streaming apps from my phone, and it’s… fine. But the apps are buggy, it’s all built on (IMO) poorly-thought-out proprietary APIs, and I have ambitions to start over using a higher quality and preferably open API standard, making modularity and pluggability a high priority, then eventually building my own cross-platform front-end as well. This is probably my biggest free time project because I need it so badly… but unfortunately even then, I get very little time for it.

Anyway, all of that aside, it sounds like the things you have in mind for Beets more or less line up with the things I want for my project, so I kind of wonder if I should just contribute to Beets instead, or at least help with planning features, particularly those having to do with finishing AURA or expanding interoperability with players.

1 Like
4

This is my uneducated take on a roadmap.

For some project management context, I have for some time been looking for volunteers to help form a “core team” of contributors to guide the direction of the project

I have no idea what drives people to contribute to open source projects. My model is: some number of users are drawn to the project, and some percentage of them have the desire / expertise to contribute. There are probably additional variables like how constructive the community is when a new user makes a suggestion. But, this is getting complicated.

If I’m right, then evangelizing beets, and tweaking features to make them more useful to novices, would lead to more developers on the project. I understand beets more now, but when I started, I was just making documentation suggestions or finding obscure parts of the source code that didn’t make sense. Those suggestions didn’t take much knowledge of beets.

There’s definitely a balancing act here. Beets probably shouldn’t be a kitchen sink, nor should our limited time be wasted on features nobody wants.

I’ll return to my idea of a novice-friendly beets later.

For anyone curious about AURA, here’s its Github page:

“AURA is a design for simple, REST API for music libraries. It’s the glue between music players and libraries of music.”

I would love to help on getting music recommendations and data visualization in beets. I know my contributions have been rare in the past, but I have a lot of background in statistics so I think this project would keep my attention. Some learning and discussion opportunities for me would be how these recommendations are sent to the user. Beets is a command line interface. Should it be a command like beet recs? Should there be a web UI?

I’ll throw in the idea of using Last.FM for listen totals. I see there’s a plugin in beets already, or users could be encouraged to import their data into ListenBrainz.

I wasn’t aware, but MusicBrainz added genre support in the last 2-4 years. Users vote on them, because they are subjective.

https://wiki.musicbrainz.org/Genre

Now, back to the idea of a novice user using beets. Up to this point, I have recommended beets to very few people, and always with the caveat that it has a steep learning curve. The question is: are there big improvements that could be made to the beets user experience with minimal effort? What might a novice user get out of beets? Or is beets just too complicated for a casual music listener, and we shouldn’t even bother?

My pitch, very much welcome to criticism, is this:

“Beets can organize your music library easily and safely. When you want more advanced control, you can fine-tune it to your liking, including:”

(the list from https://beets.io/ )

What does “easily and safely” mean?

Easily

  • The manual importer prompts should be completely avoided for new users.
  • The current beets philosophy is to “do it right, once and for all”. This would be more “do it half right, while still providing value to the user”.
  • More frequent usage of “asis” imports
  • Is it possible to import at the “release group” level rather than needing an exact release? It’s a crapshoot determining which release you have for popular artists.
  • Better autotagger matching
    – enable fromfilename and acoustID by default, and adjust those plugins as needed to accomodate
    – Luckily, a casual user is likely to have more mainstream music tastes, so hopefully Musicbrainz and the acoustID system would have greater success than you would get with some obscure 1980s vinyl.
  • smarter beets importer? Right now an easy way to get beets to choke is an unexpected folder format. Like /music/album1/disc1 and /disc2.
    – instead of going album-by-album, should beets go artist-by-artist? Once it’s through my whole library, it could look at all the Beatles albums and see that I have one called “Past Masters (Disc 1)” and another for “Past Masters (Disc 2)” and then decide to merge them.
  • As the autotagger improves, the user can go back and “autoimport” remaining albums again. Over time they might have few or no albums that the autotagger couldn’t handle. Of course, the manual import process would remain for those who want it.

Safely

  • No files left behind (using either extrafiles or copyartifacts (newer fork)). I haven’t used either of these extensively, but I know the original copyartifacts would hard-crash beets if it tried to manage an album cover that beets itself was already moving.
    – Beets should get everything on import, and pare it down later upon user request.
  • No data left behind. I couldn’t find whether or not beets preserves the original filename and folder path on import. To me this is a free lunch - not a lot of data, but an obvious feature a user might want. It builds confidence in the program knowing I could theoretically roll back anything that happens.
  • No files moved? Once a user’s library and config are perfect, they might want to move their library to one folder. But off the bat, maybe don’t move their files?
  • No freak accidents where files or data get deleted. I have no clue if this is a realistic goal, but some things that might help:
    – more backups of the files beets is working with
    – trash folder rather than insta-deleting files. There’s an existing issue for this.
    – cover autodownload should gracefully interact with existing cover.jpg files the user might have. Perhaps rename the original file to original-cover.jpg. I believe that right now fetchart is eager to overwrite local files with web sourced files.
    – should any plugin have the ability to overwrite a file?
    — Should this be a permission they explicitly invoke? Imagine a config option called delete with options ask and true. This ties into the base beets functionality too. Maybe it makes sense for beets as a whole to follow some kind of rules on deletes as well. Could there be a false option too? What would happen when the program tries to delete but gets false in return? Stop processing that track and move on to the next one?

Last suggestion (this post got really long!). Going along with getting more casual users to use beets, what’s the reward for them? There are some great bullet points on beets.io, but they are geeky (that’s the audience for beets after all!). Maybe it’s worth mentioning surface-level features?

  • Download cover art and lyrics
  • Import your local music files to Spotify playlists. (I took a quick look at doing this within Spotify, but there doesn’t seem to be an option?)
  • Visualize your music habits and get recommendations. (None of this functionality is in beets yet :stuck_out_tongue: )
  • (The rest of the existing list goes here.)
5

Awesome. I do think this would be an awesome thing to collaborate on. In particular, if we can get AURA to a state where it fulfills that need for a higher-quality, open standard for building this sort of tool, hopefully we can all save ourselves a lot of work! We can build servers, clients, and even “translators” to existing/proprietary APIs independently and interchangeably. It need not necessarily be under the “beets umbrella”—as long as beets has a way to speak the right protocols, we can let a thousand flowers bloom as independent, interoperable projects.

6

I like these ideas for generally making beets friendlier and more foolproof. I think it would be great to divide these kinds of goals into two categories: core improvements that are undebatable improvements everyone would want (e.g., add an option to move to trash instead of deleting), and a “higher-level” wrapper for novices that might amount to a different kind of interface (e.g., requiring no prompts by default). The latter might be best to explore in the context of extensible importer UIs—the sort of importer sibling of AURA, which would let anyone build their own interface that harnesses beets’s autotagging power.

7

Hello! I’ve started working on an AURA plugin for beets using flask and taking inspiration from the existing web plugin. Any feedback on what I’ve done so far would be cool.

I haven’t added anything for a couple months (trying to get another project finished), but want to get back onto it soon.

1 Like
8

Cool! It would be great to finish the spec because until then any implementations of it are at risk of being incomplete / out of date / incompatible. I’d love to help get to 1.0 on the AURA spec so that projects like this can really run with it. I will try to find some time to share more thoughts about how to tackle some of the remaining problems.

2 Likes
9

Yes getting to 1.0 would be great. I think it’s not too far off, with the main areas for improvement being:

  • filtering
  • sorting
  • images
  • authentication
  • standard errors

I saw your github issues for searching, sorting and compound filters, so I think you’ve been thinking about it harder than me :slight_smile:

1 Like
10

Haha on that particular day, yes I probably was. I’m still noodling on the filtering problem…

1 Like
11

Is there a plan for when 1.5.0 will be released?

12

One thing I think would be helpful especially for newcomers would be to have some coverarts from the get-go. I think many people identify their albums by their covers (maybe I’m old-fashioned). So when there is a proposal for an album, some way to display the coverart next to it (as soon as it exists, but for most there is either one on MB or one on amazon/discogs). I know the terminal has a possibility of displaying images, as ranger does it.
Another point that I think is making things complicated for newbies is the distinction MB makes between releases and release groups. In most cases a release group contains different editions of the same release and having to choose between them could be seen as needlessly complicated, also they end up having the same name and tracklist in many cases, the only difference being the year, catalognumber, barcode (things a newcomer wouldn’t necessary care about) and the coverart (here we are again). Also, duplicates can appear because one imports twice the same stuff and it ends up in the same release group but not the same release, duplicates which are tricky to find if you don’t know what you’re doing. I don’t say there is no point in having both release groups and releases, but I believe it complicates matters, and maybe suggestions should propose release groups and let then choose the exact release in the release group.
Different releases in a release group can have different tracklists because one in on 4 vinyls and the other on 2 CDs but the actual recordings and their ordering are the same.

13

Not sure if this is also planned, but I think it’s worth dropping support for python 3.4 as well with this release since it is no longer supported upstream by python.

14

Yeah, I think that’s right.

1 Like
15

It’s worth noting that Debian has deprecated mutagen-1.40.0 in testing so beets is currently unusable there since that was the last version that supported python 2.

16

Ah, one idea I have that would make beets a lot more useful: If I could use all the tags beets provides to be displayed somewhere when I play my music, or even be able to sort by them, that would be great! I’m using Clementine as a music player, but for example it doesn’t allow to sort by albumartist_sort, just by albumartist and when I play it I would love to be able to see all these tags somewhere (especially if the performers are getting fetched by beets). I don’t think that this is something that is to be done on beets’ side, but to try to find come compatibility with other music library systems that actually play the music could be great (or get some crude front-end to plug beets into some music player). As far as I get it, all these libraries have some SQL library where they store all the data about the recordings they have, making them communicate somehow could be a good idea.

17

Are there any particular bugs/features that are blocking the 1.5.0 release? I’d like to help with them if I can. +1 for Github Projects; it could be helpful for categorizing the “we want this done for the 1.5.0 release” and “this would be cool at some point” type issues.

18

This may be outside the intended scope of this thread (and maybe even the Beets project in general) but my focus is on the media player UI - and since Beets has a web UI, why not make it awesome?

I’d like to see something that looks more like a sophisticated, glossy magazine, put together by very talented graphic artists - rather than a “media player app”. If I was put in charge of designing a new UI, it would look something like this…

image
image1095×802 221 KB

(This may look somewhat familiar to anyone who has used Roon - which is very, very cool and beautiful but rather pricey. This is my take on the Roon UI, tweaked for cleanliness and increased functionality.)
The screen would be split (approximately) in half, with the main “control bar” dividing the two halves of the screen. The control bar could be dragged up or down to change the size of the upper/lower sections.

The upper section would consist of a nice piece of fan art as the background - swipe left/right to change the image. (Clickable arrows would be displayed if you’re on a traditional computer but I’ll focus on a tablet interface for the rest of this.) Below that, the artist bio - again, swipe left/right to switch to a different bit of text (an interview,allmusic.com artist overview, etc.)

The lower section, from left to right, would have a “feature text” section about the album. Swipe up/down to see different text about the album. In the center, a nice shot of the album cover, and to the right of that, a track list (with or without track duration times - a user preference). Tap any track for a small popup to start playing the track (with buttons for selecting the source - a disc icon if the track is in the library, Pandora, Spotify, Tidal, iTunes, etc.), favorite it, add it to the queue, add it to a playlist, etc. Swipe left/right on the bottom section to switch from album info to album personnel (including producer and studio info), the artist’s discography, awards, related artists, and a genre tag cloud.

A lot of this content would have to be contributed by the user community. The thought bubble in the control bar is for activating the social media controls, allowing the user community to add images, text (with attribution), user reviews, etc. Once the thought bubble is tapped/clicked, each of the elements on the screen would have a little overlay - thumbs up if you like it, thumbs down if you don’t, and a flag in case some moron tags a Miles Davis album with the Classic Rock genre. (It’s bound to happen and I suggest a three strikes rule – if it’s flagged by 3 separate users, it goes away without moderation.) Music lovers tend to be passionate about their music, so I think people would be quick to contribute a variety of content – including information regarding non-commercial recordings. Over time, I’m confident that this would grow into something pretty robust and extraordinary.

19

This is a really cool concept, @DonnieFontaine! With the revitalized effort to implement the new AURA protocol for beets:

It seems like a great time to be building new web-based frontends. The nice thing about the new plugin is that it doesn’t bake in any particular client—so it should be conceptually easy to provide new ones like this separately. Maybe it’s worth looking into making this the first web-based AURA client?

1 Like
20

Thanks, Adrian! Seems like the best (and simplest) way to accomplish this is to define the building blocks of the web UI (and the individual bits of content that each block can contain). To the extent that we want to allow community contributions (reviews, ratings, images, copied/pasted magazine/external website articles, interviews, reviews, etc. — AND community ratings for each bit of community contributed content) we’ll probably need to add 1 or more tables and numerous fields to the underlying database.

From there, it should be just a matter of creating themes so that users can choose something other than the default - or even create their own if they’ve got CSS/JavaScript/python skills. And from there we need to figure out the best way to get from web UI to iOS/Android/Roku/FireTV/Apple TV app.

I’m stoked that you like the idea and I’m ready to get started but I’ve only recently discovered Beets so I haven’t participated in any of the dev efforts. Nor am I familiar with how the dev community operates. What should be the next step?

1 Like