Rclone + beets

Help
1

BACKGROUND

  • VPS (4 vCPU, 16GB RAM, 80GB SSD)
  • Dockerized beets
  • rclone v1.42

rclone makes two folders, both on Google Drive, available to the beets container. _beets-import holds music that needs to be tagged, while _beets-export is where tagged and corrected music is moved.

ISSUE

When I try to run an import from one folder and export to the other, I get a bunch of errors from beets. rclone does not generate any errors.

[...]
Sending event: import_task_files
Sending event: art_set
embedart: Embedding album art into Sam Hulick & Clint Mansell - Mass Effect 3: Extended Cut
embedart: embedding /music/M/Mass Effect 3_ Extended Cut/albumart.jpg
Sending event: write
zero: genre:  -> None
zero: day: 0 -> None
zero: comments:  -> None
zero: month: 0 -> None
save failed: [Errno 9] Bad file descriptor
error writing /music/M/Mass Effect 3_ Extended Cut/01. Wake Up.flac: [Errno 9] Bad file descriptor
embedart: embedding /music/M/Mass Effect 3_ Extended Cut/albumart.jpg
Sending event: write
zero: genre:  -> None
zero: day: 0 -> None
zero: comments:  -> None
zero: month: 0 -> None
save failed: [Errno 9] Bad file descriptor
error writing /music/M/Mass Effect 3_ Extended Cut/03. A Future That Many Will Never See.flac: [Errno 9] Bad file descriptor
embedart: embedding /music/M/Mass Effect 3_ Extended Cut/albumart.jpg
Sending event: write
zero: genre:  -> None
zero: day: 0 -> None
zero: comments:  -> None
zero: month: 0 -> None
save failed: [Errno 9] Bad file descriptor
error writing /music/M/Mass Effect 3_ Extended Cut/04. I Will Watch Over the Ones Who Live On.flac: [Errno 9] Bad file descriptor
embedart: embedding /music/M/Mass Effect 3_ Extended Cut/albumart.jpg
Sending event: write
zero: genre:  -> None
zero: day: 0 -> None
zero: comments:  -> None
zero: month: 0 -> None
save failed: [Errno 9] Bad file descriptor
error writing /music/M/Mass Effect 3_ Extended Cut/05. I Am Alive and I Am Not Alone.flac: [Errno 9] Bad file descriptor
embedart: embedding /music/M/Mass Effect 3_ Extended Cut/albumart.jpg
Sending event: write
zero: genre:  -> None
zero: day: 0 -> None
zero: comments:  -> None
zero: month: 0 -> None
save failed: [Errno 9] Bad file descriptor
error writing /music/M/Mass Effect 3_ Extended Cut/06. We Fought as a United Galaxy.flac: [Errno 9] Bad file descriptor
embedart: embedding /music/M/Mass Effect 3_ Extended Cut/albumart.jpg
Sending event: write
zero: genre:  -> None
zero: day: 0 -> None
zero: comments:  -> None
zero: month: 0 -> None
save failed: [Errno 9] Bad file descriptor
error writing /music/M/Mass Effect 3_ Extended Cut/07. Resolution.flac: [Errno 9] Bad file descriptor
embedart: embedding /music/M/Mass Effect 3_ Extended Cut/albumart.jpg
Sending event: write
zero: genre:  -> None
zero: day: 0 -> None
zero: comments:  -> None
zero: month: 0 -> None
save failed: [Errno 9] Bad file descriptor
error writing /music/M/Mass Effect 3_ Extended Cut/02. An End, Once and for All (extended cut).flac: [Errno 9] Bad file descriptor
Sending event: database_change
scrub: auto-scrubbing /music/M/Mass Effect 3_ Extended Cut/06. We Fought as a United Galaxy.flac
Traceback (most recent call last):
  File "/usr/bin/beet", line 11, in <module>
    load_entry_point('beets==1.4.7', 'console_scripts', 'beet')()
  File "/usr/lib/python2.7/site-packages/beets/ui/__init__.py", line 1256, in main
    _raw_main(args)
  File "/usr/lib/python2.7/site-packages/beets/ui/__init__.py", line 1243, in _raw_main
    subcommand.func(lib, suboptions, subargs)
  File "/usr/lib/python2.7/site-packages/beets/ui/commands.py", line 943, in import_func
    import_files(lib, paths, query)
  File "/usr/lib/python2.7/site-packages/beets/ui/commands.py", line 913, in import_files
    session.run()
  File "/usr/lib/python2.7/site-packages/beets/importer.py", line 329, in run
    pl.run_parallel(QUEUE_SIZE)
  File "/usr/lib/python2.7/site-packages/beets/util/pipeline.py", line 445, in run_parallel
    six.reraise(exc_info[0], exc_info[1], exc_info[2])
  File "/usr/lib/python2.7/site-packages/beets/util/pipeline.py", line 358, in run
    self.coro.send(msg)
  File "/usr/lib/python2.7/site-packages/beets/util/pipeline.py", line 171, in coro
    task = func(*(args + (task,)))
  File "/usr/lib/python2.7/site-packages/beets/importer.py", line 1543, in manipulate_files
    session=session,
  File "/usr/lib/python2.7/site-packages/beets/importer.py", line 747, in manipulate_files
    plugins.send('import_task_files', session=session, task=self)
  File "/usr/lib/python2.7/site-packages/beets/plugins.py", line 476, in send
    result = handler(**arguments)
  File "/usr/lib/python2.7/site-packages/beets/plugins.py", line 140, in wrapper
    return func(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/beetsplug/scrub.py", line 151, in import_task_files
    self._scrub_item(item)
  File "/usr/lib/python2.7/site-packages/beetsplug/scrub.py", line 130, in _scrub_item
    self._scrub(item.path)
  File "/usr/lib/python2.7/site-packages/beetsplug/scrub.py", line 102, in _scrub
    f.delete()
  File "/usr/lib/python2.7/site-packages/mutagen/_util.py", line 140, in wrapper
    return func(self, h, *args, **kwargs)
  File "/usr/lib/python2.7/site-packages/mutagen/flac.py", line 769, in delete
    self.save(filething, padding=lambda x: 0)
  File "/usr/lib/python2.7/site-packages/mutagen/_util.py", line 169, in wrapper
    return func(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/mutagen/_util.py", line 140, in wrapper
    return func(self, h, *args, **kwargs)
  File "/usr/lib/python2.7/site-packages/mutagen/flac.py", line 858, in save
    assert content_size >= 0
AssertionError

QUESTION
What sort of issue does [Errno 9] Bad file descriptor actually indicate?

2

Odd! This is an operating system error, so it’s not really something beets-specific we can answer. You can find lots of info on what this error means on SO, for example: https://stackoverflow.com/questions/11258781/bad-file-descriptor-with-linux-socket-write-bad-file-descriptor-c

Have you tried it outside of Docker?

3

Unfortunately I have not had a chance to try it outside of Docker.

Based on what I’m seeing the problems seem to be happening when metadata gets written to the files, after they’ve been moved to their new location. Is there a way to reverse the process and have the metadata writes happen first, followed by the move?

4

No, there’s not a way to reverse that. If you want, you could try disabling the import.write option altogether, and then manually running beet write to update metadata later.

5

Or, I suppose, you could do the opposite and disable moving/copying—then use beet move later.

6

OK, I think I got it sorted. Basically I’m using beet import to move the music from Google Drive to a local location for metadata writes, then using beet move -d /directory to move them into a different folder on Google Drive. It works well so far.

One further question:

My rename setup is configured as such -

paths:
    default: %upper{%left{$album,1}}/$album%aunique{}/$disc_track. $title

item_fields:
    disc_track: u'%i-%02i' % (disc, track) if disctotal > 1 else u'%02i' % (track)

Right now this means that albums with non A-Z characters make their own folder. I have a catch-all folder setup # where I manually move albums that don’t have actual letter for the first character, but I’d like to build that logic into the renaming string.

I would like to replace

%upper{%left{$album,1}}

with something that will place all albums without a letter for the first character (A-Z) into the # folder, while all other albums are filed normally per that part of the paths command.

I’m after something like this.

if{left{album,1}=[a-z]},upper{left{album,1}},'#'}

Hopefully that makes sense.

7

For complex logic like that, please check out the inline plugin, which lets you use Python code to express what you want to happen. You might also be interested in the bucket plugin.

8

Alright, thanks for the help!

1 Like
9

OK, I’m really close to having it figured out but I’m stuck on the regex part, '^[A-Za-z]'. Basically I’m trying to say “if the first letter of the album is anything A-Z or a-z, capitalize it and use it, otherwise use #.”

album[:1].capitalize() if album[:1] is '^[A-Za-z]' else '#'

The problem I’m running into is that everything is falling under else and returning #, even strings that have A-Z/a-z first characters.

  1. Can I even use regex in the Inline plugin? All the material I’m reading online states that a re module has to be loaded for regex to work (right now I’m testing this in python3 on my VPS; once that’s done I’ll move it to Inline).
  2. Can you think of an easier/alternative way to accomplish this? Today was basically the first time I’ve ever tried to do anything in Python and I’m really lost now.
10

Yep! As the docs say, you can use multi-line Python code, which allows you to write import re and use the module. I can’t write the code for you, but I bet you can get it!

11

Alright, I got it to work in Python

album[:1].capitalize() if re.match(r"^[A-Za-z]",album) else '#'

and added it to my config.yaml

paths:
    default: $first_letter/$album%aunique{}/$disc_track. $title

item_fields:
    disc_track: u'%i-%02i' % (disc, track) if disctotal > 1 else u'%02i' % (track)
    first_letter: |
            import re
            album[:1].capitalize() if re.match(r"^[A-Za-z]",album) else '#'

But when I go to move files, I encounter a permissions error.

docker exec -it -u abc beets bash -c 'beet move -d /temp/import2 album:45:33'
Moving 8 items.
Error: Permission denied while creating /45_33 Remixes

If I use my original path: option, it works

paths:
    default: %upper{%left{$album,1}}/$album%aunique{}/$disc_track. $title

docker exec -it -u abc beets bash -c 'beet move -d /temp/import2 album:45:33'
Moving 8 items.

Any ideas?
[EDIT] - It looks like I need to return the value so beets can actually use it. Back to Google.

1 Like
12

Well. Spent most of my day trying to cram return values into my Inline block. Managed to get it figured out.

item_fields:
    disc_track: u'%i-%02i' % (disc, track) if disctotal > 1 else u'%02i' % (track)
    first_letter: |
            import re
            def first_letter(album):
                if re.match(r"^[A-Za-z]",album):
                    return album[:1].capitalize()
                else:
                    return '#'

But I still get permission errors when trying to move or import albums

docker exec -it -u abc beets bash -c 'beet move -d /temp/import2 album:45:33'
Moving 8 items.
Error: Permission denied while creating /45_33 Remixes

I thought to myself “Ho ho, hold my beer I’ve got this,” and ran the same command as root from within the container. The files moved alright, but now they’re not visible in the folder.

root@fc31a34686a9:/temp/import2$ ls -la
total 8
drwxrwxr-x 2 abc users 4096 Aug 27 23:48 .
drwxrwxr-x 9 abc users 4096 Aug 27 23:50 ..

Not sure what to do.

13

These problems really sound like issues with the container setup—it’s very difficult to see from here how beets is involved with them. You might try manually manipulating files in there, for example, or taking a look at beets’s verbose log for clues.

14

Here’s what I am using for the same objective:

item_fields:
   initial_char: |
       char = album[0].capitalize()
       if album[0].isnumeric():
           char = '#'
       return char
15

/45_33 Remixes is not the same folder as /temp/import2. Which is why root privileges allows you to write to the filesystem root.

Check that your cache configuration is properly set up so you can use beets directly against your remotes.