Preface: this is awesome software, I enjoy seeing the beginning of my cleaned music collection in the place beets is putting it, and Adrian seems to me to be one of the most attentive, friendly, helpful, positive, and supportive developers I’ve come across.
The problem(s) (running beets 1.4.7 on Gentoo Linux):
I’ve been trying to import my collection which is a little shy of 5,000 songs. I tried to import it all at once a few times. After it fails, I have to re-answer the same questions, look up and enter the same album IDs, and then skip or merge the results, (the former feels like I wasted time re-answering or entering, and the latter seems to increase the chance of a crash). I decided to separate the collection into:
- the albums that were in their own folders (which did not get the “group albums” option, while the rest of the collection did),
- one artist that the import was crashing on (the import worked fine when run on just that one artist after failing twice in different ways at that spot in the larger import)
- the rest in arbitrary smaller folders to reduce the impact of a crash and divide up the task of watching the import prompts.
The first two categories, which make up about a third of my songs have made it successfully through. More often than not, an import attempt has ended when the import prompt stops responding and generally some time later, the top command shows that the cpu usage has fallen to zero. I take those to mean beets has crashed. It’s a bit strange since beets verbose mode would be writing in the same terminal space, so the prompt always act a little funny, sometimes ready for input, sometimes not, despite showing a question.
I’ve seen that Python 3.7 can cause some problems, and in my troubleshooting I noticed that even the “beet version” command causes errors with the replaygain plugin, but not with the others I’m using:
$ beet version
beets version 1.4.7
Python version 3.6.5
plugins: replaygain
Traceback (most recent call last):
File "<frozen importlib._bootstrap>", line 971, in _find_and_load
File "<frozen importlib._bootstrap>", line 951, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 875, in _find_spec
ImportError: sys.meta_path is None, Python is likely shutting down
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "<frozen importlib._bootstrap>", line 968, in _find_and_load
SystemError: <class '_frozen_importlib._ModuleLockManager'> returned a result with an error set
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "<frozen importlib._bootstrap>", line 968, in _find_and_load
SystemError: <class '_frozen_importlib._ModuleLockManager'> returned a result with an error set
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/lib64/python3.6/site-packages/gi/types.py", line 238, in mro
File "/usr/lib64/python3.6/site-packages/gi/types.py", line 278, in mro
SystemError: <built-in function hasattr> returned a result with an error set
Segmentation fault (core dumped)
$ beet version
beets version 1.4.7
Python version 3.6.5
plugins: acousticbrainz, badfiles, chroma, duplicates, embedart, fetchart, fromfilename, ftintitle, lastgenre, lastimport, scrub
Thinking this was related to my import problem, I removed that plugin from my config file and continued, but the import seems to fail just as often, or perhaps more often since it has failed twice in a row after just getting started on a smaller folder without replaygain.
As I try over and over, I get more and more “old” entries after I’ve taken the time to answer questions or provide the album ID. So my questions are:
- What might be causing the prompt to stop responding?
1b. Is this truly a crash? The first few times I left it for hours, so I think it is. - Is there best practice for restarting an import that failed to avoid rework?
- While drafting this post, I tried again with -vv. It got much farther than those last two attempts. There have been a few mentions of different behavior with -vv, so perhaps that is to beets as “have you tried rebooting?” is to Windows? It shouldn’t matter but it seems like it does.
3a. In verbose mode, the question is sometimes very far from where the entry is, so I was concerned I might be answering a different question than I thought. To mitigate that I would press ‘m’ as my first answer to anything to either merge or repeat the prompt if that was invalid. What about a command that makes it repeat the question?
3b. After a very long time (an evening of checking the prompt, leaving the computer running and awake overnight, then another 60-90 minutes of watching in the morning), in which beets and I probably got through most of the folder it was aimed at, it locked up/crashed in a different way. Once again, the prompt would not respond to input, and the cpu use went to 0, but I could still enter text and get a new line, unlike the frozen cursor in previous crashes. Using this additional ability I pressed control-C instead of killing it and got the output below. After that the input was the same - entry was possible but not action resulted, so then I killed it. The regular terminal prompt showed several of my last commands went to that instead of to beets.
- So 4 is like 1. With what has happened so far, how can I make the process more reliable and less repetitive and frustrating?
The error mentioned in 3b:
^CTraceback (most recent call last):
File "/usr/lib64/python3.6/site-packages/beets/util/pipeline.py", line 426, in run_parallel
threads[-1].join(1)
File "/usr/lib64/python3.6/threading.py", line 1060, in join
self._wait_for_tstate_lock(timeout=max(timeout, 0))
File "/usr/lib64/python3.6/threading.py", line 1072, in _wait_for_tstate_lock
elif lock.acquire(block, timeout):
KeyboardInterrupt
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib64/python3.6/site-packages/beets/ui/__init__.py", line 1256, in main
_raw_main(args)
File "/usr/lib64/python3.6/site-packages/beets/ui/__init__.py", line 1243, in _raw_main
subcommand.func(lib, suboptions, subargs)
File "/usr/lib64/python3.6/site-packages/beets/ui/commands.py", line 943, in import_func
import_files(lib, paths, query)
File "/usr/lib64/python3.6/site-packages/beets/ui/commands.py", line 913, in import_files
session.run()
File "/usr/lib64/python3.6/site-packages/beets/importer.py", line 329, in run
pl.run_parallel(QUEUE_SIZE)
File "/usr/lib64/python3.6/site-packages/beets/util/pipeline.py", line 439, in run_parallel
thread.join()
File "/usr/lib64/python3.6/threading.py", line 1056, in join
self._wait_for_tstate_lock()
File "/usr/lib64/python3.6/threading.py", line 1072, in _wait_for_tstate_lock
elif lock.acquire(block, timeout):
KeyboardInterrupt
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python-exec/python3.6/beet", line 11, in <module>
load_entry_point('beets==1.4.7', 'console_scripts', 'beet')()
File "/usr/lib64/python3.6/site-packages/beets/ui/__init__.py", line 1284, in main
log.debug(u'{}', traceback.format_exc())
File "/usr/lib64/python3.6/logging/__init__.py", line 1294, in debug
self._log(DEBUG, msg, args, **kwargs)
File "/usr/lib64/python3.6/site-packages/beets/logging.py", line 91, in _log
return super(StrFormatLogger, self)._log(level, m, (), exc_info, extra)
File "/usr/lib64/python3.6/logging/__init__.py", line 1442, in _log
self.handle(record)
File "/usr/lib64/python3.6/logging/__init__.py", line 1452, in handle
self.callHandlers(record)
File "/usr/lib64/python3.6/logging/__init__.py", line 1514, in callHandlers
hdlr.handle(record)
File "/usr/lib64/python3.6/logging/__init__.py", line 861, in handle
self.acquire()
File "/usr/lib64/python3.6/logging/__init__.py", line 812, in acquire
self.lock.acquire()
KeyboardInterrupt