Duplicate files

Help
1

I run the command beet import --noautotag music-incoming/ and choose the B to aBort a couple of times I think is what caused it.
I can probably use some linux commands to remove the duplicates but I rather have a solution to prevent this from happening.

I noticed that a lot of imported albums have duplicate files:

music/Celsius
├── 1992
│   ├── 01 Initiation.1.flac
│   ├── 01 Initiation.flac
│   ├── 02 Not Gettin In.1.flac
│   ├── 02 Not Gettin In.flac
│   ├── 03 Raverface.1.flac
│   ├── 03 Raverface.flac
│   ├── 04 Just Getting Warm.1.flac
│   ├── 04 Just Getting Warm.flac
│   ├── 05 Test Pressing.1.flac
│   ├── 05 Test Pressing.flac
│   ├── 06 Hear Me Now Rudeboy.1.flac
│   ├── 06 Hear Me Now Rudeboy.flac
│   ├── 07 Banger.1.flac
│   ├── 07 Banger.flac
│   ├── 08 Trip To The Moon.1.flac
│   ├── 08 Trip To The Moon.flac
│   ├── 09 Manic Shuffle.1.flac
│   ├── 09 Manic Shuffle.flac
│   ├── 10 The Clock.1.flac
│   ├── 10 The Clock.flac
│   ├── 11 Revolver.1.flac
│   ├── 11 Revolver.flac
│   ├── 12 Fusion.1.flac
│   ├── 12 Fusion.flac
│   ├── 13 Acidfucker.1.flac
│   ├── 13 Acidfucker.flac
│   ├── cover.1.jpg
│   └── cover.jpg

I tried the duplicates plugin beet duplicates it does not list the above example as a duplicate.

While importing I only choose remove old or skip as shown in the log:

cat ~/beetslog.txt
import started Sat May 20 20:48:13 2023
import started Sat May 20 20:48:52 2023
asis /media/sdb/music-incoming/...
skip /media/sdb/music-incoming/...
skip /media/sdb/music-incoming/...
skip /media/sdb/music-incoming/...
skip /media/sdb/music-incoming/...
asis /media/sdb/music-incoming/...
import started Sun May 21 10:14:24 2023
import started Sun May 21 10:37:19 2023
asis /media/sdb/music-incoming/...
duplicate-replace /media/sdb/music-incoming/...
skip /media/sdb/music-incoming/...
skip /media/sdb/music-incoming/...
import started Sun May 21 11:28:11 2023
import started Sun May 21 11:46:59 2023
asis /media/sdb/music-incoming/...
duplicate-replace /media/sdb/music-incoming/...
asis /media/sdb/music-incoming/...
skip /media/sdb/music-incoming/...
import started Sun May 21 12:52:50 2023
duplicate-replace /media/sdb/music-incoming/...
import started Sun May 21 13:13:30 2023
duplicate-replace /media/sdb/music-incoming/...
import started Sun May 21 13:28:02 2023
import started Sun May 21 13:37:27 2023
import started Sun May 21 19:11:20 2023
import started Sun May 21 19:16:21 2023

Config:

directory: /media/sdb/music
library: ~/data/musiclibrary.db
import:
  log: ~/beetslog.txt
  original_date: true
  ignore_hidden: yes
match:
  required: label
plugins: discogs fetchart
discogs:
  index_tracks: yes
fetchart:
  cautious: true
  cover_names: cover
  sources: filesystem *

Running Python 3.10.6

2

Hi! It’s unfortunately pretty hard to see what happened here without a log from the original import. Any chance you could try this again and see if the same thing happens?

3

HI Adrian thank you for replying. I just removed the database to start over. I was thinking I can ommit this by adding move: yes to my config.

What I noticed is that when I import let’s say 10 albums and then press B the process stops immediately. When I run the import command again I choose continue and it started where I left.
Then at some point half way the import or later (250/500 albums) when I pressed B the proces did not stop even after waiting like 5/10 minutes, instead I pressed CTRL+C a couple of times to quit the process.
When I then run the import command again I choose continue and it started like maybe 20 albums which I already processed. So my guess is that here it went wrong. However this does not explain why the album starting with the letter C also have this.

Am I correct that I need to add import -l -vv to my import command or is there anything else I need todo?

4

Aborting is surprisingly tricky with beets’s multi-threaded importer… it tries to shut down as quickly as possible, even if that is not all that quickly, so I wouldn’t be entirely shocked if it wasn’t perfect about making sure every album was fully imported and recorded to the import log.

Verbose logs may help, but the “right” solution here might be finding ways to avoid aborting (i.e., to import only those albums you want to in a given session).

5

Verbose logs may help, but the “right” solution here might be finding ways to avoid aborting (i.e., to import only those albums you want to in a given session).

Yes I think you are right then most likely this is what caused it. I will be more patient and let it finish next time. I thought it was stuck or something.

Reason why I aborted were 2 things.

  1. To continue at a later time.
  2. Beets did not find a match. So I abort added the release to MusicBrainz or Discogs first then continue.