Proposal: Drop Windows support in favor of WSL

Development
1

It’s no secret Windows can cause a lot of headache when developing and debugging beets. I’d be curious to know the actual percentage, but I would wager the majority of users and contributors run *nix.

I am proposing we drop official Windows support in favor of recommending Windows users to run Windows Subsystem Linux. This would effectively eliminate support for those on Windows 7 and below (since WSL is only available on Windows 10). Windows 7 support has also been dropped by Microsoft as of Jan 24th, 2020.

To test this out, I booted up beets on WSL and it seems to work just fine. All windows files are available under /mnt/c/ and thus follow linux pathing. The entire test suite also passes except for a couple tests dealing with the the permissions plugin. Fortunately, it’s also possible to tell you’re not running native Linux, and those tests and just be skipped as they are now with Windows until a root cause/fix can be determined.

Thoughts? I’d love to hear from any Windows users or devs what they think as well.

2

I have mixed feelings about this. I use beets on Windows. However, I have already installed WSL 2 on my main system and use Python on it for Jupyter notebooks. So, using beets from there would not be a big deal for me.

If there were a good setup guide for users, I think that would make it an easier transition.

3

It’s worth discussing. To be honest, however, I’m not sure how much development burden we would actually save by dropping Windows support. Often, the CI problems that show up on Windows are actually revealing “ticking time bombs” that could easily affect Unix as well—and so they kind of keep us honest about making sure we do the right thing with encodings, availability of OS features, etc. There are other issues that truly do represent Windows-specific quirks, however (such as an inability to share SQLite connection objects across threads, which complicates some tests, and certainly the \\?\ prefix weirdness), but those are fairly few and far between. I also expect that some of these issues will lessen without Python 2 in the mix.

Out of curiosity, did you have a specific set of issues you were thinking of?

1 Like
4

What initially started this thought was https://github.com/beetbox/beets/pull/3665, the work that goes into handling paths, and also the frustration of random things like https://github.com/beetbox/beets/issues/3613.

It seems like these Windows idiosyncrasies plus the other ones you mentioned, coupled with the fact that debugging these issues can be annoying with appveyor, made it seem like things could be simplified by just switching to *nix support. Also, development would be a bit easier/cleaner without having to worry about potential Windows conflicts.

The barrier for entry to WSL is quite low, and once you get it set up, it’s no different than pulling up a command prompt to execute beets.

Also, I guess technically it wouldn’t be completely dropping Windows support, just a switch to supporting operation on WSL :wink:

5

Just to break down those three examples, as a kind of sampling of the issues we run into:

  • #3665 is indeed about a Windows-specific quirk about how to open a file. But a good implementation there can actually be quite self-contained and shouldn’t really cost us much—basically, we need to issue a different command to open things on Windows than on macOS or Linux (which need their own “opening things” commands themselves). With respect to WSL in particular, I’m not sure that solves this problem—we’d still need to figure out what to do on WSL to open notepad.exe or whatever as a text editor.
  • Path handling would certainly be simplified somewhat by dropping Windows, but IMO the majority of the complexity we run into is actually dealing with POSIX paths these days because of the complex and somewhat broken way that Python 3 deals with them. (I’m happy to expand on this more if anyone’s curious.) In a way, things would actually be much simpler if we dropped Unix and just had to deal with Windows paths. :crazy_face: Anyway, I think we could save some complexity here but not a lot because paths are just inherently complicated.
  • The CI thing (#3613) is indeed annoying. However, the specific issue here is encoding stuff—and in a way, it’s actually good that this is forcing us to root-cause the encoding problem or the testing setup, which probably indicates that we’re doing something subtly wrong regardless of platforms. (It’s one of those “ticking time bomb” things.)

And here’s a much less good reason to keep Windows support that I can’t quite ignore. :smiley: I have never been a Windows user and I mostly resent its weird quirks as a software developer. But on the other hand, I have a lingering concern about the downsides of an OS monoculture. If lots of software decides that POSIX is the only kind of OS in the world, we run the risk of embracing the weird quirks of Unix (and they do exist!) as if they were fundamental to how computers work. In a way, it would be nearly as bad as when lots of software vendors decided in 1999 that Windows was the only OS that mattered in the world (and IE was the only browser in the world that mattered), and they thereby stopped caring about writing code that was platform-independent in any meaningful sense.

Put in slightly more practical terms, keeping Windows CI enabled (hopefully on much faster GitHub infrastructure in the future!) has helped us identify where we were making too many assumptions about the underlying OS, which I think has led to the beets code being higher quality overall—on all platforms. I would be somewhat sad to lose that modicum of attention to the idea that software can be cross-platform.

1 Like
6

I feel like I’m one of the most intensely-customized users of Beets on Windows, and I’m tentatively fine with switching to WSL. Granted, I’m more comfortable with Linux now than I was a few years ago. Consider:

  1. a casual user shouldn’t notice a difference between cmd.exe and WSL
  2. a developer is going to be better served in a Linux environment

Paths in WSL go from d:\music\ to /mnt/d/music/ but not a huge deal imo. I would appreciate if someone could kick the tires of WSL+beets before making such a big change though. It’s also worth truly understanding which bugs in beets are unique to Windows. @adrian makes a good point.

7

I have been running beets on windows for a number of years.

Yesterday I made the plunge to try beets in WSL. I had been thinking about this for a while.

I started with Ubuntu-20.04.

I run beets from a local git copy.

From WSL I did the following:

pip3 install -e .
beet

Then I copied my

%APPDATA%\Roaming\beets directory to
~/.config/beets

Edited config file to change any windows filepath references to WSL paths.

Created in windows a library-linux.db and used DB Browser for SQLite to run the following commands (you could use sqlite command line):

update albums set artpath = replace(artpath, "i:", "/mnt/i")
update albums set artpath = replace (artpath, "\", "/")

update items set path = replace(path, "I:", "/mnt/i")
update items set path = replace (path, "\", "/")

The replace command IS case-sesitive.

Replaced my WSL library.db with the updated library-linux.db

Ran in both windows and Linux and compared the output

beet config -e
beet -v move -p
beet ls -a added:2020-07

The only other thing I had to do was install all of the plugin dependencies then treat that as a pip install on WSL:

pip freeze

That seems to be working for me (YMMV). :smile:

8

In running in WSL Ubuntu-20.04

tox -e py-test

The only errors I receive have to do with

test_permissions.py

Trying to run tox under windows was a dismal failure with Unicode errors. I might try again later.

9

The E:/ ↔ /mnt/e/ changeover in the beets database seems like something that beets should handle internally if we ask users to switch.

Trying to run tox under windows was a dismal failure with Unicode errors. I might try again later.

Are you saying WSL tox works well, but Windows tox failed?

10

I did think about moving the database and just changing the config file to point to the new location. But I have a large collections so I tried manually. It should work. @adrian thoughts?

WSL works well for tox.

Windows fails during startup with

tox -e py-test
GLOB sdist-make: h:\Git\arogl-beets\setup.py
Traceback (most recent call last):
  File "c:\python37\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "c:\python37\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Python37\Scripts\tox.exe\__main__.py", line 7, in <module>
  File "c:\python37\lib\site-packages\tox\session\__init__.py", line 44, in cmdline
    main(args)
  File "c:\python37\lib\site-packages\tox\session\__init__.py", line 68, in main
    exit_code = session.runcommand()
  File "c:\python37\lib\site-packages\tox\session\__init__.py", line 195, in runcommand
    return self.subcommand_test()
  File "c:\python37\lib\site-packages\tox\session\__init__.py", line 211, in subcommand_test
    venv.package = self.hook.tox_package(session=self, venv=venv)
  File "c:\python37\lib\site-packages\pluggy\hooks.py", line 286, in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
  File "c:\python37\lib\site-packages\pluggy\manager.py", line 93, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "c:\python37\lib\site-packages\pluggy\manager.py", line 87, in <lambda>
    firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
  File "c:\python37\lib\site-packages\pluggy\callers.py", line 208, in _multicall
    return outcome.get_result()
  File "c:\python37\lib\site-packages\pluggy\callers.py", line 80, in get_result
    raise ex[1].with_traceback(ex[2])
  File "c:\python37\lib\site-packages\pluggy\callers.py", line 187, in _multicall
    res = hook_impl.function(*args)
  File "c:\python37\lib\site-packages\tox\package\__init__.py", line 16, in tox_package
    session.package, session.dist = get_package(session)
  File "c:\python37\lib\site-packages\tox\package\__init__.py", line 29, in get_package
    package = acquire_package(config, session)
  File "c:\python37\lib\site-packages\tox\package\__init__.py", line 40, in acquire_package
    path = build_package(config, session)
  File "c:\python37\lib\site-packages\tox\package\builder\__init__.py", line 7, in build_package
    return make_sdist(config, session)
  File "c:\python37\lib\site-packages\tox\package\builder\legacy.py", line 39, in make_sdist
    returnout=True,
  File "c:\python37\lib\site-packages\tox\action.py", line 120, in popen
    lines = out_path.read_text("UTF-8").split("\n")
  File "c:\python37\lib\site-packages\py\_path\common.py", line 171, in read_text
    return f.read()
  File "c:\python37\lib\codecs.py", line 322, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x92 in position 21509: invalid start byte

That is even after I ran from witin WSL successfully in the same directory.

Tox on windows has worked before

11

I uninstalled tox and installed via
‘’’
pip install “tox<=3.8.1”

Running fine.

Some tests failed, I think I need to install some dependencies for the tests to run.

I'll compare my errors to appveyor tests.
12

Intersting, that may be a fix for https://github.com/beetbox/beets/issues/3613, would you be willing to test it out? I wonder what changed in tox after that version to cause that behavior.

13

I installed all dependencies and I got fewer errors. More around Unicode.
I tried the latest powershell under windows and that failed with similar errors.

I’ll try a chcp 65001 and see what changes.

Off to read tox to get the actual calls that failed

1 Like
14

Realize this is an old thread, but from a new user’s perspective, I had a much easier time setting things up using WSL2 than I did on native Windows. Figuring out WSL2 (I do have linux experience) was much easier than trying to get some of the plugin dependencies running on Windows.