Should we adopt the 'git flow' branching model?

jtpavlock · July 18, 2020, 12:11am

Git flow is a branching model for git that I think could provide some benefits. One of the main points of git flow is that master is a strictly production/release branch where every commit is a new release.

Some of the benefits from this approach involve improved CD through github actions with workflows such as automatic releasing on each commit with a version tag.

I’ll outline some of my thoughts for the proposal: (see our current release checklist and semver versioning for reference)

Hotfixes (bugs on released versions) get merged directly to master, and immediately trigger a new release. This would allow users to get bugfixes quicker without having to wait on a full new release or having to track the master branch.
- These releases are considered patch releases i.e v1.2.X and wouldn’t require the “human readable paragraph” required for major/minor releases and thus could be fully automated.
Once enough features are merged into develop we could create a release branch for minor and major releases that could then go through a more thorough vetting process including a changelog paragraph, etc. Once it is deemed ready, just add a tag and merge into master to automate the rest.
Also, once a release branch off develop is created, it would allow developers to start working on new features that would normally be incompatible with the current changes.
- Consider the upcoming beets v1.5.0 for example. v1.5.0 is planned to be the last release in which python 2 is supported. If we were to create a release branch right now for v1.5.0, we could give the time needed to manually go over the entire changelog and make sure everything looks good while allowing development such as porting the codebase to pure python 3.

In essence, with a good CI integration (tests, lint, etc.), I believe most, if not everything on our current release checklist could be automated allowing for more frequent releases with the right branching model.

Step 1: adopt git flow
Step 2: add automation for releases

jtpavlock · July 21, 2020, 1:13pm

@guilhermehideki:

I guess that i don’t see beets as “traditional software”, where we need to support multiple versions, like 1.5.1 with python 2 and 1.6.3 with py3, so I don’t see the need today versus using github-flow and git tags for releases

I also don’t follow many FOSS projects to give a opinion “backed by data”, but I would say “You ain’t gonna need it yet” because I think we could this instead:

publish to pypi when creating git tag ‘vXXXXXX’

keep releases small / release early & often (easier said than done

use feature flags

use conventional commits

jtpavlock · July 21, 2020, 1:43pm

True, that is a common use-case for git-flow, but definitely not the only one. I think the benefits I talked about in the OP are still very much relevant.

Agreed this should be implemented regardless of how we do things.

Definitely “easier said than done” right now considering how @adrian likes to do things i.e. the “human readable paragraph” and combing through the changelog prior to minor/major releases. Using release branches doesn’t mean we can’t release often, and it brings about a collaboration effort to going through the changelog and adding that paragraph rather than dumping all the work on @adrian. I guess the idea is that if we do it often, it’s less work required per-release.

Assuming you’re talking about Conventional Commits

This is definitely one I’ve though about too, but I’m not sure how easy it would be to enforce and get everyone on board with. A big advantage (besides the much cleaner git log) I think would be the ability to auto generate changelogs based on commit messages and types. There are also plenty of tools to help with enforcing and writing these types of commit messages.

There exists an alternate workflow to the one I posted, one I actually prefer, but may require a little bit more discipline:

master stays as development branch and is also the “release” branch
All features/bugfixes get merged into master. All commits to master would have to follow conventional commit syntax. The version numbers and changelogs are auto-generated by conventional commit tools (like semantic-release) and the release is automatically published to pypi.
If a human-readable paragraph is warranted, it may have to be added in manually after the release event, as I’m not sure if there’s a good way to integrate it automatically with conventional commits. IMO, these would only be necessary for MAJOR releases, or perhaps on a case-by-case basis, especially if we are releases small and often with this approach. However, I’ll leave that up to @adrian.

This hits the bullet of having releases small and often, and would effectively get rid of our manual changelog file in favor of having it automatically generated through commit messages.

This one I’m not too familiar with, and I’m not quite sure how it would benefit beets. Would you mind explaining what you had in mind for it’s use?

jef · July 22, 2020, 12:21pm

I really like release-please-action. It’s pretty nice, keeps up-to-date PR that includes a changelog of all merges and follows the Conventional Commit and releases via Semantic Versioning (via git tag), like you’ve mentioned! Pretty similar to semantic-release.

I 100% agree here and I think is really only manageable two ways:

Since maintainers merge in the PRs, they are able to change the title of the merge. This is only really relevant to when we lock in the merge strategy of using Squash and Merge. I think @adrian is against this idea though of “losing” history.
To circumvent that problem, we could use a lint tool that enforces that any commit is done properly. This makes it easy to enforce, but a potential pain for our contributors that don’t like process (sorry to through any of you under the bus )

Regardless of implementation, I think it would be great to release this way!

Another method here is trunk based development, which actually sounds a lot more relevant to what we’re talking about here.

As for feature flags, I also agree with @jtpavlock, not sure if it’s relevant here as we are not deploying beets as a service. I’ve used Unleash OSS, SaaS, and Launch Darkly in production. To my knowledge, each would enforce that users of beets would need to install another relational DB and load in the toggles that we provide. A little bit of overhead. I think beets’ plugins are virtually this.

jtpavlock · July 22, 2020, 9:54pm

Perhaps the first thing to do is to determine what we’d like our release cycle to be, and then choosing a workflow that best fits that release cycle. I see a few different options:

Keep current release cycle (arbitrary? or as deemed necessary by the devs - read: @adrian )
Release on every feature or bugfix
Release on every n features or bugfixes (e.g. new release every 5 new features or bugfixes)
Release every n days (e.g. release once a week)
A combination of the above - e.g. ‘git flow’ says release on every hotfix, but features are slated for release arbitrarily.

Then, it’s important to consider how we’d like to handle changelogs. I know right now, @adrian likes to add a human readable paragraph, for example. However, that becomes a lot less realistic or useful if you’re releasing on every feature or bugfix. As I mentioned earlier, maybe adding a human readable paragraph only on new minor or major releases is another option.

Anyway you cut it, I think it’s important to eliminate any arbitrary parameters so that we can minimize human interference as much as possible and let the computers do most of the work. For example, releasing only when deemed necessary or adding paragraphs to changelogs only when deemed necessary adds in more thought/work on the developer to decide when those things should happen.

The good news is, whatever we decide on, even if it’s just keeping what we got now, I think our current process can be improved

I’ll add my personal opinion:
I tend to prefer releases early and often, and automating all the things if possible. Therefore, I’d rather see us release on every feature and bugfix, as like @guilhermehideki said, we aren’t “traditional software” that needs to support multiple versions of a product. Therefore, no reason to wait or have different branches for releases. I would also choose to only add a human changelog paragraph on major releases or every n minor releases.

guilhermehideki · July 24, 2020, 10:18pm

Of course, but I don’t mind the git log master being like this (commits by different people):

feat/plugin1: good commit message 1
feat/plugin2: good commit message 2
chore/changelog: Reorganize something
feat/plugin1: good commit message
chore/changelog: Reorganize something
chore: bump version to vX.XX.XX (tag: X.XX.XX)

Squash & Merge

This is the way I would prefer because of the cons of the other solution (we want to make contributions easier). I don’t know if github lets you edit the squash message, but when I used in gitlab, I would try to include some context if necessary, like:

fix: use {{ workaround }} to fix {{ something }}

-  {{ context }}

closes #12345

If we have people reviewing the PRs, shouldn’t be too onerous to add a context when needed

Feature flags

I guess that I should explain this better because this isn’t the right wording, as @jef said. What I wanted to write is something more simple.

What I’m considering features:

Python versions, for example

so, instead of having code on master (example with python pseudocode)

def print_hello(something):
    print("Hello [}".format(something))

and on feat/drop-python-35 waiting for merge when we drop Python 3.5

def print_hello(something):
    print(f"Hello {something}")

we would have:

def print_hello(something):
    if PYTHON_VERSION >= 36:
        print(f"Hello {something}")
    else:
        print("Hello [}".format(something))

so we could trade code complexity for “merging earlier”. We do this in the AST code

github.com

beetbox/beets/blob/31855a93949a7b2e2636f709c2678d0efef1dcba/beets/util/functemplate.py#L76-L95


      
          if sys.version_info[:2] < (3, 4):
              if val is None:
                  return ast.Name('None', ast.Load())
              elif isinstance(val, six.integer_types):
                  return ast.Num(val)
              elif isinstance(val, bool):
                  return ast.Name(bytes(val), ast.Load())
              elif isinstance(val, six.string_types):
                  return ast.Str(val)
              raise TypeError(u'no literal for {0}'.format(type(val)))
          elif sys.version_info[:2] < (3, 6):
              if val in [None, True, False]:
                  return ast.NameConstant(val)
              elif isinstance(val, six.integer_types):
                  return ast.Num(val)
              elif isinstance(val, six.string_types):
                  return ast.Str(val)
              raise TypeError(u'no literal for {0}'.format(type(val)))
          else:
              return ast.Constant(val)

Off-topic: sorry for not responding earlier. I wrote in the pc and thought that I sent the text; then deleted while accessing via mobile

jtpavlock · August 3, 2020, 10:56pm

As I come back to this, I’m tending to favor less sweeping changes, which means I’m starting to go back to leaning towards git flow. I think our dev workflow can be incrementally improved over time, and git flow most closely matches what we currently do.

It may not be the optimal end goal, but I think it is a step in the right direction that people can get behind. I think for us, the following tenents are relevant:

master branch is release only. releases will be triggered by tagged commits on master.
hotfixes are merged into master, and don’t get a human readable changelog addition (just the bugfix description).
everything else gets merged into develop, which is the new main branch
release branches are cut arbitrarily from develop and merged into master

Not too comlicated, and improves our current methods without changing how we do things too much.

Topic		Replies	Views
Merging with GitHub and Node.js Help	2	661	December 10, 2019
What is the timeframe for the v1.5.0 release? Development	6	1187	July 4, 2020
Next release Development	0	209	January 20, 2024
Moving from Git to stable Help	2	416	November 1, 2020
Project roadmap? Development	26	4576	February 5, 2023

Should we adopt the 'git flow' branching model?

Squash & Merge

Feature flags

Related topics