The Git project recently released Git v2.46.0. Let's look at a few notable highlights from this release, which includes contributions from GitLab's Git team and the wider Git community.
Tooling to migrate reference backends
In the previous Git 2.45.0 release, the reftables format was introduced as a new backend for storing references. This new reference format solves some challenges that large repositories face as the number of references scales. If you are not yet familiar with the reftables backend, check out our previous Git release blog post where the feature was introduced and our beginner’s guide to learn more about how reftables work.
The reftable backend has a different on-disk format than the pre-existing files backend. Consequently, to use reftables on an existing repository requires a conversion between the different formats. To accomplish this, a new git-refs(1) command has been introduced with the migrate subcommand to perform reference backend migrations. Below is an example of how this command can be used.
# Initialize a new repository as “bare” so it does not contain reflogs.
$ git init --bare .
$ git commit --allow-empty -m "init"
# Populate repository with references in the files backend.
$ git branch foo
$ git branch bar
$ tree .git/refs
.git/refs
├── heads
│ ├── bar
│ ├── foo
│ ├── main
└── tags
# Perform reference migration to reftables format.
$ git refs migrate --ref-format=reftable
# Check that reftables backend is now in use.
$ tree .git/reftable
.git/reftable
├── 0x000000000001-0x000000000001-a3451eed.ref
└── tables.list
# Check the repository config to see the updated `refstorage` format.
$ cat config
[core]
repositoryformatversion = 1
filemode = true
bare = true
ignorecase = true
precomposeunicode = true
[extensions]
refstorage = reftable
Once a repository has been migrated, the on-disk format is changed to begin
using the reftable backend. Git operations on the repository continue to
function and interact with remotes the same as before. The migration only
affects how references are stored internally for the repository. If you wish to go back to the files reference backend, you can accomplish this with the same command by instead specifying --ref-format=files.
The migration tooling currently has some notable limitations. The reflogs in a repository are a component of a reference backend and would also require migration between formats. Unfortunately, the tooling is not yet capable of converting reflogs between the files and reftables backends. Also, a repository with worktrees essentially has multiple ref stores and the migration tool is not yet capable of handling this scenario. Therefore, if a repository contains reflogs or worktrees, reference migration is currently unavailable. These limitations may be overcome in future versions.
Because a bare Git repository does not have reflogs, it is easier to migrate. To migrate a standard non-bare repository, reflogs must be pruned first. Therefore, any repository without reflogs or worktrees can be migrated. With these limitations in mind, this tool can be used to begin taking advantage of the reftables backend in your existing repositories.
This project was led by Patrick Steinhardt.
Transactional symref updates
The git-update-ref(1) command
performs reference updates in a Git repository. These reference updates can also be performed atomically in bulk with transactions by using
git update-ref --stdin and passing update-ref instructions on stdin. Below is an example of how this is done.
$ git init .
$ git branch -m main
$ git commit --allow-empty -m "foo" && git commit --allow-empty -m "bar"
# Retrieve the object ID of the two commits created.
$ git rev-parse main~ main
567aac2b3d1fbf0bd2433f669eb0b82a0348775e
3b13462a9a42e0a3130b9cbc472ab479d3ef0631
# Start transaction, provide update-ref instructions, and commit.
$ git update-ref --stdin <<EOF
> start
> create refs/heads/new-ref 3b13462a9a42e0a3130b9cbc472ab479d3ef0631
> update refs/heads/main 567aac2b3d1fbf0bd2433f669eb0b82a0348775e
> commit
> EOF
$ git for-each-ref
567aac2b3d1fbf0bd2433f669eb0b82a0348775e commit refs/heads/main
3b13462a9a42e0a3130b9cbc472ab479d3ef0631 commit refs/heads/my-ref
From this example, once the transaction is committed, a new branch is created pointing to the “bar” commit and the main branch is updated to point to the previous “foo” commit. Committing the transaction performs the specified reference updates atomically. If an individual reference update fails, the transaction is aborted and no reference updates are performed.
A notable absence here is instructions to support symref updates in these
transactions. If a user wants to update a symref along with other references
atomically in the same transaction, there is no tooling to do so. In this
release, the symref-create, symref-update, symref-delete, and
symref-verify instructions are introduced to provide this functionality.
# Create a symref that will be updated during the next operation.
$ git symbolic-ref refs/heads/symref refs/heads/main
# The --no-deref flag is required to ensure the symref itself is updated.
$ git update-ref --stdin --no-deref <<EOF
> start
> symref-create refs/heads/new-symref refs/heads/main
> symref-update refs/heads/symref refs/heads/new-ref
> commit
> EOF
$ git symbolic-ref refs/heads/symref
refs/heads/new-ref
$ git symbolic-ref refs/heads/new-symref
refs/heads/main
From the above example, a new symbolic reference is created and another is updated in a transaction. These new symref instructions can be used in combination with the pre-existing instructions to perform all manner of reference updates now in a single transaction. Check out the documentation for more information regarding each of these new instructions.
This project was led by Karthik Nayak.
UX improvements for git-config(1)
The git-config(1) command allows repository and global options to be viewed and configured. The modes used to interact with configuration can be selected explicitly using flags or determined implicitly based on the number of arguments provided to the command. For example:
$ git config --list
# Explicit retrieval of username configuration
$ git config --get user.name
# Implicit retrieval of username configuration
$ git config user.name
# Explicit setting of username configuration
$ git config --set user.name "Sidney Jones"
# Implicit setting of username configuration
$ git config user.name "Sidney Jones"
# An optional third argument is also accepted. What do you think this does?
$ git config <name> [<value> [<value-pattern>]]
Overall, the git-config(1) user
interface is not consistent with how other more modern Git commands work where you usually use subcommands. For example, git remote list. This release introduces list, get, set, unset, rename-section, remove-section, and edit as subcommands for use with the config command while also keeping the old-style syntax available. This change aims to improve user experience by adapting the config command to follow more UI practices and better conform to other commands within Git. For example:
$ git config list
$ git config get user.name
$ git config set user.name "Sidney Jones"
This project was led by Patrick Steinhardt.
Addressed performance regression
Git operations that leverage attributes rely on reading .gitattributes files found in the repository’s working-tree. This is problematic for bare Git repositories because by definition they lack a working-tree. To get around this, Git has the attr.tree configuration that allows a source tree to be specified and used to lookup attributes from.
In Git release 2.43.0, Git started using the tree of HEAD as the source of Git attributes for bare repositories by default. Unfortunately, the additional overhead due to scanning for Git attributes files had severe performance impacts. This is because, when attr.tree is set, each attribute lookup requires walking the source tree to check for an associated .gitattributes file. The larger and deeper the source tree of the repository is, the more pronounced the performance regression becomes. For example, benchmarks run on the linux.git repository showed
git-pack-objects(1) taking 1.68 times longer to complete. This could lead to slowdowns when performing clones or fetches.
# attr.tree set to HEAD as done by default in Git version 2.43.0.
Benchmark 1: git -c attr.tree=HEAD pack-objects --all --stdout </dev/null >/dev/null
Time (mean ± σ): 133.807 s ± 4.866 s [User: 129.034 s, System: 6.671 s]
Range (min … max): 128.447 s … 137.945 s 3 runs
# attr.tree is set to an empty tree to disable attribute lookup as done in Git versions prior to 2.43.0.
Benchmark 2: git -c attr.tree=4b825dc642cb6eb9a060e54bf8d69288fbee4904 pack-objects --all --stdout </dev/null >/dev/null
Time (mean ± σ): 79.442 s ± 0.822 s [User: 77.500 s, System: 6.056 s]
Range (min … max): 78.583 s … 80.221 s 3 runs
Some of the most notable Git commands that were affected were clone, pull, fetch, and diff when, as previously mentioned, used on repositories with large or deep trees. Consequently, the attr.tree configuration was partially reverted to no longer be set to HEAD by default to address the performance regression. To learn more, check out this
thread on the mailing list.
Unit-test migration
Historically, testing in the Git project has been done via end-to-end tests implemented as shell scripts. The Git project has relatively recently introduced a unit-testing framework written in C. This new testing framework brings opportunities for more in-depth testing of low-level implementation details at the individual function call level and helps complement the existing end-to-end tests. There are some existing end-to-end tests that are a better fit as unit-tests and thus are good candidates to be ported.
This year, GitLab is again helping mentor Google Summer of Code (GSoC) contributors working in the Git project. Thanks to efforts from these ongoing GSoC projects and also the wider Git community, some existing tests are being refactored and migrated to the unit-testing framework. During this last release cycle, there have been several contributions towards the goal of improving the testing in the Git project. To follow development progress for these GSoC contributor projects, check out Chandra’s and Ghanshyam’s blogs.
Bundle URI fixes
Usually when a client fetches from a remote repository, all required objects are sent in a packfile computed by the remote server. To avoid some of this computation, servers can opt to advertise prebuilt “bundles” stored separately from the remote server which contain sets of references and objects that the client may need. The client can fetch these bundles first through a mechanism called bundle-uri.
Thanks to Xing Xin, an issue was identified and fixed where Git, despite having downloaded some bundles, was still downloading everything from the remote as if there were no bundles. This was due to Git not correctly discovering all the downloaded bundles, which resulted in having to fetch the consecutive ones from the remote. With this fixed, remotes using the bundle-uri mechanism can avoid having to perform redundant work and improve performance.
Read more
This article highlighted just a few of the contributions made by GitLab and the wider Git community for this latest release. You can learn about these from the official release announcement of the Git project. Also, check out our previous Git release blog posts to see other past highlights of contributions from GitLab team members.
