Repository navigation
Conversation
With fsmonitor and the untracked cache enabled, "git status" still
visits every directory recorded in the cache, even when the hook
reports no changes. For each one, read_directory_recursive() calls
prep_exclude() and runs treat_path() on each cached entry. In a large
tree with few untracked files, this walk is most of the time spent in
read_directory(), and it finds nothing.
The walk cannot stop at a directory just because fsmonitor left it
valid. The "valid" bit only covers the directory's own entries, and
an untracked file may be cached further down.
Instead, after refresh_fsmonitor() has applied the hook's response,
walk the cache once and mark each directory that is valid, is not a
check-only entry, has no untracked entries and no .gitignore, and has
only marked subdirectories. read_directory_recursive() then returns
early for a marked directory once open_cached_dir() has accepted its
cached data. Nothing is lost, because the subtree holds no cached
untracked paths and only check-only scans use the return value.
Check-only entries are not marked because the scan that fills them
stops at the first untracked path. Directories holding a .gitignore,
and those above them, are still visited, so ignore rules are checked
as before.
The mark is used only with fsmonitor, because without it a valid
directory must still be lstat()ed. If the hook fails or asks for a
full rescan, refresh_fsmonitor() turns off use_fsmonitor and nothing
is marked. The mark is kept in memory only, as storing it in the
UNTR extension would change the index format.
Times are medians of alternating runs of master and this patch on
ext4 under Linux (WSL2), with a warm cache and a clean worktree. For
"-uall", status.showUntrackedFiles was set to "all" so that the cache
applies. Each output matched a run without fsmonitor or the
untracked cache.
master patched
linux.git (97k files, 6.3k directories),
fsmonitor--daemon, 61 pairs:
git status 37.0ms 28.1ms
git status -uall 35.9ms 28.1ms
t/perf/repos/many-files.sh -d 5 -w 10 -f 9
(1M files, 111k directories), hook, 33 pairs:
git status 246.5ms 216.9ms
In linux.git, trace2's "directories-visited" count drops from 6,338 to
1,809. In a large internal repository with many .gitignore files,
"git status" goes from 3.45s to 1.36s, and "directories-visited" from
~300k to ~50k. When nothing can be skipped, as in synthetic trees
with an untracked file in every leaf directory, the pass slows "git
status" by up to 2%, or 4% in a tree 49 directories deep.
Add tests to t7519 checking that unchanged empty subtrees are skipped,
and that new paths, index and ignore changes, and a disabled, failing
or trivial fsmonitor are still handled.
Signed-off-by: Maik Riechert <letmaik@outlook.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The existing p7519 tests do not measure this case. perf-lib writes
its timing files into the worktree, so each timed "git status" finds
a new untracked file and rewrites the index.