Fix repository discovery precedence - #2218
Conversation
There was a problem hiding this comment.
Pull request overview
Updates repository discovery to prioritize .git metadata, validate linked repositories, preserve common-directory handling, and document the security advisory.
Changes:
- Resolve
.gitbefore bare-repository detection. - Validate
HEADandcommondirmetadata. - Preserve
GIT_COMMON_DIR, add regression tests, and update the changelog.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 4 comments.
| File | Review summary |
|---|---|
test/test_repo.py |
Adds repository discovery and common-directory tests. |
git/repo/fun.py |
Three unresolved moderate findings: malformed HEAD targets are accepted (3 votes); malformed commondir handling can escape or raise incorrectly (2 votes); FIFO HEAD files can block discovery (2 votes). |
git/repo/base.py |
One unresolved moderate finding (2 votes): relative GIT_COMMON_DIR values can cause subsequent Git operations to target the wrong repository. |
doc/source/changes.rst |
Adds the security advisory to the changelog. |
Suppressed comments (3)
git/repo/base.py:310
osp.existsfollows symlinks, so a dangling.gitsymlink makes this condition false. Discovery then continues intois_git_dir(curpath)or a parent, allowing a bare-looking worktree/ancestor repository to be selected despite an existing malformed.gitentry. Use an existence check that does not follow symlinks (for example,lexists) so broken.gitentries stop discovery like other invalid gitfiles.
if osp.exists(dotgit) and not osp.isdir(dotgit):
break
git/repo/fun.py:97
- Using
ortreats an explicitly set emptyGIT_OBJECT_DIRECTORYas if the variable were unset. Git's discovery code checks whether this variable is present and then validates that exact path, so withGIT_OBJECT_DIRECTORY=""this implementation can accept a repository that Git rejects and leave later Git commands inconsistent with discovery. Test for presence inos.environand use the empty value as invalid rather than falling back to<common_dir>/objects.
object_dir = os.getenv("GIT_OBJECT_DIRECTORY") or osp.join(common_dir, "objects")
git/repo/fun.py:103
- A Git linked-worktree administrative directory contains
gitdir,commondir, andHEAD; the.gitfile is in the working tree, not atd/gitfile. Replacing theHEADcheck withgitfilemakes this unsupported-worktree detection miss its documented layout when common storage is invalid. Keep checkingHEADhere.
and osp.isfile(osp.join(d, "gitfile"))
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
|
Codex on behalf of Byron: I also checked the three suppressed review notes. Commit 32baeab now rejects empty GIT_OBJECT_DIRECTORY and restores HEAD-based linked-worktree detection. I left dangling .git symlink behavior unchanged: Git setup.c read_gitfile_gently uses stat, so a dangling symlink is treated as missing and discovery falls through to the bare candidate or parent; local git rev-parse checks confirmed that behavior. |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.
Suppressed comments (6)
Previously missed (2) — in code that hasn't changed since the last review.
git/repo/base.py:309
- Use
osp.lexistsrather thanosp.existshere.existsfollows symlinks, so a dangling.gitsymlink makes this condition false; discovery then proceeds tois_git_dir(curpath)and can classify the directory as a bare repository (or continue to a parent) despite the invalid.gitentry. That bypasses the precedence/rejection this guard is meant to enforce.
if osp.exists(dotgit) and not osp.isdir(dotgit):
git/repo/base.py:300
- The new precedence path recursively follows
.gitpointers throughfind_submodule_git_dirbefore the bare-repository check, but that helper has no cycle detection. A malformed file containinggitdir: .git(or a two-file cycle) recurses untilRecursionErrorinstead of being rejected as invalid metadata; bound the resolution or track visited paths and return an invalid-candidate result.
sm_gitpath = find_submodule_git_dir(dotgit)
git/repo/base.py:385
- When only
GIT_DIRis set to a relative path, discovery expands it to an absoluteself.git_dir, but this conditional leaves the wrapper's inherited relativeGIT_DIRunchanged unlessGIT_COMMON_DIRis also set. Git commands run withself.working_dir, so they resolve that value from a different directory and can fail or address the wrong repository. NormalizeGIT_DIRwhenever it is supplied, independently ofGIT_COMMON_DIR.
if common_dir_env is not None:
self.git.update_environment(GIT_DIR=os.fspath(self.git_dir), GIT_COMMON_DIR=os.fspath(self.common_dir))
git/repo/base.py:300
find_submodule_git_dirreads the.gitentry with the default text decoder and does not catchUnicodeError. Consequently, a.gitfile containing invalid UTF-8 raisesUnicodeDecodeErrorfrom this new discovery path instead of being rejected as an invalid repository; use the same byte-oriented decoding approach ascommondiror catch decode errors in the resolver.
sm_gitpath = find_submodule_git_dir(dotgit)
git/repo/base.py:364
is_git_dirreadscommondirwithos.fsdecode, so a valid common-directory path containing non-UTF-8 filesystem bytes can pass discovery. The constructor then re-reads the same file withPath.read_text(), which raisesUnicodeDecodeErrorinstead of constructing the repository; use the same filesystem decoding here.
common_dir = (Path(self.git_dir) / "commondir").read_text().splitlines()[0].strip()
git/repo/fun.py:93
- Treat a dangling
commondirsymlink as malformed, not as an absent file.read_bytes()raisesFileNotFoundErrorfor the dangling link, so this branch assignsdand can accept localHEAD/objects/refs;Repothen opens the candidate as a bare repository instead of rejecting the malformed metadata. Checklexistsbefore taking the missing-file fallback.
except FileNotFoundError:
common_dir = os.fspath(d)
|
Codex on behalf of Byron: I reviewed all six suppressed notes from the latest Copilot pass. Commit b8c000e fixes one-level, regular, size-bounded Gitfile resolution, filesystem decoding for .git and commondir metadata, and relative GIT_DIR persistence. Commit 83c3e64 fixes dangling commondir symlinks. I left dangling .git symlinks unchanged because Git read_gitfile_gently() uses stat and treats them as missing; the visible FIFO commondir thread likewise documents Git's file_exists()/strbuf_read_file() behavior. |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.
Suppressed comments (1)
Previously missed (1) — in code that hasn't changed since the last review.
git/repo/base.py:310
osp.exists()returnsFalsefor a dangling symlink. Consequently, a worktree containing a broken.gitsymlink can still fall through tois_git_dir(curpath)and be accepted as a bare repository if it hasHEAD,objects, andrefs, defeating the malformed-.gitguard above. Useosp.lexists(dotgit)so every non-directory.gitentry blocks fallback.
if osp.exists(dotgit) and not osp.isdir(dotgit):
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 4 out of 4 changed files in this pull request and generated no new comments.
Suppressed comments (3)
git/repo/base.py:310
- When
.gitis a dangling symlink,find_submodule_git_dir()returnsNoneandosp.exists(dotgit)is false, so this guard does not stop discovery. Withsearch_parent_directories=True, the loop can then open an unrelated parent repository, contrary to the malformed.gitrejection described by this change. Useosp.lexists(dotgit)so a present but unresolved non-directory.gitentry blocks fallback.
if osp.exists(dotgit) and not osp.isdir(dotgit):
git/repo/fun.py:130
- The pre-open
st_sizecheck is not sufficient to enforce the 1 MiB Gitfile limit:read_bytes()then reads the entire file without a bound. If a regular file grows or is replaced betweenstatand the read, repository discovery can allocate unbounded input and defeat the size safeguard. Open the file and read at most(1 << 20) + 1bytes, rejecting an over-limit read.
try:
content = os.fsdecode(Path(dotgit).read_bytes()).rstrip("\r\n")
except OSError:
return None
return content[8:] if len(content) >= 9 and content.startswith("gitdir: ") else None
git/repo/fun.py:110
- This branch now treats a nonempty
GIT_OBJECT_DIRECTORYas the repository's object store, butRepodoes not carry that value into construction: its ODB is still rooted atcommon_dir/objectsand the Git wrapper only pinsGIT_DIR/GIT_COMMON_DIR. A repository whose objects exist only inGIT_OBJECT_DIRECTORYcan therefore use a different or nonexistent object database (and a relative value will break when the process cwd changes). Resolve and preserveGIT_OBJECT_DIRECTORYas well, or reject this environment mode consistently.
object_dir = os.getenv("GIT_OBJECT_DIRECTORY")
if object_dir is None:
object_dir = osp.join(common_dir, "objects")
if valid_head and osp.isdir(object_dir) and osp.isdir(osp.join(common_dir, "refs")):
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 4 out of 4 changed files in this pull request and generated no new comments.
Suppressed comments (5)
Previously missed (2) — in code that hasn't changed since the last review.
git/repo/base.py:388
GIT_OBJECT_DIRECTORYis validated during discovery, but it is not pinned in the Git wrapper's environment. When this value is relative andRepo(path)gives the wrapper a differentcwd, discovery checks the path relative to the caller's cwd while later Git commands resolve it relative to the repository's working directory, so object lookups can target a different store. Resolve and propagate this variable along withGIT_DIRandGIT_COMMON_DIR.
if common_dir_env is not None:
self.git.update_environment(GIT_DIR=os.fspath(self.git_dir), GIT_COMMON_DIR=os.fspath(self.common_dir))
elif git_dir_env is not None:
self.git.update_environment(GIT_DIR=os.fspath(self.git_dir))
git/repo/base.py:310
osp.existsfollows symlinks, so a dangling.gitsymlink makes this guard false. If the current directory also has bare-repository markers (or parent search is enabled), discovery can then fall through and accept that directory/parent even though an invalid.gitentry is present, defeating the precedence check above. Uselexistsso dangling metadata is treated as an existing non-directory entry, as is already done forcommondirinfun.py.
if osp.exists(dotgit) and not osp.isdir(dotgit):
git/repo/fun.py:110
- This accepts
GIT_OBJECT_DIRECTORYas the repository's object store, but the constructor still creates the object database fromself.common_dir/objects(base.py:391). A valid repository whose objects exist only in this environment-provided directory will therefore be opened with an unusableGitDB(and an inaccurateGitCmdObjectDB.root_path()); either use the resolved object-directory environment value when constructing the ODB or reject this configuration consistently.
object_dir = os.getenv("GIT_OBJECT_DIRECTORY")
if object_dir is None:
object_dir = osp.join(common_dir, "objects")
if valid_head and osp.isdir(object_dir) and osp.isdir(osp.join(common_dir, "refs")):
git/repo/fun.py:129
- The size check applies only to the result of
os.stat;Path.read_bytes()reopens the path and reads to EOF. A concurrent replacement or growth can therefore bypass the 1 MiB limit and make repository discovery read an arbitrarily large Gitfile. Read from one descriptor with a bound ofstatbuf.st_size(and reject a short read), as Git does.
try:
content = os.fsdecode(Path(dotgit).read_bytes()).rstrip("\r\n")
except (OSError, UnicodeError):
return None
test/test_repo.py:139
Repo.init(path)leaves a validpath/.gitdirectory in place, so even thebarelayout is resolved byfind_submodule_git_dir(path/.git)before the new implicit-bare branch atbase.py:313runs. Add a bare-only directory without.gitand assertRepo(path)discovers it, otherwise this new discovery path has no positive regression coverage.
Repo.init(path).close()
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.
Suppressed comments (8)
git/repo/base.py:391
- The wrapper can run with an absolute
cwdwhile receiving a relativeGIT_DIR. For example,GIT_DIR=gitwith acommondirfile makesworking_dirthe common directory, so Git resolvesGIT_DIR=gitbelow that directory instead of the original repository; the same applies when onlyGIT_COMMON_DIRis set andgit_dircame from a relativepath. NormalizeGIT_DIRbefore storing it in both branches.
if common_dir_env is not None:
self.git.update_environment(GIT_DIR=os.fspath(self.git_dir), GIT_COMMON_DIR=os.fspath(self.common_dir))
elif git_dir_env is not None:
self.git.update_environment(GIT_DIR=os.fspath(self.git_dir))
git/repo/base.py:314
- If
.gitexists as a malformed directory,find_submodule_git_dir(dotgit)returnsNone, but this condition does not stop discovery because the entry is a directory. A worktree containing repository-lookingHEAD,objects, andrefscan then be accepted as a bare repository, defeating the new malformed-.gitprecedence rule;osp.existsalso misses dangling.gitsymlinks. Treat any present but unresolved.gitentry as terminal (usinglexists) before checkingcurpathas a bare repository.
if osp.exists(dotgit) and not osp.isdir(dotgit):
break
git/repo/base.py:307
find_submodule_git_dir()has already resolved a relativegitdir:target againstdirname(d)(lines 147-149). WhenRepois opened with a relativepath,sm_gitpathis therefore relative to the process cwd, but joiningcurpathagain prefixes the worktree a second time and selects a nonexistent/wrong admin directory. Use the helper's resolved path directly here so relative Gitfiles work with relativeRepopaths.
git_dir = osp.normpath(osp.join(curpath, os.fspath(sm_gitpath)))
git/repo/base.py:369
- The commondir is normalized with a different rule than Git: trailing spaces/tabs remain in
common_dirhere even though Git'sget_common_dir_noenv()removes them. Once discovery accepts such a file, this leavesself.common_dirpointing at the wrong path and causes config/object access to diverge from Git; userstrip()here as well.
try:
common_dir = os.fsdecode((Path(self.git_dir) / "commondir").read_bytes()).rstrip("\r\n")
self._common_dir = osp.join(self.git_dir, common_dir)
git/repo/fun.py:130
- Git's
read_gitfile_gently()trims trailing whitespace from the gitdir record, not just CR/LF. With this normalization, a valid.gitfile such asgitdir: /path/to/repo \nis rejected because the target retains the trailing space, so discovery diverges from Git. Strip all trailing whitespace here before extracting the target.
content = os.fsdecode(Path(dotgit).read_bytes()).rstrip("\r\n")
except (OSError, UnicodeError):
return None
return content[8:] if len(content) >= 9 and content.startswith("gitdir: ") else None
git/repo/fun.py:93
get_common_dir_noenv()applies Git's trailing-whitespace trimming to the commondir file. Leaving spaces or tabs incommon_dirmakes this check look in the wrong directory and rejects repositories that Git opens successfully; use the same trimming rule as the laterRepoconstruction path.
common_dir_file = Path(d) / "commondir"
try:
common_dir = os.fsdecode(common_dir_file.read_bytes()).rstrip("\r\n")
except FileNotFoundError:
test/test_repo.py:252
- These assertions execute while both
cwd(tdir)and the patched environment are still active, so relativeGIT_DIR/GIT_OBJECT_DIRECTORYvalues would still resolve correctly even if the wrapper failed to retain absolute paths. Move the repository and GitDB assertions outside both context managers to actually cover the preservation behavior this regression test is intended to protect.
assert repo.git.cat_file("blob", blob_hexsha) == payload.decode()
test/test_repo.py:223
- This assertion is also inside the
cwd(tdir)/environment context, so it does not verify that the relativeGIT_DIRremains usable after construction changes the execution context. Place the assertion after the context managers (as in the preceding environment case) to cover the wrapper path normalization.
assert osp.samefile(repo.git.rev_parse("--absolute-git-dir"), git_dir)
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.
Suppressed comments (3)
git/repo/base.py:396
- Now that
GIT_OBJECT_DIRECTORYis accepted as the active object store and used forrootpath,Repo.alternatesstill reads and writescommon_dir/objects/info/alternates. With objects relocated to the environment-provided directory, that API silently ignores the active alternates file or modifies the wrong repository; derive this path from the resolved object root as well.
rootpath = object_dir_env if object_dir_env is not None else osp.join(self.common_dir, "objects")
git/repo/base.py:313
- Using
osp.existshere misses a dangling.gitsymlink:find_submodule_git_dirreturnsNone, then this guard is skipped and the loop can still acceptcurpathas a bare repository if it hasHEAD/objects/refs. That bypasses the new malformed-.gitno-fallback rule; useosp.lexists(or an equivalentlstatcheck) for this guard.
if osp.exists(dotgit) and not osp.isdir(dotgit):
git/repo/base.py:391
- When
git_dir_envandcommon_dir_envare both unset, this code does not pin the resolvedself.git_dirinto the command wrapper. A valid repository opened from a separate git directory viacore.worktreecan therefore haveself.working_dirset to its worktree while that worktree has no.gitentry; subsequentrepo.gitcommands run withoutGIT_DIRand fail discovery. Preserve the resolvedGIT_DIRfor path-based discovery too.
if common_dir_env is not None:
self.git.update_environment(GIT_DIR=os.fspath(self.git_dir), GIT_COMMON_DIR=os.fspath(self.common_dir))
elif git_dir_env is not None:
self.git.update_environment(GIT_DIR=os.fspath(self.git_dir))
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 4 out of 4 changed files in this pull request and generated no new comments.
Suppressed comments (3)
git/repo/base.py:313
- A dangling
.gitsymlink is a non-directory entry, butosp.exists()follows symlinks and returnsFalsefor it. Sincefind_submodule_git_dir()has already rejected the dangling target, discovery then reachesis_git_dir(curpath)and can still misclassify a worktree withobjects/refs/HEADas a bare repository. Use a lexists-style check for this guard so any present, invalid.gitentry stops fallback.
self._working_tree_dir = curpath
git/repo/base.py:391
- When the common directory comes from a
commondirfile and neitherGIT_COMMON_DIRnorGIT_DIRwas set, neither branch initializes the wrapper environment. For a split bare repository,working_dirbecomesself.common_dir, which may have noHEADor.git, so laterrepo.gitcommands fail repository discovery even though construction succeeded. Pin the resolvedGIT_DIR/GIT_COMMON_DIRwhenever the resolved common directory differs from the git directory, not only when those variables were inherited.
self._working_tree_dir = None
# END working dir handling
self.working_dir: PathLike = self._working_tree_dir or self.common_dir
git/repo/fun.py:127
- Git's
read_gitfile_gently()trims trailing whitespace from the singlegitdir:record, so a valid.gitfile such asgitdir: /path/to/git \nis accepted by Git. This parser only strips CR/LF and returns a target with the trailing space, causingis_git_dir()to reject an otherwise valid worktree; trim the same whitespace set before extracting the target.
with open(dotgit, "rb") as fp:
<!-- agent --> GitPython considered worktree administration and bare-repository signatures before a worktree's real .git entry. Align discovery with Git so .git files and directories win, malformed .git files stop discovery, and candidate git directories validate HEAD plus commondir-backed object and ref storage. This addresses GHSA-239g-whfq-7xj9. Regression coverage compares ambiguous layouts with git rev-parse and rejects invalid HEAD/.git metadata. Git baseline: 15c6308cf7ad276b306aa5b3ababfbdebfb1a917; setup.c setup_git_directory_gently_1(), is_git_directory(), and validate_headref(). Repository validation can use GIT_COMMON_DIR for refs and objects. Preserve the same value on Repo so later config, ref, and object access uses the directory that made discovery succeed. Capture GIT_COMMON_DIR before the first repository config read so bare-state detection uses the same metadata location as discovery. Resolve relative environment values immediately so later working-directory changes cannot retarget the Repo. Review feedback: relative GIT_COMMON_DIR left Git subprocesses resolving GIT_DIR and GIT_COMMON_DIR from a different working directory; malformed commondir data, empty GIT_OBJECT_DIRECTORY, and the linked-worktree signature were also handled inconsistently. Pin the repository environment to resolved paths, reject invalid metadata without consulting the process working directory, and restore HEAD-based linked-worktree detection. Keep the loose HEAD and dangling .git behavior because both match Git setup.c at baseline 15c6308cf7ad276b306aa5b3ababfbdebfb1a917. Review feedback identified that chained or self-referential .git pointers recurse, filesystem-encoded metadata can fail text decoding, and a relative GIT_DIR is not retained for later Git commands. Parse one regular, size-bounded Gitfile exactly once, decode Gitfile and commondir paths with the filesystem codec, and retain the resolved GIT_DIR for subprocesses. This rejects cycles like Git instead of recursing and keeps commands stable after working-directory changes. Review feedback noted that a dangling commondir symlink was treated as absent, allowing local objects and refs to validate the repository. Distinguish a truly missing commondir from a dangling symlink. This follows Git's get_common_dir_noenv(), whose file_exists check uses lstat before attempting to read the entry. Python 3.9 on Windows raised UnicodeDecodeError while the repository-discovery regression test parsed invalid commondir bytes, causing the Python package test (windows, 3.9) check to fail. Treat UnicodeError like an unreadable metadata file in both commondir and gitfile parsing. Invalid bytes now make discovery reject the candidate repository, matching Git's behavior. Review feedback noted that GIT_OBJECT_DIRECTORY made discovery succeed without becoming the Repo ODB root, while a relative value could later be resolved from the Git wrapper's different working directory. Resolve the environment value against the construction directory, use it as the ODB root, and preserve the absolute value for later Git commands. The regression moves the only object store outside the git directory and verifies access through both GitDB and git cat-file after the original environment and current directory are restored. Review feedback identified four setup mismatches: non-missing .git stat failures could fall through to another repository, explicit GIT_DIR could be redirected through a nested .git entry, Gitfile reads were not bounded to the stat-reported size, and alternates ignored GIT_OBJECT_DIRECTORY. Match Git setup.c by bypassing discovery for an environment-selected GIT_DIR, stopping discovery when Gitfile stat fails for reasons other than a missing path, and reading exactly the previously observed Gitfile size. Resolve alternates below the active ODB root so custom object stores remain internally consistent. The commit review noted that treating every present GIT_DIR as explicit broke two documented cases: an empty GIT_DIR must fall back to current-directory discovery, while an empty Repo path must still use a nonempty GIT_DIR. Assisted-by: GPT 5.6 Co-authored-by: GPT 5.6 <codex@openai.com>
Tasks
This section is for Byron only. Models continuing this PR must not add, remove, check, uncheck, rename, or reorder checkboxes here.
Everything below this line was generated by Codex GPT-5.
Created by Codex on behalf of Byron. Byron will review before this is ready to merge.
Advisory
https://github.com/gitpython-developers/GitPython/security/advisories/GHSA-239g-whfq-7xj9
GHSA-239g-whfq-7xj9 reports that repository content can be mistaken for repository metadata when GitPython opens a normal worktree.
Advisory summary
Changes
Git baseline: 15c6308cf7ad276b306aa5b3ababfbdebfb1a917, especially setup.c setup_git_directory_gently_1(), is_git_directory(), validate_headref(), read_gitfile_gently(), and get_common_dir_noenv().
Validation
Commits