Store relative paths in the DB to make beets library portable

[snejus]

Migrate Item & Album Paths to Library-Relative Storage

Fixes: #133

Core Problem Solved

Before: Beets stored absolute file system paths in SQLite (e.g. /home/user/Music/Artist/album/track.mp3). This made library databases non-portable — moving the music directory or sharing a database across machines broke all path references.

After: Paths are stored relative to the music directory (e.g. Artist/album/track.mp3), and expanded back to absolute paths transparently on read. The database is now portable.

---

Architecture Changes

1. Context Variable for Music Directory (beets/context.py)

A new contextvars.ContextVar (_music_dir_var) holds the active music directory, set once during Library.__init__ via context.set_music_dir(). This avoids passing library.directory through every call stack.

Library.__init__()
    └─ context.set_music_dir(self.directory)   ← single write point
           ↓
    PathType.to_sql()  / PathType.from_sql()   ← read at DB layer
    pathutils.normalize_path_for_db()
    pathutils.expand_path_from_db()

2. Path Relativization Moved to the DB Layer (beets/dbcore/)

Previously, path conversion lived in Item._setitem / Item.__getitem__ — model-specific overrides. It is now pushed down into PathType.to_sql / PathType.from_sql in dbcore/types.py, through two helpers in the new beets/dbcore/pathutils.py:

Helper	Direction	Behaviour
normalize_path_for_db(path)	write	Strips music dir prefix → relative path
expand_path_from_db(path)	read	Prepends music dir → absolute path

All models using PathType (currently Item and Album) benefit automatically — no per-model overrides required.

3. PathQuery Updated for Relative Storage (beets/dbcore/query.py)

Queries like path:/home/user/Music/Artist now normalize the search term to its relative form before hitting the database, so SQL comparisons match stored values correctly. Both col_clause (SQL path) and match (in-memory path) use normalize_path_for_db.

4. One-Time Database Migration (RelativePathMigration)

Existing absolute paths in path and artpath columns are migrated on startup:

Library startup
    └─ _migrate()
           └─ RelativePathMigration._migrate_data(Item, Album)
                  └─ _migrate_field("path")
                  └─ _migrate_field("artpath")
                         ↓
                  Reads rows where field starts with b"/"
                  Writes os.path.relpath(path, music_dir) in batches

self.directory assignment was moved before super().__init__() in Library.__init__ so the migration can access the music dir when it runs.

5. Context Propagation to Background Threads

The music dir context variable must be available in worker threads (pipeline stages, replaygain pool). Two propagation points were added:

• beets/util/pipeline.py: Pipeline.run_parallel() snapshots the calling context with contextvars.copy_context() and passes a per-thread copy to each PipelineThread. Each stage coroutine is invoked via ctx.run(...).
• beetsplug/replaygain.py: Pool workers and their callbacks are wrapped in ctx.run(...) so expand_path_from_db works correctly inside the process pool.
• beets/util/__init__.py: par_map similarly propagates context into its thread pool workers.

---

Data Flow: Read & Write Path

item.path = "/home/user/Music/Artist/track.mp3"   ← absolute on write
      ↓
PathType.to_sql()
      ↓
normalize_path_for_db()  →  b"Artist/track.mp3"   ← stored in SQLite

item.path                                           ← absolute on read
      ↑
PathType.from_sql()
      ↑
expand_path_from_db()    ←  b"Artist/track.mp3"   ← fetched from SQLite

---

Key Invariants

• Public API unchanged: item.path always returns an absolute bytes path.
• Raw DB value is relative: direct SQL reads return the relative form (tests assert both).
• Paths outside the music dir are stored as-is (e.g. IPFS paths — see beetsplug/ipfs.py fix).
• Migration is idempotent: rows already relative (no leading /) are skipped.

[github-actions]

Thank you for the PR! The changelog has not been updated, so here is a friendly reminder to check if you need to add an entry.

[codecov]

Codecov Report

❌ Patch coverage is 91.42857% with 12 lines in your changes missing coverage. Please review.
✅ Project coverage is 69.85%. Comparing base (72b4c77) to head (318f2fd).
✅ All tests successful. No failed tests found.

Files with missing lines	Patch %	Lines
beetsplug/ipfs.py	66.66%	2 Missing and 1 partial ⚠️
beets/dbcore/pathutils.py	91.30%	1 Missing and 1 partial ⚠️
beets/dbcore/query.py	80.00%	1 Missing and 1 partial ⚠️
beets/library/models.py	60.00%	1 Missing and 1 partial ⚠️
beets/util/pipeline.py	90.47%	1 Missing and 1 partial ⚠️
beetsplug/replaygain.py	87.50%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #6460      +/-   ##
==========================================
+ Coverage   69.74%   69.85%   +0.11%     
==========================================
  Files         145      147       +2     
  Lines       18495    18596     +101     
  Branches     3007     3021      +14     
==========================================
+ Hits        12899    12991      +92     
- Misses       4967     4972       +5     
- Partials      629      633       +4     

Files with missing lines	Coverage Δ
beets/context.py	100.00% <100.00%> (ø)
beets/dbcore/db.py	94.16% <100.00%> (+<0.01%)	⬆️
beets/dbcore/types.py	96.05% <100.00%> (+0.01%)	⬆️
beets/library/library.py	94.36% <100.00%> (+0.71%)	⬆️
beets/library/migrations.py	96.19% <100.00%> (+1.12%)	⬆️
beets/util/__init__.py	79.10% <100.00%> (+0.08%)	⬆️
beetsplug/replaygain.py	60.98% <87.50%> (+0.32%)	⬆️
beets/dbcore/pathutils.py	91.30% <91.30%> (ø)
beets/dbcore/query.py	91.09% <80.00%> (-0.25%)	⬇️
beets/library/models.py	86.88% <60.00%> (-0.22%)	⬇️
... and 2 more

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[Copilot]

Pull request overview

PR make beets store item/album path in DB as library-relative, then expand back to absolute on read. Goal: portable DB when music folder move.

Changes:

• Add context var for active music dir, used by DB path encode/decode.
• Move path normalize/expand into dbcore types + query layer, plus add migration to rewrite old rows.
• Propagate context into thread pools/pipeline; update tests to expect absolute paths on public API.

Reviewed changes

Copilot reviewed 19 out of 19 changed files in this pull request and generated 9 comments.

Show a summary per file

File	Description
beets/context.py	Add context var for music dir.
beets/library/library.py	Set music dir context early; register new migration.
beets/dbcore/pathutils.py	New helpers to normalize/expand paths for DB.
beets/dbcore/types.py	PathType encode/decode moved into DB type layer.
beets/dbcore/query.py	PathQuery now compares using DB-relative representation.
beets/library/models.py	Adjust non-PathQuery path field queries to match stored relative value.
beets/library/migrations.py	Add RelativePathMigration; add chunk size constant usage.
beets/dbcore/db.py	Add base migration chunk size constant.
beets/util/pipeline.py	Copy context into pipeline worker threads.
beets/util/__init__.py	Propagate context into par_map thread pool.
beetsplug/replaygain.py	Wrap pool work + callbacks in copied context.
beetsplug/ipfs.py	Create temp lib with /ipfs/ dir; store item paths relative to that root.
beets/ui/__init__.py	Override library DB path from BEETS_LIBRARY env var.
test/test_library.py	Update path expectations; assert raw stored relative paths.
test/test_query.py	Add query test for absolute input path.
test/library/test_migrations.py	Add migration test for absolute->relative rewrite.
test/ui/test_ui.py	Update ls -p output expectations to absolute paths.
test/ui/commands/test_list.py	Update $path output expectations to absolute paths.
test/plugins/test_ipfs.py	Normalize expected /ipfs/... path string.
test/test_files.py	Remove one remove/prune test (no replacement in this PR).

[Copilot]

Pull request overview

Copilot reviewed 22 out of 22 changed files in this pull request and generated 4 comments.

[Copilot]

Pull request overview

Copilot reviewed 22 out of 22 changed files in this pull request and generated 2 comments.

[Copilot]

grug worry normalize_path_for_db do string prefix check before path normalized. If caller pass path like "/music/../other/file.mp3", helper think it child of music_dir and store "../other/file.mp3". Later expand make absolute outside library, break invariant "outside music dir stored as-is". Fix: normalize/abspath path (and music_dir) before _is_same_path_or_child/relpath, or use util.normpath on inputs inside normalize_path_for_db.

Suggested change

	if not path or not os.path.isabs(path):
	return path
	music_dir = context.get_music_dir()
	if not music_dir:
	return path
	if _is_same_path_or_child(path, music_dir):
	return os.path.relpath(path, music_dir)
	if not path:
	return path
	music_dir = context.get_music_dir()
	if not music_dir:
	return path
	# Normalize paths before checking directory relationship to avoid
	# misclassifying paths with ".." components as children of music_dir.
	normalized_path = util.normpath(path)
	normalized_music_dir = util.normpath(music_dir)
	if not os.path.isabs(normalized_path):
	return path
	if _is_same_path_or_child(normalized_path, normalized_music_dir):
	return os.path.relpath(normalized_path, normalized_music_dir)

[Copilot]

Pull request overview

Copilot reviewed 23 out of 23 changed files in this pull request and generated 4 comments.

[Copilot]

grug see Library._fetch set music_dir only while parsing query. But Results build Item/Album later when iterating, and PathType.from_sql need music_dir then. If other Library made after this one, context reset and paths expand against wrong root. grug think wrap whole Results materialization/iteration in this library music_dir_context (or make Results/_awaken run under stored context) so paths always expand with right library.