Sonic Phoenix#

A multi-phase pipeline that transforms a disorganised local music collection into a pristine, fully-tagged library sorted by <Language>/<Artist>/<Album>/<Artist> - <Title>.<ext>. Works with any language — Hindi, English, Japanese, Spanish, or anything else your collection contains.

The pipeline never deletes audio files. Suspected duplicates are moved to a staging folder for manual review.

Quick Reference#

SituationActionFirst-time setupEnvironment SetupSort a messy music folderRun Phases 1-5 in orderIdentify unknown songsRun Phase 1 (Shazam fingerprinting)Find and remove duplicatesRun Phase 2 (SHA-256 catalog)Fix bad/missing ID3 tagsRun Phase 4 (iTunes enrichment)Sync library to SpotifyRun Phase 6 (requires Spotify app credentials)Add language classification hintsCreate hint files in config/language_hints/Check current configurationpython config.py

Environment Setup#

Prerequisites#

• Python 3.12 (required — shazamio-core only ships wheels for 3.10-3.12)
• Git (to clone the repo)
• FFmpeg (optional — only needed if your collection contains non-MP3 formats like FLAC, OGG, or WMA that Shazam needs to decode)

Installation#

bash

# Clone into your music folder (recommended) or anywhere else
cd /path/to/your/music
git clone https://github.com/drajb/sonic-phoenix.git
cd sonic-phoenix

# Create virtual environment with Python 3.12
python3.12 -m venv .venv

# Activate
source .venv/bin/activate          # macOS / Linux
# .venv\Scripts\activate           # Windows PowerShell
# .venv\Scripts\activate.bat       # Windows cmd

# Install dependencies
pip install -r requirements.txt

Configuration#

Create a .env file in the repo root (copy from .env.example):

dotenv

# REQUIRED — absolute path to the root of your music collection
MUSIC_ROOT=/path/to/your/music

# OPTIONAL — override any of these defaults:
# SORTED_ROOT=/path/to/your/music/Sorted
# DATA_DIR=/path/to/your/music/.data
# SHAZAM_CONCURRENCY=20
# ITUNES_COUNTRIES=US,GB

Verify configuration:

bash

python config.py

This prints all resolved paths and confirms MUSIC_ROOT exists.

Directory Layout After Setup#

text

<MUSIC_ROOT>/
├── sonic-phoenix/           # This repo
│   ├── config.py            # Central configuration (single source of truth)
│   ├── .env                 # Your local overrides (gitignored)
│   ├── 01A_extract_metadata.py
│   ├── 01D_shazam_all_files.py
│   ├── ...
│   └── config/
│       └── language_hints/  # Optional language classification overrides
│           └── examples/    # Templates to copy and customise
├── Sorted/                  # Pipeline output (auto-created)
│   ├── English/
│   │   ├── Adele/
│   │   │   └── 25/
│   │   │       └── Adele - Hello.mp3
│   │   └── ...
│   ├── Hindi/
│   └── <any language langdetect finds>/
├── .data/                   # Machine-generated JSON artifacts (auto-created)
│   ├── metadata_catalog.json
│   ├── shazam_final_results.json
│   ├── catalog.json
│   ├── final_catalog.json
│   └── ...
└── <your unsorted audio files>

The Pipeline#

Happy Path (Minimum Viable Run)#

For a quick, effective sort of your library, run these scripts in order:

bash

python 01A_extract_metadata.py     # Extract ID3/FLAC tags
python 01D_shazam_all_files.py     # Acoustic fingerprint every file via Shazam
python 02A_catalog_music.py        # SHA-256 hash catalog + duplicate detection
python 02D_organize_music.py         # Sort into Language/Artist/Album hierarchy
python 03A_consolidate_by_artist.py --dry-run   # Preview artist folder merges
python 03A_consolidate_by_artist.py              # Execute merges
python 03D_titanium_resort.py      # Final structural enforcement pass
python 04I_polish_and_enrich_v6.py # Enrich tags via iTunes + embed artwork
python 05I_finalize_catalog.py     # Lock the master catalog

Each script is resumable — if interrupted, re-run it and it picks up where it left off.

Phase 1: Extraction and Identification#

ScriptStatusPurpose01A_extract_metadata.pyCANONICALReads existing ID3/FLAC/MP4 tags via Mutagen. Splits files into tagged_files and untagged_files in metadata_catalog.json.01B_shazam_identify.pyUTILITYShazam-identify only the untagged files from 01A.01C_shazam_by_hash.pyUTILITYPropagate a single Shazam match to all byte-identical copies (via hash groups from Phase 2).01D_shazam_all_files.pyCANONICALForce every audio file through Shazam regardless of existing tags. Use this when tags are unreliable. 20-way concurrent by default.01E_test_matching.pyUTILITYSpot-check a handful of files against Shazam to verify the pipeline is working.

When to use 01B vs 01D: If your collection has clean tags on most files, use 01A then 01B (faster — only fingerprints untagged files). If tags are unreliable (generic names like "Track 01", meaningless album fields, garbled metadata), use 01D to fingerprint everything and cross-validate against the acoustic results.

Shazam rate limiting: If you see HTTP 429 errors, lower SHAZAM_CONCURRENCY in your .env (try 5-10).

Phase 2: Cataloguing and Deduplication#

ScriptStatusPurpose02A_catalog_music.pyCANONICALSHA-256 hash every audio file. Writes catalog.json with duplicate groups.02B_analyze_catalog.pyUTILITYAnalyses the hash catalog and prints duplicate statistics. Read-only reporter.02C_organize_files.pyUTILITYOrganises files using the hash catalog and final catalog. Alternative to 02D.02D_organize_music.pyCANONICALThe main sorter. Reads Shazam results and moves files into <SORTED_ROOT>/<Language>/<Artist>/<Album>/.

Phase 3: Consolidation and Structural Cleanup#

ScriptStatusPurpose03A_consolidate_by_artist.pyUTILITYMerge fragmented artist folders (e.g. "Akon feat Eminem" into "Akon"). Supports --dry-run.03D_titanium_resort.pyCANONICALFinal structural enforcer. Any file not matching Language/Artist/Album/ gets re-sorted.03F_reorganize_binary.pyUTILITYResolve near-duplicate files via binary comparison.

Phase 4: Metadata Enrichment#

ScriptStatusPurpose04I_polish_and_enrich_v6.pyCANONICALThe single enrichment entry point. Cleans junk from tags (quality markers, description artifacts, miscellaneous noise), enriches via iTunes Search API (artist, title, album, release date, artwork), fetches synchronized lyrics from LrcLib, embeds cover art into ID3 tags.

iTunes rate limiting: The script rotates queries across country codes (default: US, GB). Add more via ITUNES_COUNTRIES=US,GB,AU,CA in .env.

Phase 5: Finalization#

ScriptStatusPurpose05I_finalize_catalog.pyCANONICALMerges ID3 tags, Shazam results, and NLP language classification into final_catalog.json. This is the read-only master snapshot of the library.

Supporting utilities: 05A (JSON repair), 05B (sanitize results), 05C (confidence audit), 05D (residue purge), 05E/ 05H (empty-folder vacuum).

Phase 6: Spotify Sync (Optional)#

Requires a free Spotify Developer account.

ScriptStatusPurpose06B_spotify_setup.pyCANONICALOAuth handshake — run once to cache the token.06C_spotify_backup.pyCANONICALSnapshot all existing playlists before making changes.06D_spotify_sync_engine.pyCANONICALCreates per-language playlists mirroring your local library. Resumable.06E_spotify_discovery_sync.pyCANONICALCross-references local artists with Spotify listening history and generates genre-based "Essentials" playlists.

Spotify setup:

1. Create an app at https://developer.spotify.com/dashboard

2. Set Redirect URI to http://127.0.0.1:8888/callback

3. Add to .env: dotenv

   SPOTIFY_CLIENT_ID=your_client_id
   SPOTIFY_CLIENT_SECRET=your_client_secret
   
   

4. Run python 06B_spotify_setup.py — browser opens, approve scopes, token is cached.

Language Classification#

The pipeline uses langdetect (NLP) to classify tracks by language automatically. No hardcoded language list — it creates whatever folders your collection needs.

Language Hints (Optional)#

For artists that NLP can't classify from Latin-script titles (e.g. Bollywood artists romanised as "Arijit Singh"), create hint files:

text

config/language_hints/
├── examples/           # Copy and customise these templates
│   ├── English.json
│   ├── Hindi.json
│   └── README.md       # Full schema reference
├── Hindi.json          # Your custom hints (create from examples)
└── Spanish.json

Hint file schema:

json

{
  "artists": ["Arijit Singh", "A.R. Rahman", "Shreya Ghoshal"],
  "dna": ["bollywood", "bhangra", "desi"],
  "keywords": ["ishq", "dil", "pyaar"],
  "lang_codes": ["hi", "pa"]
}

• artists — exact artist names to classify into this language
• dna — substrings to match in folder/file paths
• keywords — substrings to match in track titles
• lang_codes — ISO 639-1 codes that langdetect might return for this language

Configuration Reference#

All configuration lives in config.py and is driven by environment variables (set in .env or your shell).

VariableRequiredDefaultPurposeMUSIC_ROOTYesParent of repo dirRoot of your music collectionSORTED_ROOTNo<MUSIC_ROOT>/SortedDestination for sorted filesDATA_DIRNo<MUSIC_ROOT>/.dataMachine-generated JSON artifactsDUPLICATES_STAGINGNo<repo>/Duplicates_StagingQuarantine for suspected duplicatesUNIDENTIFIED_DIRNo<SORTED_ROOT>/UnidentifiedFiles Shazam couldn't identifyFFMPEG_BINNo<MUSIC_ROOT>/ffmpeg/binPortable FFmpeg path (Windows)SHAZAM_CONCURRENCYNo20Parallel Shazam calls (lower if rate-limited)ITUNES_COUNTRIESNoUS,GBCountry codes for iTunes API rotationSPOTIFY_CLIENT_IDPhase 6 only—Spotify app client IDSPOTIFY_CLIENT_SECRETPhase 6 only—Spotify app client secretSPOTIFY_REDIRECT_URINohttp://127.0.0.1:8888/callbackSpotify OAuth redirect

Supported Audio Formats#

.mp3, .m4a, .flac, .wav, .ogg, .wma, .aac, .opus

Safety Guarantees#

• No deletion. Audio files are never deleted. Duplicates are moved to Duplicates_Staging/ for manual review.
• No writes into the music hierarchy. All working state lives in <DATA_DIR> or the repo directory.
• Resumable. Every script can be interrupted and re-run safely.
• Non-destructive tags. Enrichment writes new tags but does not remove existing valid data.

Troubleshooting#

ProblemFixshazamio-core fails to installYou're on Python 3.13+. Use Python 3.12: python3.12 -m venv .venvShazam returns HTTP 429Lower SHAZAM_CONCURRENCY to 5-10 in .enviTunes returns 403Add more country codes: ITUNES_COUNTRIES=US,GB,AU,CA,DE``MUSIC_ROOT does not existSet MUSIC_ROOT in .env to the absolute path of your music folderTags still show junk after enrichmentRe-run 04I_polish_and_enrich_v6.py — it's idempotentSpotify 403 on playlist creationAdd your Spotify account under "Users and Access" in the Developer Dashboard (Development Mode restriction)langdetect misclassifies a languageCreate a hint file in config/language_hints/ for that languageEmpty artist/album folders after sortingRun 05E_final_cleanup.py or 05H_final_vacuum.py to vacuum empties

Scripts#

Helper scripts that automate common workflows. Run from the repo root.

ScriptPurposeultimate-music-manager/scripts/preflight.shValidates Python 3.12, venv, dependencies, .env, MUSIC_ROOT, FFmpeg. Run before first pipeline execution.ultimate-music-manager/scripts/run-pipeline.shExecutes the happy-path sequence (Phases 1-5) with progress and timing. Supports --skip-shazam, --spotify, --dry-run.ultimate-music-manager/scripts/status.shDashboard showing file counts, language breakdown, data file status, and pipeline progress.

Usage#

bash

# Check environment is ready
bash ultimate-music-manager/scripts/preflight.sh

# Preview what the pipeline will do
bash ultimate-music-manager/scripts/run-pipeline.sh --dry-run

# Run the full pipeline
bash ultimate-music-manager/scripts/run-pipeline.sh

# Run including Spotify sync
bash ultimate-music-manager/scripts/run-pipeline.sh --spotify

# Check pipeline status at any time
bash ultimate-music-manager/scripts/status.sh

Hook Integration#

A safety guard hook prevents accidental execution of destructive utility scripts.

Setup (Claude Code)#

Add to .claude/settings.json:

json

{
  "hooks": {
    "PreToolUse": [{
      "matcher": "Bash",
      "hooks": [{
        "type": "command",
        "command": "./ultimate-music-manager/hooks/safety-guard.sh"
      }]
    }]
  }
}

The hook intercepts commands referencing 05D_force_delete_residue, 05F_final_scrub, total_scrub, or absolute_zero_sort and injects a warning requiring explicit user confirmation. Silent (zero overhead) for all other commands.

See hooks/HOOK.md for details.

References#

Detailed documentation for deeper dives:

DocumentContentsreferences/data-files.mdSchema and lineage for every JSON artifact — which script writes it, which scripts read it, full data flow diagram.references/language-hints-guide.mdHow to create language hint files, full schema, examples for Hindi/Japanese/Spanish, tips for getting classification right.

Design Principles#

1. Config-driven. One config.py module, one .env file. No hardcoded paths anywhere.
2. Language-agnostic. No baked-in language lists. langdetect + optional hint files handle any language.
3. Phased and incremental. Each phase builds on the previous. You can stop after any phase and have a useful result.
4. Auditable. Every move, merge, and enrichment decision is logged to JSON in <DATA_DIR>.
5. Zero-config happy path. Clone into your music folder, create a .env with MUSIC_ROOT, and run.