Catalogizer - Advanced Multi-Protocol Media Collection Management System

Catalogizer is a comprehensive media collection management system that automatically detects, categorizes, and organizes your media files across multiple storage protocols including SMB, FTP, NFS, WebDAV, and local filesystem. It provides real-time monitoring, advanced analytics, and a modern web interface for managing your entire media library.

🚀 Features

Core Capabilities

• Automated Media Detection: Identifies 50+ media types including movies, TV shows, music, games, software, documentaries, and more
• Multi-Protocol Support: Works with SMB, FTP, NFS, WebDAV, and local filesystem protocols
• Real-time Monitoring: Continuously monitors storage sources for changes and updates metadata automatically
• Protocol Resilience: Handles temporary disconnections gracefully with automatic reconnection and offline caching
• Advanced Analytics: Comprehensive statistics, growth trends, and quality analysis
• Modern Web Interface: React-based responsive UI with real-time updates
• Secure Authentication: JWT-based auth with role-based access control
• External Metadata Integration: Fetches data from TMDB, IMDB, TVDB, MusicBrainz, Spotify, Steam, and more
• Quality Analysis: Automatic quality detection and version tracking
• Encrypted Database: SQLCipher for secure data storage
• PDF Conversion Service: Convert PDF documents to images, text, or HTML formats
• Favorites Export/Import: Export and import favorites in JSON and CSV formats with metadata
• Cloud Storage Sync: Synchronize files with Amazon S3, Google Cloud Storage, or local folders
• Advanced Reporting: Generate professional PDF reports with charts and analytics
• NFS Support: Full NFS mounting and file operations for macOS systems

Technical Highlights

• Go Backend: High-performance REST API with Gin framework
• React Frontend: Modern TypeScript React application with Tailwind CSS
• Real-time Updates: WebSocket integration for live data synchronization
• Resilient Architecture: Handles temporary SMB disconnections gracefully
• Scalable Design: Modular architecture supporting multiple media sources

📋 Table of Contents

• Architecture Overview
• Modular Architecture (Submodules)
• Installation & Setup
• Configuration
• SMB Resilience & Offline Handling
• API Documentation
• Frontend Documentation
• Database Schema
• Media Detection Engine
• External Providers
• Real-time Monitoring
• Security & Authentication
• Security Testing
• Deployment
• Troubleshooting
• Contributing

🏗️ Architecture Overview

┌─────────────────┐    ┌─────────────────┐    ┌─────────────────┐
│   React Web    │    │   Go REST API   │    │ SQLCipher DB    │
│   Application  │◄──►│     Server      │◄──►│   (Encrypted)   │
└─────────────────┘    └─────────────────┘    └─────────────────┘
          │                       │                       │
          │                       │                       │
          ▼                       ▼                       ▼
┌─────────────────┐    ┌─────────────────┐    ┌─────────────────┐
│   WebSocket     │    │ Media Detection │    │  External APIs  │
│   Real-time     │    │     Engine      │    │ TMDB, IMDB, etc │
│    Updates      │    └─────────────────┘    └─────────────────┘
└─────────────────┘             │
                                 ▼
                     ┌─────────────────┐
                     │ Multi-Protocol  │
                     │ File System     │
                     │   Clients       │
                     └─────────────────┘
                               │
                ┌──────────────┼──────────────┐
                │              │              │
        ┌───────▼─────┐ ┌──────▼─────┐ ┌──────▼─────┐
        │ SMB Sources │ │ FTP/NFS    │ │ WebDAV/Local│
        │ Monitoring  │ │ Sources     │ │ Sources     │
        │ (Resilient) │ │ Monitoring  │ │ Monitoring  │
        └─────────────┘ └─────────────┘ └─────────────┘

System Components

1. catalog-api: Go-based REST API server

   • Authentication and authorization
   • Media detection and analysis
   • Multi-protocol file system monitoring with resilience
   • External metadata integration
   • WebSocket server for real-time updates

2. catalog-web: React TypeScript web application

   • Modern responsive UI
   • Real-time data synchronization
   • Advanced search and filtering
   • Analytics dashboard
   • User management interface

3. Database Layer: SQLCipher encrypted SQLite

   • Media metadata storage
   • User and session management
   • Configuration and settings

Modular Architecture (Submodules)

All generic, reusable functionality has been extracted into independent modules registered as git submodules. Each module follows the

Go Modules (21):

TypeScript Modules (2):

To initialize submodules after cloning:

- Analysis results and statistics

Supported Protocols

Catalogizer supports multiple file system protocols for maximum flexibility:

• SMB/CIFS: Windows file sharing with automatic reconnection and resilience
• FTP/FTPS: File Transfer Protocol with secure variants
• NFS: Network File System with automatic mounting
• WebDAV: HTTP-based file access over web protocols
• Local Filesystem: Direct access to local storage

Each protocol is abstracted through a common interface, allowing seamless switching and future protocol additions.

🛠️ Installation & Setup

Prerequisites

• Go 1.21+ (for backend)
• Node.js 18+ and npm/yarn (for frontend)
• SQLCipher (for encrypted database)
• Git (for source control)

Backend Setup

1. Clone the repository:

2. Install Go dependencies:

3. Set up environment variables:

4. Initialize the database:

5. Run the API server:

Frontend Setup

1. Navigate to frontend directory:

2. Install dependencies:

3. Set up environment variables:

4. Start development server:

5. Access the application:

   • Frontend: http://localhost:3000
   • API: http://localhost:8080
   • API Documentation: http://localhost:8080/swagger/index.html

⚙️ Configuration

Environment Variables

Backend (.env)

Frontend (.env.local)

🔌 SMB Resilience & Offline Handling

Catalogizer is designed to handle temporary SMB source disconnections gracefully, ensuring uninterrupted service even when network storage becomes unavailable.

Resilience Features

🔄 Automatic Reconnection

• Exponential Backoff: Intelligent retry strategy that reduces load during outages
• Circuit Breaker Pattern: Prevents cascade failures and enables fast recovery
• Health Monitoring: Continuous connection health checks with real-time alerts

💾 Offline Caching

🔧 Failure Recovery Process

1. Detection: System detects SMB connection failure
2. Circuit Breaker: Opens to prevent further failed attempts
3. Offline Mode: Activates local cache to serve requests
4. Background Retry: Exponential backoff reconnection attempts
5. Recovery: Gradual transition back to normal operation
6. Synchronization: Cache sync when connection is restored

📊 Real-time Status Monitoring

Connection States

Configuration

Recovery Strategies

Automatic Recovery

• Health Checks: Every 60 seconds by default
• Exponential Backoff: 30s, 60s, 120s, 300s, 600s
• Circuit Breaker: Half-open testing after timeout
• Cache Synchronization: Automatic when reconnected

Manual Recovery

Benefits

✅ Uninterrupted Service: Users can browse and search even when SMB sources are down ✅ Automatic Recovery: No manual intervention required for temporary outages ✅ Data Integrity: Changes are queued and synchronized when connection is restored ✅ Performance: Circuit breaker prevents slowdowns from failed connection attempts ✅ Monitoring: Real-time visibility into connection health and recovery status

For detailed technical documentation, see SMB Resilience Architecture.

📱 Client Applications

Catalogizer provides comprehensive client applications for all major platforms:

🛠️ Installation Wizard

• Desktop Installation Tool: Cross-platform SMB configuration wizard
• Network Discovery: Automatic SMB device scanning and discovery
• Configuration Management: Visual SMB source setup with validation
• User-Friendly Interface: Step-by-step wizard for easy configuration
• File Operations: Load/save configuration files with native dialogs
• Real-time Testing: Live SMB connection validation and feedback

Component	Coverage	Tests	Status
React Components			✅
Context Management			✅
Service Layer			✅
Tauri Backend			✅

🤖 Android Mobile & TV

• Modern Android Architecture: MVVM with manual dependency injection
• Jetpack Compose UI: Material Design 3 theming
• Offline Support: Room database with automatic sync
• Media Playback: ExoPlayer integration
• Android TV: Leanback UI with D-pad navigation

🖥️ Desktop Applications

• Cross-Platform: Windows, macOS, Linux support
• Tauri Framework: Rust backend with React frontend
• Native Performance: System integration and file access
• Auto-Updates: Built-in update mechanism

🌐 Web Interface

• Responsive Design: Works on all screen sizes
• Real-time Updates: WebSocket synchronization
• Progressive Web App: Offline capabilities

🚀 Quick Installation

Option 1: Installation Wizard (Recommended)

Use the graphical installation wizard for easy SMB configuration:

Installation Wizard Features:

• 🔍 Network Discovery: Automatically finds SMB devices
• ⚙️ Visual Configuration: Step-by-step setup wizard
• 🧪 Connection Testing: Real-time SMB validation
• 💾 File Management: Save/load configuration files
• 📊 Test Coverage: 93% coverage, 30/30 tests passing

Option 2: Automated Script

Use the automated installer to set up the complete Catalogizer ecosystem:

Installation Modes

🐳 Docker Deployment

Quick Start

Production Deployment

Service Architecture

📋 Environment Configuration

Server Environment (.env)

Override Configuration

🏗️ Build & Deploy Clients

Android Deployment

Desktop Deployment

Cross-Platform API Client

🔧 Development Setup

Prerequisites

• Docker & Docker Compose
• Node.js 18+ (for web/desktop clients)
• Android Studio (for Android development)
• Rust & Tauri CLI (for desktop development)

Development Environment

Client Development

📊 Monitoring & Health Checks

Built-in Health Endpoints

Monitoring Stack (Optional)

Log Management

🔐 Security Configuration

SSL/TLS Setup

Authentication

• JWT-based authentication with refresh tokens
• Role-based access control (Admin, User, Read-only)
• Session management with Redis
• OAuth2 integration support

Database Security

• SQLCipher encryption for SQLite (standalone)
• PostgreSQL with encrypted connections (Docker)
• Automated backups with encryption
• Point-in-time recovery support

🚀 Deployment Strategies

Rolling Deployment

Blue-Green Deployment

Canary Deployment

📚 Documentation

• API Documentation: Complete REST API reference
• Deployment Guide: Production deployment
• Architecture Overview: System design
• Security Guide: Security best practices
• Build System: Release build system with versioning
• Troubleshooting: Common issues and solutions

🆘 Support & Troubleshooting

Common Issues

Docker Issues

Permission Issues

Database Issues

Performance Tuning

Getting Help

• 📖 Wiki
• 🐛 Issue Tracker
• 💬 Discussions
• 📧 Email: support@catalogizer.dev

🧪 Testing

Security Testing

Catalogizer includes comprehensive security testing using industry-standard tools:

SonarQube Code Quality Analysis (Freemium)

• Version: Community Edition (Free)
• Purpose: Static code analysis for bugs, vulnerabilities, and code smells
• Coverage: All languages (Go, JavaScript/TypeScript, Kotlin)
• Integration: Mandatory in CI/CD pipeline
• Reports: Available at
  reports/sonarqube-report.json
• Setup:
  SONAR_TOKEN
  from https://sonarcloud.io (free tier)

Snyk Security Scanning (Freemium)

• Version: Free tier with unlimited private repos
• Purpose: Dependency vulnerability scanning and SAST (Static Application Security Testing)
• Coverage: All project modules and dependencies
• Integration: Mandatory in CI/CD pipeline
• Reports: Available at
  reports/snyk-*-results.json
• Setup:
  SNYK_TOKEN
  from https://snyk.io/account (free tier)

Additional Security Tools

• Trivy: Container and filesystem vulnerability scanning
• OWASP Dependency Check: Third-party dependency analysis

Running Tests

Full Test Suite (Including Security)

Prerequisites for Security Testing

1. Setup Freemium Accounts: Run
   ./scripts/setup-freemium-tokens.sh
2. SonarQube: Free account at https://sonarcloud.io +
   SONAR_TOKEN
3. Snyk: Free account at https://snyk.io +
   SNYK_TOKEN
4. Docker: Required for running security services
5. Environment Variables: Set tokens in environment or
   .env
   file

Environment Setup

Test Reports

All test results are stored in the

• comprehensive-security-report.html
  - Main security report
• sonarqube-report.json
  - Code quality analysis
• snyk-*-results.json
  - Vulnerability scans
• trivy-results.json
  - Container scans

Quality Gates

• SonarQube: Quality gate must pass (no critical issues)
• Snyk: No high or critical severity vulnerabilities
• Test Coverage: Minimum 80% for all modules
• Zero Defects: All tests must pass with 100% success rate

🤝 Contributing

We welcome contributions! Please see our Contributing Guide for details.

Development Workflow

1. Fork the repository
2. Create a feature branch
3. Set up development environment:
   ./scripts/install.sh --mode=development
4. Make your changes
5. Run tests:
   ./scripts/run-tests.sh
6. Submit a pull request

Code Standards

• Go: gofmt, golint, go vet
• TypeScript/React: ESLint, Prettier
• Kotlin: ktlint, detekt
• Rust: rustfmt, clippy

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

🙏 Acknowledgments

• Gin - HTTP web framework for Go
• React - JavaScript library for building user interfaces
• Tauri - Cross-platform desktop applications
• Jetpack Compose - Modern UI toolkit for Android
• TMDB - Movie and TV show metadata
• MusicBrainz - Open music encyclopedia

Catalogizer - Organize your media collection with intelligence and style. 🎬📺🎵🎮