getpkg/.kiro/specs/multi-server-support/design.md

Design Document

Overview

This design extends getpkg to support multiple package servers while maintaining full backward compatibility. The solution introduces a server configuration system, updates the client architecture to handle multiple servers, and reorganizes package metadata storage. The design prioritizes minimal disruption to existing functionality while providing powerful multi-server capabilities.

Architecture

High-Level Architecture

graph TB
    CLI[CLI Commands] --> SM[ServerManager]
    CLI --> PM[PackageManager]
    PM --> SM
    PM --> GC[GetbinClient]
    SM --> CF[servers.json]
    PM --> PF[packages/*.json]
    GC --> S1[Server 1]
    GC --> S2[Server 2]
    GC --> SN[Server N]

Server Management Flow

sequenceDiagram
    participant User
    participant CLI
    participant ServerManager
    participant Config
    
    User->>CLI: getpkg server add example.com
    CLI->>ServerManager: addServer("example.com")
    ServerManager->>Config: load servers.json
    ServerManager->>ServerManager: validate URL
    ServerManager->>Config: save updated servers.json
    ServerManager->>CLI: success confirmation
    CLI->>User: Server added successfully

Package Installation Flow

sequenceDiagram
    participant User
    participant CLI
    participant PackageManager
    participant GetbinClient
    participant Server1
    participant Server2
    
    User->>CLI: getpkg install tool
    CLI->>PackageManager: install("tool")
    PackageManager->>GetbinClient: download("tool", servers[0])
    GetbinClient->>Server1: GET /object/tool:arch
    alt Package found
        Server1-->>GetbinClient: 200 + package data
        GetbinClient-->>PackageManager: success
    else Package not found
        Server1-->>GetbinClient: 404
        GetbinClient->>Server2: GET /object/tool:arch
        Server2-->>GetbinClient: 200 + package data
        GetbinClient-->>PackageManager: success
    end
    PackageManager->>PackageManager: install package
    PackageManager->>CLI: installation complete

Components and Interfaces

ServerManager Class

Purpose: Manages server configuration, write tokens, and provides server list to other components.

Interface:

class ServerManager {
public:
    ServerManager();
    
    // Server management
    bool addServer(const std::string& serverUrl, const std::string& writeToken = "");
    bool removeServer(const std::string& serverUrl);
    std::vector<std::string> getServers() const;
    std::string getDefaultServer() const;
    std::string getDefaultPublishServer() const;  // First server with write token
    
    // Token management
    bool setWriteToken(const std::string& serverUrl, const std::string& token);
    std::string getWriteToken(const std::string& serverUrl) const;
    bool hasWriteToken(const std::string& serverUrl) const;
    std::vector<std::string> getServersWithTokens() const;
    
    // Configuration
    bool loadConfiguration();
    bool saveConfiguration();
    void ensureDefaultConfiguration();
    
    // Migration
    bool migrateFromLegacy();
    
private:
    std::vector<ServerConfig> servers_;
    std::filesystem::path configPath_;
    
    bool validateServerUrl(const std::string& url) const;
    bool isServerReachable(const std::string& url) const;
    ServerConfig* findServer(const std::string& url);
};

Enhanced GetbinClient Class

Purpose: Extended to support multiple servers with fallback logic.

Interface Changes:

class GetbinClient {
public:
    GetbinClient(const std::vector<std::string>& servers);
    
    // Existing methods with server selection
    bool download(const std::string& toolName, const std::string& arch, 
                  const std::string& outPath, ProgressCallback progressCallback = nullptr);
    bool downloadFromServer(const std::string& serverUrl, const std::string& toolName, 
                           const std::string& arch, const std::string& outPath, 
                           ProgressCallback progressCallback = nullptr);
    
    // Server-specific operations
    bool upload(const std::string& serverUrl, const std::string& archivePath, 
                std::string& outUrl, std::string& outHash, const std::string& token,
                ProgressCallback progressCallback = nullptr);
    bool getHash(const std::string& serverUrl, const std::string& toolName, 
                 const std::string& arch, std::string& outHash);
    
    // Multi-server operations
    bool findPackageServer(const std::string& toolName, const std::string& arch, 
                          std::string& foundServer) const;
    
private:
    std::vector<std::string> servers_;
    std::string buildUrl(const std::string& serverUrl, const std::string& endpoint) const;
};

PackageMetadata Structure

Purpose: Enhanced metadata structure to track server source.

Structure:

struct PackageMetadata {
    std::string name;
    std::string version;
    std::string hash;
    std::string arch;
    std::string sourceServer;  // New field
    std::string installDate;   // New field for better tracking
    
    // Serialization
    nlohmann::json toJson() const;
    static PackageMetadata fromJson(const nlohmann::json& j);
    
    // Migration support
    static PackageMetadata fromLegacyJson(const nlohmann::json& j, const std::string& defaultServer);
};

Migration Manager

Purpose: Handles migration from single-server to multi-server configuration.

Interface:

class MigrationManager {
public:
    MigrationManager();
    
    bool needsMigration() const;
    bool performMigration();
    
private:
    bool migrateServerConfiguration();
    bool migratePackageMetadata();
    bool movePackageFiles();
    bool updatePackageMetadata();
    
    std::filesystem::path oldConfigDir_;
    std::filesystem::path newConfigDir_;
    std::filesystem::path packagesDir_;
};

Data Models

Server Configuration Format

File: ~/.config/getpkg/servers.json

{
  "version": "1.0",
  "servers": [
    {
      "url": "getpkg.xyz",
      "name": "Official getpkg Registry",
      "default": true,
      "writeToken": "",
      "added": "2024-01-15T10:30:00Z"
    },
    {
      "url": "packages.example.com",
      "name": "Example Corporate Registry",
      "default": false,
      "writeToken": "abc123token456",
      "added": "2024-01-16T14:20:00Z"
    }
  ],
  "lastUpdated": "2024-01-16T14:20:00Z"
}

Enhanced Package Metadata Format

File: ~/.config/getpkg/packages/<tool_name>.json

{
  "name": "example-tool",
  "version": "2024.0115.1430",
  "hash": "1234567890123456",
  "arch": "x86_64",
  "sourceServer": "getpkg.xyz",
  "installDate": "2024-01-15T14:30:00Z",
  "lastUpdated": "2024-01-15T14:30:00Z"
}

Directory Structure Changes

~/.config/getpkg/
├── servers.json              # New: Server configuration with embedded tokens
├── packages/                 # New: Package metadata directory
│   ├── tool1.json
│   ├── tool2.json
│   └── ...
└── getpkg.xyz/              # Legacy: Will be migrated to servers.json
    └── write_token.txt       # Legacy: Will be migrated

Error Handling

Server Connectivity Issues

1. Network Failures: Graceful fallback to next server in list
2. Invalid Responses: Clear error messages with server identification
3. Authentication Failures: Server-specific error handling with token guidance

Configuration Corruption

1. Invalid JSON: Automatic backup and reset to default configuration
2. Missing Files: Automatic creation with default settings
3. Permission Issues: Clear error messages with resolution steps

Migration Failures

1. Partial Migration: Rollback capability with clear status reporting
2. File Conflicts: Safe handling with backup creation
3. Metadata Corruption: Individual file recovery without breaking entire system

Testing Strategy

Unit Tests

1. ServerManager: Configuration loading, validation, server management
2. GetbinClient: Multi-server communication, fallback logic
3. PackageMetadata: Serialization, migration, validation
4. MigrationManager: Legacy data handling, file operations

Integration Tests

1. End-to-End Installation: Multi-server package discovery and installation
2. Server Management: Add/remove servers with real configuration
3. Migration Testing: Legacy to new format conversion
4. Publish/Unpublish: Server-specific operations

Compatibility Tests

1. Backward Compatibility: Existing installations continue working
2. Legacy Format: Old package files are properly migrated
3. Default Behavior: No configuration changes for existing users

Implementation Phases

Phase 1: Core Infrastructure

• Implement ServerManager class
• Create server configuration format
• Add basic server validation

Phase 2: Client Enhancement

• Extend GetbinClient for multi-server support
• Implement fallback logic
• Add server-specific operations

Phase 3: Package Management

• Update package metadata format
• Implement packages directory structure
• Add server tracking to installations

Phase 4: Migration System

• Create MigrationManager
• Implement automatic migration
• Add backward compatibility layer

Phase 5: CLI Integration

• Add server management commands
• Update existing commands for multi-server
• Implement server selection options

Phase 6: Testing and Polish

• Comprehensive testing suite
• Error handling refinement
• Documentation updates