getpkg/ARCHITECTURE.md

Architecture Overview

This document provides a detailed technical overview of the getpkg project and its associated tools.

Project Structure

The repository contains multiple tools in the dropshell ecosystem:

• getpkg - The main C++ package manager
• sos - Simple object storage upload utility
• whatsdirty - Git repository status checker for subdirectories
• dehydrate - File to C++ code generator
• bb64 - Base64 encoder/decoder
• gp - Git push utility

getpkg Architecture

Core Components

The main getpkg application is built in C++ using modern C++23 standards with the following key components:

Package Management

• GetbinClient (src/GetbinClient.{hpp,cpp})

  • HTTP client with multi-server support for downloading/uploading packages
  • Implements fallback logic for server failures
  • Progress callback support for downloads/uploads
  • Server-specific and multi-server operations

• PackageMetadata (src/PackageMetadata.{hpp,cpp})

  • Enhanced metadata structure with server source tracking
  • Supports migration from legacy single-server format
  • Stores: name, version, hash, architecture, source server, install date
  • JSON serialization/deserialization
  • Validation methods for all fields

• PackageMetadataManager (src/PackageMetadata.{hpp,cpp})

  • Manages package metadata directory structure
  • Handles legacy format migration
  • Package enumeration and validation

Server Management

• ServerManager (src/ServerManager.{hpp,cpp})
  • Manages multiple package servers with write tokens
  • Server configuration persistence
  • Token management for publishing
  • Server reachability validation

System Integration

• BashrcEditor (src/BashrcEditor.{hpp,cpp})

  • Manages ~/.bashrc_getpkg file modifications
  • Handles PATH updates and bash completions
  • Safe file editing with atomic operations

• DropshellScriptManager (src/DropshellScriptManager.{hpp,cpp})

  • Manages tool installation and configuration
  • Handles setup scripts execution
  • Directory structure management

• MigrationManager (src/MigrationManager.{hpp,cpp})

  • Handles migrations between getpkg versions
  • Legacy format conversions
  • Configuration updates

Common Utilities

Located in src/common/:

• archive_tgz - TAR.GZ archive creation/extraction
• hash - File hashing utilities
• output - Formatted console output
• temp_directory - Temporary directory management
• xxhash - Fast hashing algorithm implementation

Build System

Docker-Based Build

• Uses containerized build environment: gitea.jde.nz/public/dropshell-build-base:latest
• Ensures consistent builds across different host systems
• Static linking for maximum portability

CMake Configuration

• C++23 standard required
• Static executable building (-static linker flags)
• External dependencies:
  • nlohmann_json - JSON parsing
  • CPRStatic - HTTP client library
• Version format: YYYY.MMDD.HHMM (timestamp-based)

Build Scripts

• build.sh - Individual tool build script
• test.sh - Run tests for individual tools
• publish.sh - Publish tool to getpkg.xyz (requires SOS_WRITE_TOKEN)
• buildtestpublish_all.sh - Master script that builds, tests, and publishes all tools

File Locations and Structure

User Installation

• Tool installations: ~/.getpkg/<tool_name>/
• Executable symlinks: ~/.local/bin/getpkg/
• Configuration: ~/.config/getpkg/
  • packages/<tool_name>.json - Package metadata
  • servers.json - Server configuration
• Bash integration: ~/.bashrc_getpkg (sourced by ~/.bashrc)

Repository Structure

getpkg/
├── getpkg/           # Main package manager
│   ├── src/          # Source code
│   ├── test/         # Test suite
│   └── build.sh      # Build script
├── sos/              # Simple object storage
├── whatsdirty/       # Git status checker
├── dehydrate/        # File to C++ converter
├── bb64/             # Base64 utility
├── gp/               # Git push utility
└── buildtestpublish_all.sh  # Master build script

Multi-Server Architecture

getpkg supports multiple package servers with intelligent fallback:

1. Server Configuration

   • Multiple servers can be configured
   • Each server can have an optional write token
   • First server with token becomes default publish target

2. Download Strategy

   • Attempts servers in configured order
   • Falls back to next server on failure
   • Tracks which server provided each package

3. Publishing

   • Requires SOS_WRITE_TOKEN environment variable
   • Publishes to first server with valid token
   • Supports architecture-specific uploads

Security Considerations

• No root access required - installs to user home directory
• Static linking prevents dependency attacks
• Hash verification for downloaded packages
• Token-based authentication for publishing

Testing

• Docker-based test environment for consistency
• Integration tests for package operations
• Unit tests for individual components
• Test artifacts isolated in test_temp/ directory

Development Workflow

1. Local Development

   cd getpkg && ./build.sh              # Build
   cd getpkg && ./test.sh               # Test
   

2. Full Build

   ./buildtestpublish_all.sh            # Build all tools
   

3. Publishing (requires SOS_WRITE_TOKEN)

   export SOS_WRITE_TOKEN="your-token"
   cd getpkg && ./publish.sh            # Publish single tool