getpkg/ARCHITECTURE.md

# Architecture Overview

This document provides a detailed technical overview of the getpkg project and its associated tools.

## Project Structure

The repository contains multiple tools in the dropshell ecosystem:

- **getpkg** - The main C++ package manager
- **sos** - Simple object storage upload utility
- **whatsdirty** - Git repository status checker for subdirectories
- **dehydrate** - File to C++ code generator
- **bb64** - Base64 encoder/decoder
- **gp** - Git push utility

## getpkg Architecture

### Core Components

The main getpkg application is built in C++ using modern C++23 standards with the following key components:

#### Package Management
- **`GetbinClient`** (`src/GetbinClient.{hpp,cpp}`)
  - HTTP client with multi-server support for downloading/uploading packages
  - Implements fallback logic for server failures
  - Progress callback support for downloads/uploads
  - Server-specific and multi-server operations

- **`PackageMetadata`** (`src/PackageMetadata.{hpp,cpp}`)
  - Enhanced metadata structure with server source tracking
  - Supports migration from legacy single-server format
  - Stores: name, version, hash, architecture, source server, install date
  - JSON serialization/deserialization
  - Validation methods for all fields

- **`PackageMetadataManager`** (`src/PackageMetadata.{hpp,cpp}`)
  - Manages package metadata directory structure
  - Handles legacy format migration
  - Package enumeration and validation

#### Server Management
- **`ServerManager`** (`src/ServerManager.{hpp,cpp}`)
  - Manages multiple package servers with write tokens
  - Server configuration persistence
  - Token management for publishing
  - Server reachability validation

#### System Integration
- **`BashrcEditor`** (`src/BashrcEditor.{hpp,cpp}`)
  - Manages `~/.bashrc_getpkg` file modifications
  - Handles PATH updates and bash completions
  - Safe file editing with atomic operations

- **`DropshellScriptManager`** (`src/DropshellScriptManager.{hpp,cpp}`)
  - Manages tool installation and configuration
  - Handles setup scripts execution
  - Directory structure management

- **`MigrationManager`** (`src/MigrationManager.{hpp,cpp}`)
  - Handles migrations between getpkg versions
  - Legacy format conversions
  - Configuration updates

### Common Utilities
Located in `src/common/`:
- **`archive_tgz`** - TAR.GZ archive creation/extraction
- **`hash`** - File hashing utilities
- **`output`** - Formatted console output
- **`temp_directory`** - Temporary directory management
- **`xxhash`** - Fast hashing algorithm implementation

## Build System

### Docker-Based Build
- Uses containerized build environment: `gitea.jde.nz/public/dropshell-build-base:latest`
- Ensures consistent builds across different host systems
- Static linking for maximum portability

### CMake Configuration
- C++23 standard required
- Static executable building (`-static` linker flags)
- External dependencies:
  - nlohmann_json - JSON parsing
  - CPRStatic - HTTP client library
- Version format: `YYYY.MMDD.HHMM` (timestamp-based)

### Build Scripts
- **`build.sh`** - Individual tool build script
- **`test.sh`** - Run tests for individual tools
- **`publish.sh`** - Publish tool to getpkg.xyz (requires SOS_WRITE_TOKEN)
- **`buildtestpublish_all.sh`** - Master script that builds, tests, and publishes all tools

## File Locations and Structure

### User Installation
- **Tool installations**: `~/.getpkg/<tool_name>/`
- **Executable symlinks**: `~/.local/bin/getpkg/`
- **Configuration**: `~/.config/getpkg/`
  - `packages/<tool_name>.json` - Package metadata
  - `servers.json` - Server configuration
- **Bash integration**: `~/.bashrc_getpkg` (sourced by ~/.bashrc)

### Repository Structure
```
getpkg/
├── getpkg/           # Main package manager
│   ├── src/          # Source code
│   ├── test/         # Test suite
│   └── build.sh      # Build script
├── sos/              # Simple object storage
├── whatsdirty/       # Git status checker
├── dehydrate/        # File to C++ converter
├── bb64/             # Base64 utility
├── gp/               # Git push utility
└── buildtestpublish_all.sh  # Master build script
```

## Multi-Server Architecture

getpkg supports multiple package servers with intelligent fallback:

1. **Server Configuration**
   - Multiple servers can be configured
   - Each server can have an optional write token
   - First server with token becomes default publish target

2. **Download Strategy**
   - Attempts servers in configured order
   - Falls back to next server on failure
   - Tracks which server provided each package

3. **Publishing**
   - Requires SOS_WRITE_TOKEN environment variable
   - Publishes to first server with valid token
   - Supports architecture-specific uploads

## Security Considerations

- No root access required - installs to user home directory
- Static linking prevents dependency attacks
- Hash verification for downloaded packages
- Token-based authentication for publishing

## Testing

- Docker-based test environment for consistency
- Integration tests for package operations
- Unit tests for individual components
- Test artifacts isolated in `test_temp/` directory

## Development Workflow

1. **Local Development**
   ```bash
   cd getpkg && ./build.sh              # Build
   cd getpkg && ./test.sh               # Test
   ```

2. **Full Build**
   ```bash
   ./buildtestpublish_all.sh            # Build all tools
   ```

3. **Publishing** (requires SOS_WRITE_TOKEN)
   ```bash
   export SOS_WRITE_TOKEN="your-token"
   cd getpkg && ./publish.sh            # Publish single tool
   ```