Add 1 and update 2 files
Some checks failed
Build-Test-Publish / build (linux/amd64) (push) Failing after 9s
Build-Test-Publish / build (linux/arm64) (push) Failing after 10s

This commit is contained in:
Your Name
2025-08-17 21:49:07 +12:00
parent fbafdb0b61
commit 4e5253f1a3
3 changed files with 105 additions and 10 deletions

69
CLAUDE.md Normal file
View File

@@ -0,0 +1,69 @@
# CLAUDE.md
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
## Overview
SOS (Simple Object Storage) is a Bash script client for uploading files to a simple object storage server. The repository contains the main `sos` script for uploading files, along with testing and publishing infrastructure.
## Key Commands
### Testing
```bash
./test.sh # Run comprehensive test suite with Docker container
```
### Publishing
```bash
./publish.sh # Publish sos tool to getpkg.xyz and getbin.xyz
```
### Direct Usage
```bash
./sos upload <server> <file> <label:tag> [label:tag ...]
# Example: ./sos upload getbin.xyz ./file.txt file:latest version:1.0
```
## Architecture
### Core Components
1. **sos** - Main bash script (233 lines)
- Handles file uploads to object storage servers
- Supports deduplication via SHA-256 hashing
- Manages authentication via SOS_WRITE_TOKEN environment variable
- Falls back to getpkg for hash calculation if sha256sum/shasum unavailable
2. **test.sh** - Comprehensive test suite
- Sets up Docker-based test environment
- Tests upload, retrieval, deduplication, metadata updates
- Validates authentication and error handling
- Automatically cleans up test resources
3. **publish.sh** - Release automation
- Publishes to getpkg.xyz (package registry)
- Uploads to getbin.xyz using sos itself
- Injects version timestamps during build
### Key Technical Details
- **Authentication**: Uses Bearer token authentication stored in `SOS_WRITE_TOKEN` or `~/.config/sos/write_token.txt`
- **Deduplication**: Calculates SHA-256 hash locally before upload to check if file already exists
- **Label System**: Files are tagged with `label:tag` format for versioning and organization
- **Test Mode**: Set `SOS_TEST_MODE=1` to use HTTP instead of HTTPS for local testing
- **Dependencies**: Requires curl, jq for JSON parsing; optionally uses sha256sum/shasum or falls back to getpkg
### Server Endpoints Used
- `/exists/<hash>` - Check if file already exists
- `/upload` - Upload new file with metadata
- `/update` - Update metadata for existing file
- `/hash/<label:tag>` - Get hash for a labeled file
- `/meta/<hash>` - Get metadata for a file by hash
- `/<label:tag>` or `/<hash>` - Download file
## Development Notes
- The script uses strict error handling (`set -euo pipefail`)
- Temporary directories are automatically cleaned up on exit
- Version placeholders are replaced during publishing
- Tests require Docker for running the SOS server container

View File

@@ -17,22 +17,28 @@ function heading() {
#--------------------------------------------------------------------------------
heading "Publishing ${PROJECT} as tool to ${PROJECT}:${ARCH} (on getpkg.xyz)"
# Generate version timestamp
VERSION=$(date -u +"%Y.%m%d.%H%M")
# Update version in the main sos script
echo "Updating version in main ${PROJECT} script to ${VERSION}..."
sed -i "s/^VERSION=\".*\"/VERSION=\"${VERSION}\"/" "${SCRIPT_DIR}/${PROJECT}"
# Copy the tool to the tool directory
TOOLDIR="${SCRIPT_DIR}/tool"
mkdir -p "${TOOLDIR}"
cp "${SCRIPT_DIR}/${PROJECT}" "${TOOLDIR}/${PROJECT}"
# Replace version placeholder with actual timestamp
VERSION=$(date -u +"%Y.%m%d.%H%M")
sed -i "s/__VERSION_PLACEHOLDER__/${VERSION}/g" "${TOOLDIR}/${PROJECT}"
# Print the version of the tool
echo "Version of the tool:"
"${TOOLDIR}/${PROJECT}" version
# Use getpkg to publish the tool
# Use getpkg to publish the tool - check standard location first
GETPKG="${HOME}/.local/bin/getpkg/getpkg"
if [ ! -f "$GETPKG" ]; then
GETPKG="${SCRIPT_DIR}/../getpkg/output/getpkg"
fi
if [ ! -f "$GETPKG" ]; then
GETPKG="${SCRIPT_DIR}/../getpkg/getpkg"
fi

28
sos
View File

@@ -1,7 +1,7 @@
#!/bin/bash
set -euo pipefail
VERSION="__VERSION_PLACEHOLDER__"
VERSION="2025.0817.0946"
SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )"
TEMP_DIR=$(mktemp -d)
@@ -21,8 +21,11 @@ function get_getpkg() {
return
fi
# get getpkg
export GETPKG="${SCRIPT_DIR}/../getpkg/output/getpkg"
# get getpkg - check standard locations first
export GETPKG="${HOME}/.local/bin/getpkg/getpkg"
if [ ! -f "${GETPKG}" ]; then
GETPKG="${SCRIPT_DIR}/../getpkg/output/getpkg"
fi
if [ ! -f "${GETPKG}" ]; then
ARCH=$(uname -m)
curl -L -s -o "${TEMP_DIR}/getpkg" "https://getbin.xyz/getpkg:latest-${ARCH}" || die "Failed to download getpkg"
@@ -90,6 +93,23 @@ function upload() {
echo "Uploading $file to $TARGET_SERVER"
DATETIME=$(datetime)
VERSION_TO_USE="$DATETIME"
# Check if file is executable and try to get its version
if [ -x "$file" ]; then
echo "File is executable, attempting to get version..."
set +e # Temporarily disable exit on error
FILE_VERSION=$("$file" version 2>/dev/null | head -1)
set -e # Re-enable exit on error
# Check if output matches YYYY.MMDD.HHMM format
if [[ "$FILE_VERSION" =~ ^[0-9]{4}\.[0-9]{4}\.[0-9]{4}$ ]]; then
echo "Found version from executable: $FILE_VERSION"
VERSION_TO_USE="$FILE_VERSION"
else
echo "Executable version output not in expected format, using upload timestamp"
fi
fi
# deduplicate the labeltags
mapfile -t LABELTAGS < <(printf "%s\n" "${LABELTAGS[@]}" | sort -u)
@@ -112,7 +132,7 @@ METADATA_JSON=$(cat <<EOF
{
"labeltags": $LABELTAGS_JSON,
"description": "Uploaded by sos",
"version": "$DATETIME"
"version": "$VERSION_TO_USE"
}
EOF
)