simple-object-server/ClientTesting.md

Client Testing Guide

This guide explains how to set up a test Simple Object Server instance using Docker for developing and testing client applications. No repository cloning required!

Quick Start

1. Generate Authentication Token

First, generate an authentication token using the hash_token utility from the Docker image:

# Generate a hashed token for your test environment
docker run --rm gitea.jde.nz/public/simple-object-server /sos/hash_token "my-test-token"
# Copy the output hash (starts with $2b$10$...)

2. Create Test Configuration

Create a file named test_config.json with the hashed token from above:

{
    "write_tokens": [
        "<paste-hash-from-previous-step>"
    ],
    "rate_limiting": {
        "auth_rate_limit": 5,
        "auth_window_seconds": 2
    },
    "logging": {
        "log_file_path": "/data/test.log",
        "log_level": "info"
    },
    "storage_path": "/data/storage",
    "port": 7703,
    "host": "0.0.0.0"
}

3. Run Test Server

# Create a directory for storage
mkdir -p sos-test-data

# Run the server using the production Docker image
docker run -d \
  --name sos-test \
  -p 7703:7703 \
  -v $(pwd)/test_config.json:/data/sos_config.json:ro \
  -v $(pwd)/sos-test-data:/data/storage \
  gitea.jde.nz/public/simple-object-server

The test server will be accessible at http://localhost:7703

4. Verify Server is Running

# Check server status
curl http://localhost:7703/status

# Expected response:
# {"result": "success", "status": "running"}

Test Authentication Token

You created a test token in step 1: my-test-token

Use this plaintext token in the Authorization: Bearer <token> header for write operations.

API Testing Examples

Upload a File

# Upload with metadata
curl -X PUT \
  -H "Authorization: Bearer my-test-token" \
  -F "file=@test.txt" \
  -F 'metadata={"labeltags":["project:latest","version:1.0"],"description":"Test file"}' \
  http://localhost:7703/upload

Retrieve by Hash

# Get file by hash (replace with actual hash from upload response)
curl http://localhost:7703/<hash>

Retrieve by Label:Tag

# Get file by label:tag
curl http://localhost:7703/project:latest

Get Metadata

# Get metadata by hash or label:tag
curl http://localhost:7703/meta/<hash>
curl http://localhost:7703/meta/project:latest

Update Metadata

# Update metadata for existing object
curl -X PUT \
  -H "Authorization: Bearer my-test-token" \
  -H "Content-Type: application/json" \
  -d '{"hash":"<hash>","metadata":{"labeltags":["project:v2"],"new_field":"value"}}' \
  http://localhost:7703/update

Delete Object

# Delete object by hash
curl -H "Authorization: Bearer my-test-token" \
  http://localhost:7703/deleteobject?hash=<hash>

Test Server Configuration

The test configuration uses:

• Port: 7703
• Storage Path: /data/storage (inside container, mapped to ./sos-test-data)
• Rate Limiting: 5 auth attempts per 2 seconds (for testing rate limits)
• Log Level: info

Client Development Tips

1. Content Deduplication

The server deduplicates files based on SHA-256 hash. Uploading the same file multiple times will not consume additional storage.

2. Label:Tag System

• Labels and tags provide human-friendly names for objects
• Format: label:tag (e.g., project:latest, data:v1.2.3)
• When uploading with an existing label:tag, the tag moves to the new file
• Previous versions remain accessible by their hash

3. Custom Metadata

• All metadata fields are preserved as JSON
• You can add any custom fields beyond the required labeltags
• Metadata is returned with all retrieval operations

4. Large File Support

• The server efficiently handles files up to 6GB
• Uses streaming to minimize memory usage
• Test with 100MB files using testing/test_100MB_file_upload.sh

5. Error Handling

Test your client's error handling:

• Invalid authentication tokens (401 Unauthorized)
• Missing required fields (400 Bad Request)
• Non-existent objects (404 Not Found)
• Rate limiting (429 Too Many Requests)

Monitoring Test Server

# View container logs
docker logs -f sos-test

# Check container health
docker inspect sos-test --format='{{.State.Health.Status}}'

# Access container shell for debugging
docker exec -it sos-test /bin/sh

Cleanup

# Stop and remove test container
docker stop sos-test
docker rm sos-test

# Remove test data (optional)
rm -rf sos-test-data
rm test_config.json

# Remove Docker volume if using persistent storage (optional)
docker volume rm sos-storage

Advanced Testing

Generate Additional Auth Tokens

To create additional authentication tokens beyond the test tokens:

# Use the hash_token utility from the Docker image
docker run --rm gitea.jde.nz/public/simple-object-server /sos/hash_token "your-secret-token-here"
# Output: $2b$10$... (bcrypt hash)

# Or download the standalone utility
wget -q https://getbin.xyz/sos-hash:latest -O sos-hash && chmod +x sos-hash
./sos-hash "your-secret-token-here"

Then add the hashed tokens to your configuration:

{
    "write_tokens": ["your-bcrypt-hashes-here"],
    "rate_limiting": {
        "auth_rate_limit": 10,
        "auth_window_seconds": 60
    },
    "logging": {
        "log_file_path": "/data/test.log",
        "log_level": "debug"
    },
    "storage_path": "/data/storage",
    "port": 8080,
    "host": "0.0.0.0"
}

Run with your custom config:

docker run -d \
  --name sos-custom \
  -p 8080:8080 \
  -v $(pwd)/custom_config.json:/data/sos_config.json:ro \
  -v $(pwd)/sos-data:/data/storage \
  gitea.jde.nz/public/simple-object-server

Persistent Storage

To persist data between container restarts, use a Docker volume:

# Create a named volume
docker volume create sos-storage

# Run with persistent storage
docker run -d \
  --name sos-test \
  -p 7703:7703 \
  -v $(pwd)/test_config.json:/data/sos_config.json:ro \
  -v sos-storage:/data/storage \
  gitea.jde.nz/public/simple-object-server

Troubleshooting

Container Won't Start

• Check if port 7703 is already in use: lsof -i :7703
• Verify Docker image built successfully
• Check container logs: docker logs sos-test

Authentication Failures

• Ensure you're using one of the test tokens exactly as shown
• Include Bearer prefix in Authorization header
• Check for rate limiting (5 attempts per 2 seconds in test config)

Connection Refused

• Verify container is running: docker ps
• Check container health: docker inspect sos-test --format='{{.State.Health.Status}}'
• Ensure port mapping is correct: -p 7703:7703

Further Information

• API Documentation
• Build Instructions
• Configuration Options