diff --git a/ClientTesting.md b/ClientTesting.md new file mode 100644 index 0000000..32f35de --- /dev/null +++ b/ClientTesting.md @@ -0,0 +1,277 @@ +# Client Testing Guide + +This guide explains how to set up a test Simple Object Server instance using Docker for developing and testing client applications. No repository cloning required! + +## Quick Start + +### 1. Generate Authentication Token + +First, generate an authentication token using the hash_token utility from the Docker image: + +```bash +# Generate a hashed token for your test environment +docker run --rm gitea.jde.nz/public/simple-object-server /sos/hash_token "my-test-token" +# Copy the output hash (starts with $2b$10$...) +``` + +### 2. Create Test Configuration + +Create a file named `test_config.json` with the hashed token from above: + +```json +{ + "write_tokens": [ + "" + ], + "rate_limiting": { + "auth_rate_limit": 5, + "auth_window_seconds": 2 + }, + "logging": { + "log_file_path": "/data/test.log", + "log_level": "info" + }, + "storage_path": "/data/storage", + "port": 7703, + "host": "0.0.0.0" +} +``` + +### 3. Run Test Server + +```bash +# Create a directory for storage +mkdir -p sos-test-data + +# Run the server using the production Docker image +docker run -d \ + --name sos-test \ + -p 7703:7703 \ + -v $(pwd)/test_config.json:/data/sos_config.json:ro \ + -v $(pwd)/sos-test-data:/data/storage \ + gitea.jde.nz/public/simple-object-server +``` + +The test server will be accessible at `http://localhost:7703` + +### 4. Verify Server is Running + +```bash +# Check server status +curl http://localhost:7703/status + +# Expected response: +# {"result": "success", "status": "running"} +``` + +## Test Authentication Token + +You created a test token in step 1: `my-test-token` + +Use this plaintext token in the `Authorization: Bearer ` header for write operations. + +## API Testing Examples + +### Upload a File + +```bash +# Upload with metadata +curl -X PUT \ + -H "Authorization: Bearer my-test-token" \ + -F "file=@test.txt" \ + -F 'metadata={"labeltags":["project:latest","version:1.0"],"description":"Test file"}' \ + http://localhost:7703/upload +``` + +### Retrieve by Hash + +```bash +# Get file by hash (replace with actual hash from upload response) +curl http://localhost:7703/ +``` + +### Retrieve by Label:Tag + +```bash +# Get file by label:tag +curl http://localhost:7703/project:latest +``` + +### Get Metadata + +```bash +# Get metadata by hash or label:tag +curl http://localhost:7703/meta/ +curl http://localhost:7703/meta/project:latest +``` + +### Update Metadata + +```bash +# Update metadata for existing object +curl -X PUT \ + -H "Authorization: Bearer my-test-token" \ + -H "Content-Type: application/json" \ + -d '{"hash":"","metadata":{"labeltags":["project:v2"],"new_field":"value"}}' \ + http://localhost:7703/update +``` + +### Delete Object + +```bash +# Delete object by hash +curl -H "Authorization: Bearer my-test-token" \ + http://localhost:7703/deleteobject?hash= +``` + + +## Test Server Configuration + +The test configuration uses: + +- **Port**: 7703 +- **Storage Path**: /data/storage (inside container, mapped to ./sos-test-data) +- **Rate Limiting**: 5 auth attempts per 2 seconds (for testing rate limits) +- **Log Level**: info + +## Client Development Tips + +### 1. Content Deduplication +The server deduplicates files based on SHA-256 hash. Uploading the same file multiple times will not consume additional storage. + +### 2. Label:Tag System +- Labels and tags provide human-friendly names for objects +- Format: `label:tag` (e.g., `project:latest`, `data:v1.2.3`) +- When uploading with an existing label:tag, the tag moves to the new file +- Previous versions remain accessible by their hash + +### 3. Custom Metadata +- All metadata fields are preserved as JSON +- You can add any custom fields beyond the required `labeltags` +- Metadata is returned with all retrieval operations + +### 4. Large File Support +- The server efficiently handles files up to 6GB +- Uses streaming to minimize memory usage +- Test with 1GB files using `testing/test_1GB_file_upload.sh` + +### 5. Error Handling +Test your client's error handling: +- Invalid authentication tokens (401 Unauthorized) +- Missing required fields (400 Bad Request) +- Non-existent objects (404 Not Found) +- Rate limiting (429 Too Many Requests) + +## Monitoring Test Server + +```bash +# View container logs +docker logs -f sos-test + +# Check container health +docker inspect sos-test --format='{{.State.Health.Status}}' + +# Access container shell for debugging +docker exec -it sos-test /bin/sh +``` + +## Cleanup + +```bash +# Stop and remove test container +docker stop sos-test +docker rm sos-test + +# Remove test data (optional) +rm -rf sos-test-data +rm test_config.json + +# Remove Docker volume if using persistent storage (optional) +docker volume rm sos-storage +``` + +## Advanced Testing + +### Generate Additional Auth Tokens + +To create additional authentication tokens beyond the test tokens: + +```bash +# Use the hash_token utility from the Docker image +docker run --rm gitea.jde.nz/public/simple-object-server /sos/hash_token "your-secret-token-here" +# Output: $2b$10$... (bcrypt hash) + +# Or download the standalone utility +wget -q https://getbin.xyz/sos-hash:latest -O sos-hash && chmod +x sos-hash +./sos-hash "your-secret-token-here" +``` + +Then add the hashed tokens to your configuration: + +```json +{ + "write_tokens": ["your-bcrypt-hashes-here"], + "rate_limiting": { + "auth_rate_limit": 10, + "auth_window_seconds": 60 + }, + "logging": { + "log_file_path": "/data/test.log", + "log_level": "debug" + }, + "storage_path": "/data/storage", + "port": 8080, + "host": "0.0.0.0" +} +``` + +Run with your custom config: + +```bash +docker run -d \ + --name sos-custom \ + -p 8080:8080 \ + -v $(pwd)/custom_config.json:/data/sos_config.json:ro \ + -v $(pwd)/sos-data:/data/storage \ + gitea.jde.nz/public/simple-object-server +``` + +### Persistent Storage + +To persist data between container restarts, use a Docker volume: + +```bash +# Create a named volume +docker volume create sos-storage + +# Run with persistent storage +docker run -d \ + --name sos-test \ + -p 7703:7703 \ + -v $(pwd)/test_config.json:/data/sos_config.json:ro \ + -v sos-storage:/data/storage \ + gitea.jde.nz/public/simple-object-server +``` + +## Troubleshooting + +### Container Won't Start +- Check if port 7703 is already in use: `lsof -i :7703` +- Verify Docker image built successfully +- Check container logs: `docker logs sos-test` + +### Authentication Failures +- Ensure you're using one of the test tokens exactly as shown +- Include `Bearer` prefix in Authorization header +- Check for rate limiting (5 attempts per 2 seconds in test config) + +### Connection Refused +- Verify container is running: `docker ps` +- Check container health: `docker inspect sos-test --format='{{.State.Health.Status}}'` +- Ensure port mapping is correct: `-p 7703:7703` + +## Further Information + +- [API Documentation](../README.md) +- [Build Instructions](../README.md#building) +- [Configuration Options](../README.md#configuration) \ No newline at end of file diff --git a/Dockerfile.sos b/Dockerfile.sos index a42e3e7..3f1ecb6 100644 --- a/Dockerfile.sos +++ b/Dockerfile.sos @@ -11,6 +11,7 @@ RUN apk add --no-cache wget curl bash jq RUN mkdir -p /sos && mkdir -p /data/storage COPY --chmod=0755 output/simple-object-server /sos/sos +COPY --chmod=0755 output/hash_token /sos/hash_token COPY testing/ /testing/ # Expose port