dropshell/TEMPLATES.md

Dropshell Template Guide

This guide explains how to create templates for Dropshell - a system management tool for deploying and managing services on remote servers via Docker containers.

Template Structure

Every template must follow this directory structure:

template-name/
├── config/
│   ├── .template_info.env    # REQUIRED: Template metadata
│   └── service.env            # REQUIRED: Default service configuration
├── install.sh                 # REQUIRED: Installation script
├── uninstall.sh              # REQUIRED: Uninstallation script
├── start.sh                  # OPTIONAL: Start the service
├── stop.sh                   # OPTIONAL: Stop the service
├── status.sh                 # OPTIONAL: Check service status
├── logs.sh                   # OPTIONAL: View service logs
├── backup.sh                 # OPTIONAL: Backup service data
├── restore.sh                # OPTIONAL: Restore service data
├── destroy.sh                # OPTIONAL: Destroy service and data
├── ssh.sh                    # OPTIONAL: SSH into service container
├── ports.sh                  # OPTIONAL: List exposed ports
└── README.txt                # OPTIONAL: Template documentation

Important Script Requirements

All template scripts MUST be non-interactive. Scripts should never prompt for user input or require any manual interaction. This is critical for automated deployments and remote management.

• install.sh, uninstall.sh: Must complete without any user prompts
• Data preservation: Use backup.sh/restore.sh for data management instead of prompting during uninstall
• Configuration: All settings must come from environment variables or config files
• Confirmations: Never use read, select, or any interactive prompts
• Errors: Use _die from common.sh to handle errors and exit cleanly

Critical Data Preservation Rules

NEVER DESTROY DATA VOLUMES IN UNINSTALL.SH

This is a fundamental principle of Dropshell templates:

1. uninstall.sh MUST preserve all data volumes - Only remove containers, never volumes
2. Data volumes contain irreplaceable user data - Timestamps, logs, databases, configurations
3. Only destroy.sh should remove volumes - And it must be a separate, explicit action
4. Volume removal is IRREVERSIBLE - Once deleted, data cannot be recovered

Correct Implementation

uninstall.sh - Preserves data:

#!/bin/bash
source "${AGENT_PATH}/common.sh"
_check_required_env_vars "CONTAINER_NAME"

# Stop and remove container only
bash ./stop.sh || _die "Failed to stop container"
_remove_container "$CONTAINER_NAME" || _die "Failed to remove container"

# DO NOT remove volumes here!
# Data volumes are preserved for potential reinstallation

echo "Uninstallation complete. Data volumes preserved."
echo "To remove data permanently, run: ./destroy.sh"

destroy.sh - Explicitly removes everything:

#!/bin/bash
source "${AGENT_PATH}/common.sh"
_check_required_env_vars "CONTAINER_NAME" "DATA_VOLUME"

echo "WARNING: This will PERMANENTLY DELETE all data for ${CONTAINER_NAME}"
echo "This action cannot be undone!"

# Since scripts must be non-interactive, only proceed if explicitly called
bash ./stop.sh
_remove_container "$CONTAINER_NAME"
_remove_volume "$DATA_VOLUME"  # Only destroy.sh removes volumes

echo "Service and all data destroyed"

Why This Matters

• User Trust: Users expect uninstall to be reversible by reinstalling
• Data Safety: Application data often contains critical business information
• Timestamps: Historical data and audit logs are irreplaceable
• Recovery: Allows users to reinstall without losing their work
• Best Practice: Follows industry standards for application management

Essential Files

1. config/.template_info.env

Template metadata file that defines the template and its requirements. This file is managed by the template author and should NOT be modified by users:

# Template identifier - MUST match the directory name
TEMPLATE=template-name

# Requirements
REQUIRES_HOST_ROOT=false      # Whether root access on host is needed
REQUIRES_DOCKER=true          # Whether Docker is required
REQUIRES_DOCKER_ROOT=false    # Whether Docker root privileges are needed

# Docker image settings
IMAGE_REGISTRY="docker.io"
IMAGE_REPO="vendor/image"
IMAGE_TAG="latest"

# Volume definitions (if needed)
DATA_VOLUME="${CONTAINER_NAME}_data"
CONFIG_VOLUME="${CONTAINER_NAME}_config"

Important: This file is replaced when the template is updated. Users should never edit this file.

2. config/service.env

Default service configuration that will be modified per installation. When a user creates a service from this template, they edit this file to customize the deployment:

# Service identification (REQUIRED)
CONTAINER_NAME=service-name

# Server settings (REQUIRED by dropshell)
SSH_USER="root"  # CRITICAL: This must always be defined!

# Docker settings (optional)
IMAGE_TAG="latest"

# Service-specific variables
# (Add any configuration specific to your service)

CRITICAL REQUIREMENT: The SSH_USER variable MUST be defined in every service.env file. Dropshell uses this internally for server management. Without it, the template will fail with "SSH_USER variable not defined" error.

Important: This is the file users edit to customize each service instance. Different deployments of the same template will have different service.env files.

How Environment Variables Work

When Dropshell runs any template script (install.sh, start.sh, etc.), it:

1. First loads all variables from config/.template_info.env
2. Then loads all variables from config/service.env (which can override template defaults)
3. Exports all these variables to the environment
4. Executes the requested script with these variables available

This means:

• Scripts can use any variable defined in either file directly (e.g., $CONTAINER_NAME, $IMAGE_TAG)
• Variables in service.env override those in .template_info.env if they have the same name
• Users customize deployments by editing service.env, never .template_info.env

3. install.sh

Installation script that sets up the service:

#!/bin/bash
source "${AGENT_PATH}/common.sh"
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"

# Check required environment variables
_check_required_env_vars "CONTAINER_NAME" "IMAGE_REGISTRY" "IMAGE_REPO" "IMAGE_TAG"

# Check Docker is available
_check_docker_installed || _die "Docker test failed"

# Pull the Docker image
docker pull "$IMAGE_REGISTRY/$IMAGE_REPO:$IMAGE_TAG" || _die "Failed to pull image"

# Stop any existing container
bash ./stop.sh || _die "Failed to stop container"

# Remove old container
_remove_container "$CONTAINER_NAME" || _die "Failed to remove container"

# Start the new container
bash ./start.sh || _die "Failed to start container"

echo "Installation of ${CONTAINER_NAME} complete"

4. uninstall.sh

Uninstallation script to remove the service (must be non-interactive and MUST preserve data):

#!/bin/bash
source "${AGENT_PATH}/common.sh"
_check_required_env_vars "CONTAINER_NAME"

# Stop the container
bash ./stop.sh || _die "Failed to stop container"

# Remove the container
_remove_container "$CONTAINER_NAME" || _die "Failed to remove container"

# CRITICAL: Never remove data volumes in uninstall.sh!
# Data volumes must be preserved for potential reinstallation
# Only destroy.sh should remove volumes, and it must be explicit

echo "Uninstallation of ${CONTAINER_NAME} complete"
echo "Note: Data volumes have been preserved. To remove all data, use destroy.sh"

Optional Command Scripts

start.sh

Starts the Docker container with appropriate configuration:

#!/bin/bash
source "${AGENT_PATH}/common.sh"
_check_required_env_vars "CONTAINER_NAME" "IMAGE_REGISTRY" "IMAGE_REPO" "IMAGE_TAG"

docker run -d \
    --name "$CONTAINER_NAME" \
    --restart unless-stopped \
    -v "$DATA_VOLUME:/data" \
    "$IMAGE_REGISTRY/$IMAGE_REPO:$IMAGE_TAG"

stop.sh

Stops the running container:

#!/bin/bash
source "${AGENT_PATH}/common.sh"
_check_required_env_vars "CONTAINER_NAME"

docker stop "$CONTAINER_NAME" 2>/dev/null || true

status.sh

Reports the service status.

Expected Output Format:

• Must output a single line with one of these exact status values:
  • Running - Service is active and operational
  • Stopped - Service is stopped but configured
  • Error - Service is in an error state
  • Unknown - Status cannot be determined

The output is parsed by Dropshell for the list command, so it must be exactly one of these values with no additional text.

#!/bin/bash
source "${AGENT_PATH}/common.sh"
_check_required_env_vars "CONTAINER_NAME"

if docker ps --format "table {{.Names}}" | grep -q "^${CONTAINER_NAME}$"; then
    echo "Running"
else
    echo "Stopped"
fi

For more complex status checks:

#!/bin/bash
source "${AGENT_PATH}/common.sh"
_check_required_env_vars "CONTAINER_NAME"

# Check if container exists
if ! docker ps -a --format "{{.Names}}" | grep -q "^${CONTAINER_NAME}$"; then
    echo "Unknown"
    exit 0
fi

# Check container state
STATE=$(docker inspect -f '{{.State.Status}}' "$CONTAINER_NAME" 2>/dev/null)
case "$STATE" in
    running)
        # Additional health check if needed
        if docker inspect -f '{{.State.Health.Status}}' "$CONTAINER_NAME" 2>/dev/null | grep -q "unhealthy"; then
            echo "Error"
        else
            echo "Running"
        fi
        ;;
    exited|stopped)
        echo "Stopped"
        ;;
    restarting|paused)
        echo "Error"
        ;;
    *)
        echo "Unknown"
        ;;
esac

logs.sh

Shows container logs:

#!/bin/bash
source "${AGENT_PATH}/common.sh"
_check_required_env_vars "CONTAINER_NAME"

docker logs "$CONTAINER_NAME" "$@"

backup.sh / restore.sh

Handle data backup and restoration for services with persistent data.

Common Functions (from common.sh)

Templates have access to these utility functions:

• _check_required_env_vars VAR1 VAR2 ... - Verify required variables are set
• _die "message" - Exit with error message
• _check_docker_installed - Verify Docker is available
• _remove_container NAME - Safely remove a container
• create_items ITEM1 ITEM2 ... - Create volumes or networks
• _remove_volume NAME - Remove a Docker volume

Environment Variables

Templates receive these variables at runtime:

• AGENT_PATH - Path to the Dropshell agent directory
• CONFIG_PATH - Path to the service configuration directory
• SCRIPT_DIR - Directory containing the template scripts
• All variables from .template_info.env and service.env

Validation Requirements

Templates must pass these validation checks:

1. Required files exist: config/.template_info.env, config/service.env, install.sh, uninstall.sh
2. Scripts are executable: All .sh files must have execute permissions
3. TEMPLATE variable matches: The TEMPLATE variable in .template_info.env must match the directory name
4. Valid environment files: Both .env files must be parseable
5. SSH_USER defined: The SSH_USER variable MUST be present in service.env

Best Practices

1. Always use _check_required_env_vars at the start of scripts to validate inputs
2. Handle errors gracefully with || _die "Error message"
3. Use Docker volumes for persistent data that survives container recreation
4. NEVER remove data volumes in uninstall.sh - Only destroy.sh should remove volumes
5. Document configuration in README.txt for users
6. Support idempotency - scripts should handle being run multiple times
7. Separate concerns - uninstall.sh removes service, destroy.sh removes data
8. Use standard naming - volumes as ${CONTAINER_NAME}_data, configs as ${CONTAINER_NAME}_config
9. Test thoroughly - use dropshell test-template <path> to validate
10. Preserve user data - Always prioritize data safety over convenience

Examples

Example 1: Basic Web Server (Nginx)

A minimal template for deploying an Nginx web server:

nginx-server/
├── config/
│   ├── .template_info.env
│   └── service.env
├── install.sh
├── uninstall.sh
├── start.sh
├── stop.sh
└── logs.sh

.template_info.env:

TEMPLATE=nginx-server
REQUIRES_DOCKER=true
REQUIRES_DOCKER_ROOT=false
IMAGE_REGISTRY="docker.io"
IMAGE_REPO="nginx"
CONTENT_VOLUME="${CONTAINER_NAME}_content"

service.env:

CONTAINER_NAME=nginx-server
SSH_USER="root"
IMAGE_TAG="alpine"
HTTP_PORT=8080

start.sh:

#!/bin/bash
source "${AGENT_PATH}/common.sh"
_check_required_env_vars "CONTAINER_NAME" "IMAGE_REGISTRY" "IMAGE_REPO" "IMAGE_TAG" "HTTP_PORT"

docker run -d \
    --name "$CONTAINER_NAME" \
    --restart unless-stopped \
    -p "${HTTP_PORT}:80" \
    -v "${CONTENT_VOLUME}:/usr/share/nginx/html:ro" \
    "$IMAGE_REGISTRY/$IMAGE_REPO:$IMAGE_TAG"

Example 2: Stateful Application with Persistent Storage

A more complex template for a database or application requiring data persistence, configuration files, and network settings:

postgres-db/
├── config/
│   ├── .template_info.env
│   ├── service.env
│   └── postgres.conf        # Additional config files
├── install.sh
├── uninstall.sh
├── start.sh
├── stop.sh
├── status.sh
├── backup.sh                # Data backup
├── restore.sh               # Data restoration
├── destroy.sh               # Complete cleanup including volumes
├── logs.sh
└── README.txt

.template_info.env:

TEMPLATE=postgres-db
REQUIRES_DOCKER=true
REQUIRES_DOCKER_ROOT=true    # Needed for volume management
IMAGE_REGISTRY="docker.io"
IMAGE_REPO="postgres"

# Multiple volumes for different purposes
DATA_VOLUME="${CONTAINER_NAME}_data"
BACKUP_VOLUME="${CONTAINER_NAME}_backup"
CONFIG_VOLUME="${CONTAINER_NAME}_config"

service.env:

CONTAINER_NAME=postgres-db
SSH_USER="root"
IMAGE_TAG="15-alpine"
DB_PORT=5432
POSTGRES_PASSWORD="changeme"
POSTGRES_USER="admin"
POSTGRES_DB="myapp"

backup.sh:

#!/bin/bash
source "${AGENT_PATH}/common.sh"
_check_required_env_vars "CONTAINER_NAME" "BACKUP_VOLUME" "POSTGRES_USER" "POSTGRES_DB"

TIMESTAMP=$(date +%Y%m%d_%H%M%S)
BACKUP_FILE="backup_${TIMESTAMP}.sql"

echo "Creating backup ${BACKUP_FILE}..."
docker exec "$CONTAINER_NAME" pg_dump -U "$POSTGRES_USER" "$POSTGRES_DB" > "/tmp/${BACKUP_FILE}"
docker cp "/tmp/${BACKUP_FILE}" "${CONTAINER_NAME}:/${BACKUP_VOLUME}/${BACKUP_FILE}"
rm "/tmp/${BACKUP_FILE}"

echo "Backup completed: ${BACKUP_FILE}"

destroy.sh:

#!/bin/bash
source "${AGENT_PATH}/common.sh"
_check_required_env_vars "CONTAINER_NAME" "DATA_VOLUME" "BACKUP_VOLUME" "CONFIG_VOLUME"

echo "WARNING: This will destroy all data for ${CONTAINER_NAME}"
read -p "Are you sure? (yes/no): " confirm

if [ "$confirm" = "yes" ]; then
    bash ./stop.sh
    _remove_container "$CONTAINER_NAME"
    _remove_volume "$DATA_VOLUME"
    _remove_volume "$BACKUP_VOLUME"
    _remove_volume "$CONFIG_VOLUME"
    echo "Service and all data destroyed"
else
    echo "Cancelled"
fi

Docker Compose Templates

When creating templates that use Docker Compose:

Important: Always Build on Install

When using docker compose with custom images (defined with build: in your compose file), always use the --build flag in your install.sh:

# CORRECT - Forces rebuild of images during installation
docker compose up -d --build

# WRONG - May use stale cached images
docker compose up -d

This ensures that:

• Custom images are rebuilt with the latest code changes
• No stale images are used from previous installations
• The service always starts with the most recent image version

Example Docker Compose install.sh

#!/bin/bash
source "${AGENT_PATH}/common.sh"
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"

_check_required_env_vars "CONTAINER_NAME"
_check_docker_installed || _die "Docker test failed"

# Stop any existing containers
docker compose down || true

# Start with rebuild to ensure fresh images
docker compose up -d --build || _die "Failed to start services"

echo "Installation of ${CONTAINER_NAME} complete"

Docker Compose Best Practices

1. Always include --build in install.sh when using custom images
2. Use .env files for configuration that Docker Compose can read
3. Define service names consistently with CONTAINER_NAME
4. Handle cleanup properly in stop.sh and uninstall.sh using docker compose down
5. Use named volumes for persistent data that matches Dropshell conventions

Testing Templates

After creating a template, validate it:

dropshell test-template /path/to/template

This checks all requirements and reports any issues.