swtich from ELK to Loki!

simple-logs/README.md

@@ -0,0 +1,61 @@
+# Simple Logs
+
+A dead-simple log viewer using Grafana + Loki. Much easier than ELK/Kibana!
+
+## Features
+- **ALL logs in one place** - Docker containers + system logs
+- **Simple interface** - Just one dashboard, no complex setup
+- **Fast & lightweight** - Uses 10x less resources than ELK
+- **Real-time** - Logs appear instantly
+- **No configuration** - Works out of the box
+
+## Quick Start
+
+1. **Install**
+```bash
+dropshell install simple-logs
+```
+
+2. **Access**
+- Go to: `http://<server-ip>:3000`
+- Login: `admin` / `admin` (change on first login)
+
+3. **View Logs**
+- Click the **Dashboards** icon (4 squares) on left
+- Click **"Simple Logs"**
+- That's it! All your logs are there
+
+## How to Use
+
+### Filter by Container
+Click any `container_name` label to see only that container's logs
+
+### Search Text
+Use the search box at the top to find specific text
+
+### Change Time Range
+Top-right corner - set to "Last 5 minutes" for recent logs
+
+### View Details
+Click any log line to expand and see full details
+
+## What Gets Logged
+- All Docker container logs
+- System logs (/var/log/syslog)
+- Authentication logs (/var/log/auth.log)
+
+## Ports
+- `3000` - Grafana Web UI
+- `3100` - Loki API (internal)
+
+## Why This Instead of ELK?
+- **10x simpler** - One dashboard vs Kibana's complexity
+- **10x smaller** - ~200MB RAM vs 2-3GB for ELK
+- **Instant setup** - No index patterns, no field mappings
+- **Just works** - No configuration needed
+
+## Advanced: Custom Queries
+In the logs panel, you can use LogQL queries:
+- `{container_name="myapp"}` - Specific container
+- `{job="syslog"} |= "error"` - System errors
+- `{job="docker"} |~ "(?i)error|warn"` - All Docker warnings/errors

simple-logs/config/dashboards/simple-logs.json

@@ -0,0 +1,69 @@
+{
+  "annotations": {
+    "list": [
+      {
+        "builtIn": 1,
+        "datasource": "-- Grafana --",
+        "enable": true,
+        "hide": true,
+        "iconColor": "rgba(0, 211, 255, 1)",
+        "name": "Annotations & Alerts",
+        "type": "dashboard"
+      }
+    ]
+  },
+  "editable": true,
+  "gnetId": null,
+  "graphTooltip": 0,
+  "id": null,
+  "links": [],
+  "panels": [
+    {
+      "datasource": "Loki",
+      "fieldConfig": {
+        "defaults": {},
+        "overrides": []
+      },
+      "gridPos": {
+        "h": 20,
+        "w": 24,
+        "x": 0,
+        "y": 0
+      },
+      "id": 2,
+      "options": {
+        "dedupStrategy": "none",
+        "enableLogDetails": true,
+        "prettifyLogMessage": false,
+        "showCommonLabels": false,
+        "showLabels": true,
+        "showTime": true,
+        "sortOrder": "Descending",
+        "wrapLogMessage": true
+      },
+      "targets": [
+        {
+          "expr": "{job=~\"docker|syslog|auth\"}",
+          "refId": "A"
+        }
+      ],
+      "title": "All Logs",
+      "type": "logs"
+    }
+  ],
+  "schemaVersion": 27,
+  "style": "dark",
+  "tags": ["logs"],
+  "templating": {
+    "list": []
+  },
+  "time": {
+    "from": "now-15m",
+    "to": "now"
+  },
+  "timepicker": {},
+  "timezone": "",
+  "title": "Simple Logs",
+  "uid": "simple-logs",
+  "version": 0
+}

simple-logs/config/grafana-dashboard.yml

@@ -0,0 +1,12 @@
+apiVersion: 1
+
+providers:
+  - name: 'default'
+    orgId: 1
+    folder: ''
+    type: file
+    disableDeletion: false
+    updateIntervalSeconds: 10
+    allowUiUpdates: true
+    options:
+      path: /var/lib/grafana/dashboards

simple-logs/config/grafana-datasources.yml

@@ -0,0 +1,9 @@
+apiVersion: 1
+
+datasources:
+  - name: Loki
+    type: loki
+    access: proxy
+    url: http://loki:3100
+    isDefault: true
+    editable: false

simple-logs/config/loki.yaml

@@ -0,0 +1,36 @@
+auth_enabled: false
+
+server:
+  http_listen_port: 3100
+  grpc_listen_port: 9096
+
+common:
+  path_prefix: /loki
+  storage:
+    filesystem:
+      chunks_directory: /loki/chunks
+      rules_directory: /loki/rules
+  replication_factor: 1
+  ring:
+    instance_addr: 127.0.0.1
+    kvstore:
+      store: inmemory
+
+schema_config:
+  configs:
+    - from: 2020-10-24
+      store: boltdb-shipper
+      object_store: filesystem
+      schema: v11
+      index:
+        prefix: index_
+        period: 24h
+
+ruler:
+  alertmanager_url: http://localhost:9093
+
+limits_config:
+  retention_period: 168h  # 7 days
+  enforce_metric_name: false
+  reject_old_samples: true
+  reject_old_samples_max_age: 168h

simple-logs/config/promtail.yaml

@@ -0,0 +1,67 @@
+server:
+  http_listen_port: 9080
+  grpc_listen_port: 0
+
+positions:
+  filename: /tmp/positions/positions.yaml
+
+clients:
+  - url: http://loki:3100/loki/api/v1/push
+
+scrape_configs:
+  # Docker container logs
+  - job_name: docker
+    static_configs:
+      - targets:
+          - localhost
+        labels:
+          job: docker
+          __path__: /var/lib/docker/containers/*/*-json.log
+    pipeline_stages:
+      - json:
+          expressions:
+            stream: stream
+            time: time
+            log: log
+      - timestamp:
+          source: time
+          format: RFC3339Nano
+      - labels:
+          stream:
+      - regex:
+          expression: '(?P<container_id>[0-9a-f]{12})'
+          source: filename
+      - labels:
+          container_id:
+
+  # System logs
+  - job_name: syslog
+    static_configs:
+      - targets:
+          - localhost
+        labels:
+          job: syslog
+          __path__: /var/log/syslog
+          host: ${HOSTNAME}
+
+  - job_name: auth
+    static_configs:
+      - targets:
+          - localhost
+        labels:
+          job: auth
+          __path__: /var/log/auth.log
+          host: ${HOSTNAME}
+
+  # Docker events via Docker socket
+  - job_name: docker_events
+    docker_sd_configs:
+      - host: unix:///var/run/docker.sock
+        refresh_interval: 5s
+    relabel_configs:
+      - source_labels: [__meta_docker_container_name]
+        target_label: container_name
+      - source_labels: [__meta_docker_container_id]
+        target_label: container_id
+      - source_labels: [__meta_docker_container_image]
+        target_label: image

simple-logs/config/service.env

@@ -0,0 +1,15 @@
+# Simple Log Viewer Configuration
+CONTAINER_NAME=simple-logs
+
+# Server settings (REQUIRED by dropshell)
+SSH_USER="root"
+
+# Web UI Port
+WEB_PORT=3000
+
+# Log retention (days)
+LOG_RETENTION=7
+
+# Authentication (optional - leave empty for no auth)
+AUTH_USERNAME=""
+AUTH_PASSWORD=""

simple-logs/docker-compose.yml

@@ -0,0 +1,54 @@
+version: '3.8'
+
+services:
+  # Grafana - Simple Web UI for viewing logs
+  grafana:
+    image: grafana/grafana:latest
+    container_name: ${CONTAINER_NAME}_grafana
+    environment:
+      - GF_SECURITY_ADMIN_USER=${AUTH_USERNAME:-admin}
+      - GF_SECURITY_ADMIN_PASSWORD=${AUTH_PASSWORD:-admin}
+      - GF_AUTH_ANONYMOUS_ENABLED=${AUTH_USERNAME:+false}
+      - GF_AUTH_ANONYMOUS_ORG_ROLE=Viewer
+      - GF_INSTALL_PLUGINS=
+    volumes:
+      - grafana_data:/var/lib/grafana
+      - ${CONFIG_PATH}/grafana-datasources.yml:/etc/grafana/provisioning/datasources/datasources.yaml:ro
+      - ${CONFIG_PATH}/grafana-dashboard.yml:/etc/grafana/provisioning/dashboards/dashboard.yaml:ro
+      - ${CONFIG_PATH}/dashboards:/var/lib/grafana/dashboards:ro
+    ports:
+      - "${WEB_PORT:-3000}:3000"
+    restart: unless-stopped
+    depends_on:
+      - loki
+
+  # Loki - Lightweight log storage (100x simpler than Elasticsearch)
+  loki:
+    image: grafana/loki:2.9.0
+    container_name: ${CONTAINER_NAME}_loki
+    volumes:
+      - loki_data:/loki
+      - ${CONFIG_PATH}/loki.yaml:/etc/loki/local-config.yaml:ro
+    ports:
+      - "3100:3100"
+    command: -config.file=/etc/loki/local-config.yaml
+    restart: unless-stopped
+
+  # Promtail - Collects all logs (Docker + System)
+  promtail:
+    image: grafana/promtail:2.9.0
+    container_name: ${CONTAINER_NAME}_promtail
+    volumes:
+      - /var/run/docker.sock:/var/run/docker.sock:ro
+      - /var/log:/var/log:ro
+      - ${CONFIG_PATH}/promtail.yaml:/etc/promtail/config.yml:ro
+      - promtail_positions:/tmp/positions
+    command: -config.file=/etc/promtail/config.yml
+    restart: unless-stopped
+    depends_on:
+      - loki
+
+volumes:
+  grafana_data:
+  loki_data:
+  promtail_positions:

simple-logs/install.sh

@@ -0,0 +1,62 @@
+#!/bin/bash
+source "${AGENT_PATH}/common.sh"
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+
+_check_required_env_vars "CONTAINER_NAME"
+
+# Check Docker
+_check_docker_installed || _die "Docker test failed"
+docker compose version >/dev/null 2>&1 || _die "Docker Compose V2 is required"
+
+# Stop any existing containers
+bash ./stop.sh 2>/dev/null || true
+
+# Create config directory
+mkdir -p "${CONFIG_PATH}/dashboards"
+
+# Copy configuration files
+cp "$SCRIPT_DIR/config/"*.yaml "$SCRIPT_DIR/config/"*.yml "${CONFIG_PATH}/" 2>/dev/null || true
+cp "$SCRIPT_DIR/config/dashboards/"*.json "${CONFIG_PATH}/dashboards/" 2>/dev/null || true
+
+# Start the stack
+echo "Starting Simple Logs stack..."
+docker compose up -d || _die "Failed to start"
+
+# Wait for Grafana to be ready
+echo -n "Waiting for Grafana to start..."
+MAX_WAIT=60
+WAITED=0
+while [ $WAITED -lt $MAX_WAIT ]; do
+    if curl -s "http://localhost:${WEB_PORT:-3000}/api/health" 2>/dev/null | grep -q "ok"; then
+        echo " Ready!"
+        break
+    fi
+    echo -n "."
+    sleep 2
+    WAITED=$((WAITED + 2))
+done
+echo ""
+
+echo "========================================="
+echo "Simple Logs Installed!"
+echo "========================================="
+echo ""
+echo "Access at: http://$(hostname -I | awk '{print $1}'):${WEB_PORT:-3000}"
+echo ""
+if [ -n "${AUTH_USERNAME}" ]; then
+    echo "Login: ${AUTH_USERNAME} / ${AUTH_PASSWORD}"
+else
+    echo "Login: admin / admin"
+    echo "(You'll be asked to change password on first login)"
+fi
+echo ""
+echo "TO VIEW LOGS:"
+echo "1. Click 'Dashboards' icon (4 squares) on left"
+echo "2. Click 'Simple Logs'"
+echo "3. Your logs appear immediately!"
+echo ""
+echo "TO FILTER:"
+echo "- Click any label to filter by it"
+echo "- Use the search box for text search"
+echo "- Time range selector is in top-right"
+echo "========================================="

simple-logs/logs.sh

@@ -0,0 +1,12 @@
+#!/bin/bash
+source "${AGENT_PATH}/common.sh"
+_check_required_env_vars "CONTAINER_NAME"
+
+echo "=== Grafana Logs ==="
+docker logs "${CONTAINER_NAME}_grafana" --tail 20
+echo ""
+echo "=== Loki Logs ==="
+docker logs "${CONTAINER_NAME}_loki" --tail 20
+echo ""
+echo "=== Promtail Logs ==="
+docker logs "${CONTAINER_NAME}_promtail" --tail 20

simple-logs/start.sh

@@ -0,0 +1,6 @@
+#!/bin/bash
+source "${AGENT_PATH}/common.sh"
+_check_required_env_vars "CONTAINER_NAME"
+
+docker compose up -d || _die "Failed to start Simple Logs"
+echo "Simple Logs started"

simple-logs/status.sh

@@ -0,0 +1,11 @@
+#!/bin/bash
+source "${AGENT_PATH}/common.sh"
+_check_required_env_vars "CONTAINER_NAME"
+
+if docker ps | grep -q "${CONTAINER_NAME}_grafana"; then
+    echo "Running"
+    echo "  Grafana: http://$(hostname -I | awk '{print $1}'):${WEB_PORT:-3000}"
+    docker ps --format "table {{.Names}}\t{{.Status}}" | grep "${CONTAINER_NAME}_"
+else
+    echo "Stopped"
+fi

simple-logs/stop.sh

@@ -0,0 +1,6 @@
+#!/bin/bash
+source "${AGENT_PATH}/common.sh"
+_check_required_env_vars "CONTAINER_NAME"
+
+docker compose stop || _die "Failed to stop Simple Logs"
+echo "Simple Logs stopped"

simple-logs/uninstall.sh

@@ -0,0 +1,6 @@
+#!/bin/bash
+source "${AGENT_PATH}/common.sh"
+_check_required_env_vars "CONTAINER_NAME"
+
+docker compose down || true
+echo "Simple Logs uninstalled (data preserved)"