swtich from ELK to Loki!
All checks were successful
Test and Publish Templates / test-and-publish (push) Successful in 40s
All checks were successful
Test and Publish Templates / test-and-publish (push) Successful in 40s
This commit is contained in:
Your Name
@@ -1,279 +0,0 @@
|
||||
# Dropshell LogServer Template
|
||||
|
||||
A comprehensive centralized logging solution using the ELK Stack (Elasticsearch, Logstash, Kibana) for receiving, processing, and visualizing logs from multiple hosts.
|
||||
|
||||
## Overview
|
||||
|
||||
This template deploys a full-featured ELK stack that:
|
||||
- Receives logs from multiple sources via Beats protocol
|
||||
- Stores and indexes logs in Elasticsearch
|
||||
- Provides powerful search and visualization through Kibana
|
||||
- Supports automatic log parsing and enrichment
|
||||
- Handles Docker container logs and system logs from clients
|
||||
|
||||
## Architecture
|
||||
|
||||
### Components
|
||||
|
||||
1. **Elasticsearch** (7.17.x)
|
||||
- Distributed search and analytics engine
|
||||
- Stores and indexes all log data
|
||||
- Provides fast full-text search capabilities
|
||||
- Single-node configuration for simplicity (can be scaled)
|
||||
|
||||
2. **Logstash** (7.17.x)
|
||||
- Log processing pipeline
|
||||
- Receives logs from Filebeat clients
|
||||
- Parses and enriches log data
|
||||
- Routes logs to appropriate Elasticsearch indices
|
||||
|
||||
3. **Kibana** (7.17.x)
|
||||
- Web UI for log exploration and visualization
|
||||
- Create dashboards and alerts
|
||||
- Real-time log streaming
|
||||
- Advanced search queries
|
||||
|
||||
## Features
|
||||
|
||||
### Minimum Configuration Design
|
||||
- Auto-discovery of log formats
|
||||
- Pre-configured dashboards for common services
|
||||
- Automatic index lifecycle management
|
||||
- Built-in parsing for Docker and syslog formats
|
||||
- Zero-configuration client connectivity
|
||||
|
||||
### Log Processing
|
||||
- Automatic timestamp extraction
|
||||
- Docker metadata enrichment (container name, image, labels)
|
||||
- Syslog parsing with severity levels
|
||||
- JSON log support
|
||||
- Multi-line log handling (stacktraces, etc.)
|
||||
- Grok pattern matching for common formats
|
||||
|
||||
### Security & Performance
|
||||
- **Mutual TLS (mTLS)** authentication for client connections
|
||||
- **API key authentication** as an alternative to certificates
|
||||
- **Per-client authentication** with unique keys/certificates
|
||||
- **SSL/TLS encryption** for all client connections
|
||||
- **Basic authentication** for Kibana web access
|
||||
- **IP whitelisting** for additional security
|
||||
- Index lifecycle management for storage optimization
|
||||
- Automatic old log cleanup
|
||||
- Resource limits to prevent overconsumption
|
||||
|
||||
## Port Configuration
|
||||
|
||||
- **5601**: Kibana Web UI (HTTP/HTTPS with authentication)
|
||||
- **9200**: Elasticsearch REST API (HTTP) - internal only
|
||||
- **5044**: Logstash Beats input (TCP/TLS) - authenticated client connections
|
||||
- **514**: Syslog input (UDP/TCP) - optional, unauthenticated
|
||||
- **24224**: Fluentd forward input - optional Docker logging driver
|
||||
|
||||
## Storage Requirements
|
||||
|
||||
- **Minimum**: 10GB for basic operation
|
||||
- **Recommended**: 50GB+ depending on log volume
|
||||
- **Log Retention**: Default 30 days (configurable)
|
||||
|
||||
## Client Authentication
|
||||
|
||||
### Authentication Methods
|
||||
|
||||
1. **Mutual TLS (mTLS) - Recommended**
|
||||
- Each client gets a unique certificate signed by the server's CA
|
||||
- Strongest security with mutual authentication
|
||||
- Automatic certificate validation
|
||||
|
||||
2. **API Keys**
|
||||
- Each client gets a unique API key
|
||||
- Simpler to manage than certificates
|
||||
- Good for environments where certificate management is difficult
|
||||
|
||||
3. **Basic Auth (Not Recommended)**
|
||||
- Shared username/password
|
||||
- Least secure, only for testing
|
||||
|
||||
### Client Configuration
|
||||
|
||||
Clients using the `logclient` template will:
|
||||
1. Authenticate using provided credentials (cert/key or API key)
|
||||
2. Establish encrypted TLS connection
|
||||
3. Ship all Docker container logs
|
||||
4. Ship system logs (syslog, auth, kernel)
|
||||
5. Maintain connection with automatic reconnection
|
||||
6. Buffer logs locally during network outages
|
||||
|
||||
## Dashboard Features
|
||||
|
||||
### Pre-configured Dashboards
|
||||
- **System Overview**: Overall health and log volume metrics
|
||||
- **Docker Containers**: Container-specific logs and metrics
|
||||
- **Error Analysis**: Aggregated error logs from all sources
|
||||
- **Security Events**: Authentication and access logs
|
||||
- **Application Logs**: Parsed application-specific logs
|
||||
|
||||
### Search Capabilities
|
||||
- Full-text search across all logs
|
||||
- Filter by time range, host, container, severity
|
||||
- Save and share search queries
|
||||
- Export search results
|
||||
|
||||
## Resource Requirements
|
||||
|
||||
### Minimum
|
||||
- CPU: 2 cores
|
||||
- RAM: 4GB
|
||||
- Storage: 10GB
|
||||
|
||||
### Recommended
|
||||
- CPU: 4+ cores
|
||||
- RAM: 8GB+
|
||||
- Storage: 50GB+ SSD
|
||||
|
||||
## Configuration Options
|
||||
|
||||
### Environment Variables (service.env)
|
||||
|
||||
```bash
|
||||
# Elasticsearch settings
|
||||
ES_HEAP_SIZE=2g
|
||||
ES_MAX_MAP_COUNT=262144
|
||||
|
||||
# Logstash settings
|
||||
LS_HEAP_SIZE=1g
|
||||
LS_PIPELINE_WORKERS=2
|
||||
|
||||
# Kibana settings
|
||||
KIBANA_PASSWORD=changeme
|
||||
KIBANA_BASE_PATH=/
|
||||
|
||||
# Log retention
|
||||
LOG_RETENTION_DAYS=30
|
||||
LOG_MAX_SIZE_GB=50
|
||||
|
||||
# Authentication Mode
|
||||
AUTH_MODE=mtls # Options: mtls, apikey, basic
|
||||
ENABLE_TLS=true
|
||||
|
||||
# mTLS Settings (if AUTH_MODE=mtls)
|
||||
CA_CERT_PATH=/certs/ca.crt
|
||||
SERVER_CERT_PATH=/certs/server.crt
|
||||
SERVER_KEY_PATH=/certs/server.key
|
||||
CLIENT_CERT_REQUIRED=true
|
||||
|
||||
# API Key Settings (if AUTH_MODE=apikey)
|
||||
API_KEYS_PATH=/config/api-keys.yml
|
||||
|
||||
# Network Security
|
||||
ALLOWED_IPS="" # Comma-separated list, empty = all
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
### Installation
|
||||
```bash
|
||||
dropshell install logserver
|
||||
```
|
||||
|
||||
### Generate Client Credentials
|
||||
|
||||
#### For mTLS Authentication:
|
||||
```bash
|
||||
# Generate client certificate for a new host
|
||||
dropshell exec logserver /scripts/generate-client-cert.sh hostname
|
||||
# This creates hostname.crt and hostname.key files
|
||||
```
|
||||
|
||||
#### For API Key Authentication:
|
||||
```bash
|
||||
# Generate API key for a new client
|
||||
dropshell exec logserver /scripts/generate-api-key.sh hostname
|
||||
# Returns an API key to configure in the client
|
||||
```
|
||||
|
||||
### Access Kibana
|
||||
Navigate to `https://<server-ip>:5601` in your browser.
|
||||
|
||||
Default credentials:
|
||||
- Username: `elastic`
|
||||
- Password: `changeme` (change in service.env)
|
||||
|
||||
### View Logs
|
||||
```bash
|
||||
dropshell logs logserver
|
||||
```
|
||||
|
||||
### Backup
|
||||
```bash
|
||||
dropshell backup logserver
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Common Issues
|
||||
|
||||
1. **Elasticsearch failing to start**
|
||||
- Check vm.max_map_count: `sysctl vm.max_map_count` (should be 262144+)
|
||||
- Verify sufficient memory available
|
||||
|
||||
2. **No logs appearing in Kibana**
|
||||
- Check Logstash is receiving data: port 5044 should be open
|
||||
- Verify client connectivity
|
||||
- Check index patterns in Kibana
|
||||
|
||||
3. **High memory usage**
|
||||
- Adjust heap sizes in service.env
|
||||
- Configure index lifecycle management
|
||||
- Reduce retention period
|
||||
|
||||
## Integration
|
||||
|
||||
This template is designed to work seamlessly with the `logclient` template. Simply:
|
||||
1. Deploy this logserver
|
||||
2. Deploy logclient on each host you want to monitor
|
||||
3. Configure logclient with the logserver address
|
||||
4. Logs will automatically start flowing
|
||||
|
||||
## Security Considerations
|
||||
|
||||
1. **Authentication Setup**
|
||||
- Use mTLS for production environments
|
||||
- Generate unique credentials for each client
|
||||
- Rotate certificates/keys regularly
|
||||
- Store credentials securely
|
||||
|
||||
2. **Network Security**
|
||||
- Always use TLS encryption for client connections
|
||||
- Configure IP whitelisting when possible
|
||||
- Use firewall rules to restrict access
|
||||
- Consider VPN or private networks
|
||||
|
||||
3. **Access Control**
|
||||
- Change default Kibana password immediately
|
||||
- Create read-only users for viewing logs
|
||||
- Implement role-based access control (RBAC)
|
||||
- Audit access logs regularly
|
||||
|
||||
4. **Data Protection**
|
||||
- Regular backups of Elasticsearch indices
|
||||
- Encrypt data at rest (optional)
|
||||
- Monitor disk usage to prevent data loss
|
||||
- Implement log retention policies
|
||||
|
||||
## Maintenance
|
||||
|
||||
### Daily Tasks
|
||||
- Monitor disk usage
|
||||
- Check for failed log shipments
|
||||
- Review error dashboards
|
||||
|
||||
### Weekly Tasks
|
||||
- Verify all clients are reporting
|
||||
- Check index health
|
||||
- Review and optimize slow queries
|
||||
|
||||
### Monthly Tasks
|
||||
- Update ELK stack components
|
||||
- Archive old indices
|
||||
- Review retention policies
|
||||
- Performance tuning based on usage patterns
|
||||
@@ -1,50 +1,73 @@
|
||||
# LogServer
|
||||
# Log Server (Loki + Grafana)
|
||||
|
||||
Centralized logging with ELK Stack (Elasticsearch, Logstash, Kibana).
|
||||
Central log server using Grafana + Loki. MUCH simpler than ELK/Kibana!
|
||||
|
||||
## Quick Start
|
||||
|
||||
1. **System Setup**
|
||||
```bash
|
||||
sudo sysctl -w vm.max_map_count=262144
|
||||
```
|
||||
|
||||
2. **Configure**
|
||||
Edit `config/service.env`:
|
||||
- Set `SERVER_PUBLICBASEURL` to your actual server URL
|
||||
- Set `KIBANA_USERNAME` to your preferred username
|
||||
- Change `KIBANA_USER_PASSWORD` from default
|
||||
|
||||
3. **Install**
|
||||
1. **Install**
|
||||
```bash
|
||||
dropshell install logserver
|
||||
```
|
||||
|
||||
4. **Generate Client Keys**
|
||||
2. **Access Web UI**
|
||||
- URL: `http://<server-ip>:3000`
|
||||
- Login: `admin` / `changeme` (set in service.env)
|
||||
|
||||
3. **View Logs**
|
||||
- Click **Dashboards** (4 squares icon)
|
||||
- Click **"Central Logs"**
|
||||
- All logs from all servers appear!
|
||||
|
||||
## For Client Servers
|
||||
|
||||
On each server that should send logs here:
|
||||
|
||||
1. Edit `logclient/config/service.env`:
|
||||
```bash
|
||||
./generate-api-key.sh
|
||||
# Enter hostname when prompted
|
||||
# Save the API key for client configuration
|
||||
LOGSERVER_HOST=<this-server-ip>
|
||||
LOGSERVER_PORT=3100
|
||||
LOKI_USER=logclient # Default username
|
||||
LOKI_PASSWORD=<your-password> # From server's service.env
|
||||
```
|
||||
|
||||
5. **Setup Kibana** (first time only)
|
||||
2. Install the client:
|
||||
```bash
|
||||
./setup-kibana.sh
|
||||
dropshell install logclient
|
||||
```
|
||||
|
||||
6. **Access Kibana**
|
||||
- URL: `http://<server-ip>:5601`
|
||||
- Username: Set in `service.env` (KIBANA_USERNAME, default: `admin`)
|
||||
- Password: Set in `service.env` (KIBANA_USER_PASSWORD)
|
||||
- Click "Discover" → View your logs!
|
||||
## Features
|
||||
- **Super simple interface** - Just one dashboard
|
||||
- **Lightweight** - Uses 10x less resources than ELK
|
||||
- **Real-time** - Logs appear instantly
|
||||
- **Multi-server** - See all servers in one place
|
||||
- **Automatic labels** - Filter by hostname, container, etc.
|
||||
|
||||
## Security
|
||||
- Loki endpoint requires authentication (HTTP Basic Auth)
|
||||
- Default: `logclient` / `changeme`
|
||||
- **IMPORTANT**: Change `LOKI_PASSWORD` in service.env!
|
||||
|
||||
## Ports
|
||||
- `5601` - Kibana Web UI
|
||||
- `5044` - Log ingestion (Filebeat)
|
||||
- `3000` - Grafana Web UI
|
||||
- `3100` - Loki API (authenticated)
|
||||
|
||||
## Files
|
||||
- `config/service.env` - Configuration
|
||||
- `config/api-keys.yml` - Client API keys
|
||||
- `generate-api-key.sh` - Add new clients
|
||||
## How to Use
|
||||
|
||||
See [DOCUMENTATION.md](DOCUMENTATION.md) for full details.
|
||||
### Filter by Server
|
||||
- Use the "Server" dropdown at the top
|
||||
- Or click any `hostname` label
|
||||
|
||||
### Filter by Container
|
||||
- Click any `container_name` label
|
||||
|
||||
### Search Text
|
||||
- Use the search box in each panel
|
||||
|
||||
### Time Range
|
||||
- Top-right corner - adjust as needed
|
||||
|
||||
## Why This Instead of ELK?
|
||||
- **10x simpler** - One clean dashboard vs Kibana complexity
|
||||
- **10x smaller** - ~200MB RAM vs 2-3GB for ELK
|
||||
- **Zero configuration** - Just works
|
||||
- **Fast setup** - No index patterns, mappings, or complexity
|
||||
@@ -1,100 +0,0 @@
|
||||
# LogServer Quick Setup Guide
|
||||
|
||||
## Prerequisites
|
||||
- Docker and Docker Compose installed
|
||||
- 4GB+ RAM, 10GB+ disk space
|
||||
- Port 5601 (Kibana) and 5044 (Logstash) available
|
||||
|
||||
## Initial Setup
|
||||
|
||||
### 1. System Configuration
|
||||
```bash
|
||||
# Required for Elasticsearch
|
||||
sudo sysctl -w vm.max_map_count=262144
|
||||
echo "vm.max_map_count=262144" | sudo tee -a /etc/sysctl.conf
|
||||
```
|
||||
|
||||
### 2. Configure Server
|
||||
Edit `config/service.env`:
|
||||
```bash
|
||||
# Change default password
|
||||
KIBANA_PASSWORD=your-secure-password
|
||||
```
|
||||
|
||||
### 3. Install
|
||||
```bash
|
||||
dropshell install logserver
|
||||
```
|
||||
|
||||
## Generate Client API Keys
|
||||
|
||||
Run the interactive key generator:
|
||||
```bash
|
||||
./generate-api-key.sh
|
||||
```
|
||||
|
||||
Follow the prompts:
|
||||
1. Enter hostname for each client
|
||||
2. Script generates secure API key
|
||||
3. Shows configuration to copy to client
|
||||
4. Repeat for additional clients
|
||||
|
||||
## Access Kibana
|
||||
|
||||
1. Open browser: `http://your-server-ip:5601`
|
||||
2. Login: `elastic` / `your-secure-password`
|
||||
3. Create index pattern: `filebeat-*`
|
||||
4. View logs in Discover tab
|
||||
|
||||
## Add Log Clients
|
||||
|
||||
On each client machine:
|
||||
```bash
|
||||
# Get API key from server admin (they run ./generate-api-key.sh)
|
||||
|
||||
# Edit logclient/config/service.env:
|
||||
LOGSERVER_HOST=your-server-ip
|
||||
LOGSERVER_PORT=5044
|
||||
API_KEY=your-api-key-here
|
||||
|
||||
# Install and start
|
||||
dropshell install logclient
|
||||
```
|
||||
|
||||
## Verify Setup
|
||||
|
||||
```bash
|
||||
# Check server status
|
||||
dropshell status logserver
|
||||
|
||||
# View server logs
|
||||
dropshell logs logserver
|
||||
|
||||
# Test client connection (from client)
|
||||
docker logs logclient-filebeat | grep "connection"
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
**Elasticsearch won't start**: Check `vm.max_map_count` is 262144+
|
||||
|
||||
**No logs in Kibana**:
|
||||
- Verify client can reach server port 5044
|
||||
- Check API key is correct in client's service.env
|
||||
- Verify API key exists in server's api-keys.yml
|
||||
- Refresh index pattern in Kibana
|
||||
|
||||
**High memory usage**: Adjust heap sizes in `service.env`:
|
||||
```bash
|
||||
ES_HEAP_SIZE=1g # Reduce from 2g
|
||||
LS_HEAP_SIZE=512m # Reduce from 1g
|
||||
```
|
||||
|
||||
## Security Checklist
|
||||
|
||||
- [ ] Changed default Kibana password
|
||||
- [ ] Generated unique API key per client
|
||||
- [ ] API keys stored securely
|
||||
- [ ] Firewall allows only necessary ports (5601, 5044)
|
||||
- [ ] Regular backup configured
|
||||
- [ ] Reviewed api-keys.yml for old/unused keys
|
||||
@@ -1,84 +0,0 @@
|
||||
# Simple Guide to Viewing Logs in Kibana
|
||||
|
||||
## First Time Setup
|
||||
Run this once after installing LogServer:
|
||||
```bash
|
||||
./setup-kibana.sh
|
||||
```
|
||||
|
||||
## Viewing Logs - The Easy Way
|
||||
|
||||
### 1. Open Kibana
|
||||
Go to: `http://<your-server>:5601`
|
||||
|
||||
### 2. Login
|
||||
Use the username and password from your service.env
|
||||
|
||||
### 3. Click "Discover"
|
||||
It's in the left menu (looks like a compass icon)
|
||||
|
||||
### 4. You're Done!
|
||||
Your logs are now visible. That's it!
|
||||
|
||||
## Simple Controls
|
||||
|
||||
### See Recent Logs Only
|
||||
- Top-right corner: Click the time picker
|
||||
- Choose "Last 15 minutes" or "Last 1 hour"
|
||||
|
||||
### Filter by Container
|
||||
- Find any log entry
|
||||
- Next to `container_name`: click the `+` button
|
||||
- Now you only see logs from that container
|
||||
|
||||
### Filter by Server
|
||||
- Next to `host.name`: click the `+` button
|
||||
- Now you only see logs from that host
|
||||
|
||||
### Search for Text
|
||||
- Top search bar: Type any word
|
||||
- Press Enter
|
||||
- Shows only logs containing that word
|
||||
|
||||
### Live Updates
|
||||
- Top-right: Click "Refresh"
|
||||
- Choose "Every 5 seconds"
|
||||
- Logs update automatically
|
||||
|
||||
### Remove Filters
|
||||
- Look for filter pills under the search bar
|
||||
- Click the `x` on any filter to remove it
|
||||
|
||||
## Common Searches
|
||||
|
||||
**Show errors only:**
|
||||
```
|
||||
error OR ERROR OR Error
|
||||
```
|
||||
|
||||
**Show warnings and errors:**
|
||||
```
|
||||
error OR ERROR OR warn OR WARN
|
||||
```
|
||||
|
||||
**Show specific container:**
|
||||
```
|
||||
container_name: "myapp"
|
||||
```
|
||||
|
||||
**Show multiple containers:**
|
||||
```
|
||||
container_name: ("app1" OR "app2")
|
||||
```
|
||||
|
||||
## Tips
|
||||
|
||||
1. **Too many columns?** Click "container_name" and "message" in the left sidebar to show just those
|
||||
|
||||
2. **Want raw logs?** Click the ">" arrow next to any log entry to expand it
|
||||
|
||||
3. **Export logs?** Click "Share" → "CSV Reports" → "Generate CSV"
|
||||
|
||||
4. **Time zone wrong?** Click your profile icon → "Advanced Settings" → search "timezone"
|
||||
|
||||
That's all you need to know! Kibana has many advanced features, but for basic log viewing and searching, these commands are sufficient.
|
||||
@@ -1,244 +0,0 @@
|
||||
# LogServer Template - Implementation TODO
|
||||
|
||||
## Phase 1: Core Infrastructure (Priority 1)
|
||||
|
||||
### Configuration Files
|
||||
- [ ] Create `config/.template_info.env` with template metadata
|
||||
- [ ] Create `config/service.env` with user-configurable settings
|
||||
- [ ] Define all required environment variables (ports, passwords, heap sizes)
|
||||
- [ ] Set appropriate default values for zero-config experience
|
||||
|
||||
### Docker Compose Setup
|
||||
- [ ] Create `docker-compose.yml` with ELK stack services
|
||||
- [ ] Configure Elasticsearch single-node setup
|
||||
- [ ] Configure Logstash with Beats input pipeline
|
||||
- [ ] Configure Kibana with Elasticsearch connection
|
||||
- [ ] Set up proper networking between services
|
||||
- [ ] Define named volumes for data persistence
|
||||
- [ ] Configure health checks for each service
|
||||
|
||||
### Required Scripts
|
||||
- [ ] Implement `install.sh` - Pull images, create volumes, start services
|
||||
- [ ] Implement `uninstall.sh` - Stop and remove containers (preserve volumes!)
|
||||
- [ ] Implement `start.sh` - Start all ELK services with docker-compose
|
||||
- [ ] Implement `stop.sh` - Gracefully stop all services
|
||||
- [ ] Implement `status.sh` - Check health of all three services
|
||||
|
||||
## Phase 2: Logstash Configuration (Priority 1)
|
||||
|
||||
### Input Configuration
|
||||
- [ ] Configure Beats input on port 5044 with TLS/SSL
|
||||
- [ ] Set up mutual TLS (mTLS) authentication
|
||||
- [ ] Configure client certificate validation
|
||||
- [ ] Add API key authentication option
|
||||
- [ ] Implement IP whitelisting
|
||||
- [ ] Add Syslog input on port 514 (UDP/TCP) - unauthenticated
|
||||
- [ ] Add Docker Fluentd input on port 24224 (optional)
|
||||
|
||||
### Filter Pipeline
|
||||
- [ ] Create Docker log parser (extract container metadata)
|
||||
- [ ] Create Syslog parser (RFC3164 and RFC5424)
|
||||
- [ ] Add JSON parser for structured logs
|
||||
- [ ] Implement multiline pattern for stack traces
|
||||
- [ ] Add timestamp extraction and normalization
|
||||
- [ ] Create field enrichment (add host metadata)
|
||||
- [ ] Implement conditional routing based on log type
|
||||
|
||||
### Output Configuration
|
||||
- [ ] Configure Elasticsearch output with index patterns
|
||||
- [ ] Set up index templates for different log types
|
||||
- [ ] Configure index lifecycle management (ILM)
|
||||
|
||||
## Phase 3: Elasticsearch Setup (Priority 1)
|
||||
|
||||
### System Configuration
|
||||
- [ ] Set appropriate heap size defaults (ES_HEAP_SIZE)
|
||||
- [ ] Configure vm.max_map_count requirement check
|
||||
- [ ] Set up single-node discovery settings
|
||||
- [ ] Configure data persistence volume
|
||||
- [ ] Set up index templates for:
|
||||
- [ ] Docker logs (docker-*)
|
||||
- [ ] System logs (syslog-*)
|
||||
- [ ] Application logs (app-*)
|
||||
- [ ] Error logs (errors-*)
|
||||
|
||||
### Index Management
|
||||
- [ ] Configure ILM policies for log rotation
|
||||
- [ ] Set retention period (default 30 days)
|
||||
- [ ] Configure max index size limits
|
||||
- [ ] Set up automatic cleanup of old indices
|
||||
- [ ] Create snapshot repository configuration
|
||||
|
||||
## Phase 4: Kibana Configuration (Priority 2)
|
||||
|
||||
### Initial Setup
|
||||
- [ ] Configure Kibana with Elasticsearch URL
|
||||
- [ ] Set up basic authentication
|
||||
- [ ] Configure server base path
|
||||
- [ ] Set appropriate memory limits
|
||||
|
||||
### Pre-built Dashboards
|
||||
- [ ] Create System Overview dashboard
|
||||
- [ ] Create Docker Containers dashboard
|
||||
- [ ] Create Error Analysis dashboard
|
||||
- [ ] Create Security Events dashboard
|
||||
- [ ] Create Host Metrics dashboard
|
||||
|
||||
### Saved Searches
|
||||
- [ ] Error logs across all sources
|
||||
- [ ] Authentication events
|
||||
- [ ] Container lifecycle events
|
||||
- [ ] Slow queries/performance issues
|
||||
- [ ] Critical system events
|
||||
|
||||
### Index Patterns
|
||||
- [ ] Configure docker-* pattern
|
||||
- [ ] Configure syslog-* pattern
|
||||
- [ ] Configure app-* pattern
|
||||
- [ ] Configure filebeat-* pattern
|
||||
|
||||
## Phase 5: Optional Scripts (Priority 2)
|
||||
|
||||
### Operational Scripts
|
||||
- [ ] Implement `logs.sh` - Show logs from all ELK services
|
||||
- [ ] Implement `backup.sh` - Snapshot Elasticsearch indices
|
||||
- [ ] Implement `restore.sh` - Restore from snapshots
|
||||
- [ ] Implement `destroy.sh` - Complete removal including volumes
|
||||
- [ ] Implement `ports.sh` - Display all exposed ports
|
||||
- [ ] Implement `ssh.sh` - Shell into specific container
|
||||
|
||||
### Helper Scripts
|
||||
- [ ] Create `_volumes.sh` for volume management helpers
|
||||
- [ ] Add health check script for all services
|
||||
- [ ] Create performance tuning script
|
||||
- [ ] Add certificate generation script for SSL
|
||||
|
||||
## Phase 6: Security Features (Priority 1 - CRITICAL)
|
||||
|
||||
### Certificate Authority Setup
|
||||
- [ ] Create CA certificate and key for signing client certs
|
||||
- [ ] Generate server certificate for Logstash
|
||||
- [ ] Create certificate generation script for clients
|
||||
- [ ] Set up certificate storage structure
|
||||
- [ ] Implement certificate rotation mechanism
|
||||
|
||||
### mTLS Authentication
|
||||
- [ ] Configure Logstash for mutual TLS
|
||||
- [ ] Set up client certificate validation
|
||||
- [ ] Create client certificate generation script
|
||||
- [ ] Implement certificate revocation list (CRL)
|
||||
- [ ] Add certificate expiry monitoring
|
||||
|
||||
### API Key Authentication
|
||||
- [ ] Create API key generation script
|
||||
- [ ] Configure Logstash to accept API keys
|
||||
- [ ] Implement API key storage (encrypted)
|
||||
- [ ] Add API key rotation mechanism
|
||||
- [ ] Create API key revocation process
|
||||
|
||||
### Network Security
|
||||
- [ ] Implement IP whitelisting in Logstash
|
||||
- [ ] Configure firewall rules
|
||||
- [ ] Set up rate limiting
|
||||
- [ ] Add connection throttling
|
||||
- [ ] Implement DDoS protection
|
||||
|
||||
### Kibana Security
|
||||
- [ ] Configure Kibana HTTPS
|
||||
- [ ] Set up basic authentication
|
||||
- [ ] Create user management scripts
|
||||
- [ ] Implement session management
|
||||
- [ ] Add audit logging
|
||||
|
||||
## Phase 7: Performance & Optimization (Priority 3)
|
||||
|
||||
### Resource Management
|
||||
- [ ] Configure CPU limits for each service
|
||||
- [ ] Set memory limits appropriately
|
||||
- [ ] Add swap handling configuration
|
||||
- [ ] Configure JVM options files
|
||||
- [ ] Add performance monitoring
|
||||
|
||||
### Optimization
|
||||
- [ ] Configure pipeline workers
|
||||
- [ ] Set batch sizes for optimal throughput
|
||||
- [ ] Configure queue sizes
|
||||
- [ ] Add caching configuration
|
||||
- [ ] Optimize index refresh intervals
|
||||
|
||||
## Phase 8: Testing & Documentation (Priority 3)
|
||||
|
||||
### Testing
|
||||
- [ ] Test installation process
|
||||
- [ ] Test uninstall (verify volume preservation)
|
||||
- [ ] Test log ingestion from sample client
|
||||
- [ ] Test all dashboard functionality
|
||||
- [ ] Test backup and restore procedures
|
||||
- [ ] Load test with high log volume
|
||||
- [ ] Test failover and recovery
|
||||
|
||||
### Documentation
|
||||
- [ ] Create README.txt for dropshell format
|
||||
- [ ] Document all configuration options
|
||||
- [ ] Add troubleshooting guide
|
||||
- [ ] Create quick start guide
|
||||
- [ ] Document upgrade procedures
|
||||
- [ ] Add performance tuning guide
|
||||
|
||||
## Phase 9: Integration Testing (Priority 3)
|
||||
|
||||
### With LogClient
|
||||
- [ ] Test automatic discovery
|
||||
- [ ] Verify log flow from client to server
|
||||
- [ ] Test reconnection scenarios
|
||||
- [ ] Verify all log types are parsed correctly
|
||||
- [ ] Test SSL communication
|
||||
- [ ] Measure end-to-end latency
|
||||
|
||||
### Compatibility Testing
|
||||
- [ ] Test with different Docker versions
|
||||
- [ ] Test on various Linux distributions
|
||||
- [ ] Verify with different log formats
|
||||
- [ ] Test with high-volume producers
|
||||
- [ ] Validate resource usage
|
||||
|
||||
## Phase 10: Production Readiness (Priority 4)
|
||||
|
||||
### Monitoring & Alerting
|
||||
- [ ] Add Elasticsearch monitoring
|
||||
- [ ] Configure disk space alerts
|
||||
- [ ] Set up index health monitoring
|
||||
- [ ] Add performance metrics collection
|
||||
- [ ] Create alert rules in Kibana
|
||||
|
||||
### Maintenance Features
|
||||
- [ ] Add automatic update check
|
||||
- [ ] Create maintenance mode
|
||||
- [ ] Add data export functionality
|
||||
- [ ] Create migration scripts
|
||||
- [ ] Add configuration validation
|
||||
|
||||
## Notes
|
||||
|
||||
### Design Principles
|
||||
1. **Minimum configuration**: Should work with just `dropshell install logserver`
|
||||
2. **Data safety**: Never delete volumes in uninstall.sh
|
||||
3. **Non-interactive**: All scripts must run without user input
|
||||
4. **Idempotent**: Scripts can be run multiple times safely
|
||||
5. **Clear feedback**: Provide clear status and error messages
|
||||
|
||||
### Dependencies
|
||||
- Docker and Docker Compose
|
||||
- Sufficient system resources (4GB+ RAM recommended)
|
||||
- Network connectivity for clients
|
||||
- Persistent storage for logs
|
||||
|
||||
### Testing Checklist
|
||||
- [ ] All required scripts present and executable
|
||||
- [ ] Template validates with dropshell test-template
|
||||
- [ ] Services start and connect properly
|
||||
- [ ] Logs flow from client to Kibana
|
||||
- [ ] Data persists across container restarts
|
||||
- [ ] Uninstall preserves data volumes
|
||||
- [ ] Resource limits are enforced
|
||||
- [ ] Error handling works correctly
|
||||
@@ -1,9 +0,0 @@
|
||||
#!/bin/bash
|
||||
# Define volume items for logserver container
|
||||
# These are used across backup, restore, create, and destroy operations
|
||||
|
||||
get_logserver_volumes() {
|
||||
echo "volume:elasticsearch_data:${CONTAINER_NAME}_elasticsearch_data" \
|
||||
"volume:logstash_data:${CONTAINER_NAME}_logstash_data" \
|
||||
"volume:kibana_data:${CONTAINER_NAME}_kibana_data"
|
||||
}
|
||||
@@ -1,37 +0,0 @@
|
||||
#!/bin/bash
|
||||
source "${AGENT_PATH}/common.sh"
|
||||
source "$(dirname "${BASH_SOURCE[0]}")/_volumes.sh"
|
||||
|
||||
_check_required_env_vars "CONTAINER_NAME"
|
||||
|
||||
echo "Backing up ${CONTAINER_NAME} ELK stack volumes..."
|
||||
echo "Note: This may take a while for large log databases"
|
||||
|
||||
# Stop the containers to ensure data consistency
|
||||
bash ./stop.sh || true
|
||||
|
||||
# Backup volumes
|
||||
BACKUP_DIR="${CONFIG_PATH}/backups/$(date +%Y%m%d_%H%M%S)"
|
||||
mkdir -p "$BACKUP_DIR"
|
||||
|
||||
# Export volumes
|
||||
for volume in $(get_logserver_volumes); do
|
||||
volume_name=$(echo $volume | cut -d: -f3)
|
||||
echo "Backing up volume: $volume_name"
|
||||
docker run --rm -v "$volume_name:/source:ro" -v "$BACKUP_DIR:/backup" alpine \
|
||||
tar -czf "/backup/${volume_name}.tar.gz" -C /source .
|
||||
done
|
||||
|
||||
# Backup configuration
|
||||
cp -r "${CONFIG_PATH}" "$BACKUP_DIR/config_backup"
|
||||
|
||||
# Backup docker-compose.yml
|
||||
cp docker-compose.yml "$BACKUP_DIR/" 2>/dev/null || true
|
||||
|
||||
echo "Backup completed to: $BACKUP_DIR"
|
||||
echo "Size: $(du -sh $BACKUP_DIR | cut -f1)"
|
||||
|
||||
# Restart the containers
|
||||
bash ./start.sh
|
||||
|
||||
echo "ELK stack restarted"
|
||||
@@ -1,17 +0,0 @@
|
||||
# Template identifier - MUST match the directory name
|
||||
TEMPLATE=logserver
|
||||
|
||||
# Requirements
|
||||
REQUIRES_HOST_ROOT=false # No root access on host needed
|
||||
REQUIRES_DOCKER=true # Docker is required
|
||||
REQUIRES_DOCKER_ROOT=false # Docker root privileges not specifically needed
|
||||
|
||||
# Docker compose used for ELK stack
|
||||
USES_DOCKER_COMPOSE=true
|
||||
|
||||
# Volume definitions for persistence
|
||||
DATA_VOLUME="${CONTAINER_NAME}_elasticsearch_data"
|
||||
LOGSTASH_VOLUME="${CONTAINER_NAME}_logstash_data"
|
||||
KIBANA_VOLUME="${CONTAINER_NAME}_kibana_data"
|
||||
CERTS_VOLUME="${CONTAINER_NAME}_certs"
|
||||
CONFIG_VOLUME="${CONTAINER_NAME}_config"
|
||||
@@ -1,6 +0,0 @@
|
||||
# API Keys for LogServer Authentication
|
||||
# Format: hostname:api_key
|
||||
# Generated by generate-api-key.sh
|
||||
|
||||
api_keys:
|
||||
video: a7798c63c2ac439b5ba20f3bf8bf27b5361231cdcbdc4fc9d7af715308fdf707
|
||||
154
logserver/config/dashboards/central-logs.json
Normal file
154
logserver/config/dashboards/central-logs.json
Normal file
@@ -0,0 +1,154 @@
|
||||
{
|
||||
"annotations": {
|
||||
"list": [
|
||||
{
|
||||
"builtIn": 1,
|
||||
"datasource": "-- Grafana --",
|
||||
"enable": true,
|
||||
"hide": true,
|
||||
"iconColor": "rgba(0, 211, 255, 1)",
|
||||
"name": "Annotations & Alerts",
|
||||
"type": "dashboard"
|
||||
}
|
||||
]
|
||||
},
|
||||
"editable": true,
|
||||
"gnetId": null,
|
||||
"graphTooltip": 0,
|
||||
"id": null,
|
||||
"links": [],
|
||||
"panels": [
|
||||
{
|
||||
"datasource": "Loki",
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"custom": {}
|
||||
},
|
||||
"overrides": []
|
||||
},
|
||||
"gridPos": {
|
||||
"h": 3,
|
||||
"w": 24,
|
||||
"x": 0,
|
||||
"y": 0
|
||||
},
|
||||
"id": 4,
|
||||
"options": {
|
||||
"content": "## Central Log Viewer\n\n**Quick Filters:** Click any label (hostname, container_name, job) to filter | **Search:** Use the search box above each panel | **Time Range:** Top right corner",
|
||||
"mode": "markdown"
|
||||
},
|
||||
"pluginVersion": "7.5.7",
|
||||
"title": "",
|
||||
"type": "text"
|
||||
},
|
||||
{
|
||||
"datasource": "Loki",
|
||||
"fieldConfig": {
|
||||
"defaults": {},
|
||||
"overrides": []
|
||||
},
|
||||
"gridPos": {
|
||||
"h": 10,
|
||||
"w": 24,
|
||||
"x": 0,
|
||||
"y": 3
|
||||
},
|
||||
"id": 2,
|
||||
"options": {
|
||||
"dedupStrategy": "none",
|
||||
"enableLogDetails": true,
|
||||
"prettifyLogMessage": false,
|
||||
"showCommonLabels": false,
|
||||
"showLabels": true,
|
||||
"showTime": true,
|
||||
"sortOrder": "Descending",
|
||||
"wrapLogMessage": true
|
||||
},
|
||||
"targets": [
|
||||
{
|
||||
"expr": "{job=\"docker\"}",
|
||||
"legendFormat": "{{hostname}} - {{container_name}}",
|
||||
"refId": "A"
|
||||
}
|
||||
],
|
||||
"title": "Docker Container Logs (All Servers)",
|
||||
"type": "logs"
|
||||
},
|
||||
{
|
||||
"datasource": "Loki",
|
||||
"fieldConfig": {
|
||||
"defaults": {},
|
||||
"overrides": []
|
||||
},
|
||||
"gridPos": {
|
||||
"h": 8,
|
||||
"w": 24,
|
||||
"x": 0,
|
||||
"y": 13
|
||||
},
|
||||
"id": 3,
|
||||
"options": {
|
||||
"dedupStrategy": "none",
|
||||
"enableLogDetails": true,
|
||||
"prettifyLogMessage": false,
|
||||
"showCommonLabels": false,
|
||||
"showLabels": true,
|
||||
"showTime": true,
|
||||
"sortOrder": "Descending",
|
||||
"wrapLogMessage": true
|
||||
},
|
||||
"targets": [
|
||||
{
|
||||
"expr": "{job=~\"syslog|auth\"}",
|
||||
"legendFormat": "{{hostname}} - {{job}}",
|
||||
"refId": "A"
|
||||
}
|
||||
],
|
||||
"title": "System Logs (All Servers)",
|
||||
"type": "logs"
|
||||
}
|
||||
],
|
||||
"refresh": "10s",
|
||||
"schemaVersion": 27,
|
||||
"style": "dark",
|
||||
"tags": ["logs", "central"],
|
||||
"templating": {
|
||||
"list": [
|
||||
{
|
||||
"allValue": ".*",
|
||||
"current": {
|
||||
"selected": true,
|
||||
"text": ["All"],
|
||||
"value": ["$__all"]
|
||||
},
|
||||
"datasource": "Loki",
|
||||
"definition": "label_values(hostname)",
|
||||
"description": "Filter by server",
|
||||
"error": null,
|
||||
"hide": 0,
|
||||
"includeAll": true,
|
||||
"label": "Server",
|
||||
"multi": true,
|
||||
"name": "hostname",
|
||||
"options": [],
|
||||
"query": "label_values(hostname)",
|
||||
"refresh": 1,
|
||||
"regex": "",
|
||||
"skipUrlSync": false,
|
||||
"sort": 1,
|
||||
"type": "query"
|
||||
}
|
||||
]
|
||||
},
|
||||
"time": {
|
||||
"from": "now-15m",
|
||||
"to": "now"
|
||||
},
|
||||
"timepicker": {
|
||||
"refresh_intervals": ["5s", "10s", "30s", "1m", "5m", "15m", "30m", "1h"]
|
||||
},
|
||||
"timezone": "",
|
||||
"title": "Central Logs",
|
||||
"uid": "central-logs",
|
||||
"version": 0
|
||||
}
|
||||
69
logserver/config/dashboards/simple-logs.json
Normal file
69
logserver/config/dashboards/simple-logs.json
Normal file
@@ -0,0 +1,69 @@
|
||||
{
|
||||
"annotations": {
|
||||
"list": [
|
||||
{
|
||||
"builtIn": 1,
|
||||
"datasource": "-- Grafana --",
|
||||
"enable": true,
|
||||
"hide": true,
|
||||
"iconColor": "rgba(0, 211, 255, 1)",
|
||||
"name": "Annotations & Alerts",
|
||||
"type": "dashboard"
|
||||
}
|
||||
]
|
||||
},
|
||||
"editable": true,
|
||||
"gnetId": null,
|
||||
"graphTooltip": 0,
|
||||
"id": null,
|
||||
"links": [],
|
||||
"panels": [
|
||||
{
|
||||
"datasource": "Loki",
|
||||
"fieldConfig": {
|
||||
"defaults": {},
|
||||
"overrides": []
|
||||
},
|
||||
"gridPos": {
|
||||
"h": 20,
|
||||
"w": 24,
|
||||
"x": 0,
|
||||
"y": 0
|
||||
},
|
||||
"id": 2,
|
||||
"options": {
|
||||
"dedupStrategy": "none",
|
||||
"enableLogDetails": true,
|
||||
"prettifyLogMessage": false,
|
||||
"showCommonLabels": false,
|
||||
"showLabels": true,
|
||||
"showTime": true,
|
||||
"sortOrder": "Descending",
|
||||
"wrapLogMessage": true
|
||||
},
|
||||
"targets": [
|
||||
{
|
||||
"expr": "{job=~\"docker|syslog|auth\"}",
|
||||
"refId": "A"
|
||||
}
|
||||
],
|
||||
"title": "All Logs",
|
||||
"type": "logs"
|
||||
}
|
||||
],
|
||||
"schemaVersion": 27,
|
||||
"style": "dark",
|
||||
"tags": ["logs"],
|
||||
"templating": {
|
||||
"list": []
|
||||
},
|
||||
"time": {
|
||||
"from": "now-15m",
|
||||
"to": "now"
|
||||
},
|
||||
"timepicker": {},
|
||||
"timezone": "",
|
||||
"title": "Simple Logs",
|
||||
"uid": "simple-logs",
|
||||
"version": 0
|
||||
}
|
||||
12
logserver/config/grafana-dashboard.yml
Normal file
12
logserver/config/grafana-dashboard.yml
Normal file
@@ -0,0 +1,12 @@
|
||||
apiVersion: 1
|
||||
|
||||
providers:
|
||||
- name: 'default'
|
||||
orgId: 1
|
||||
folder: ''
|
||||
type: file
|
||||
disableDeletion: false
|
||||
updateIntervalSeconds: 10
|
||||
allowUiUpdates: true
|
||||
options:
|
||||
path: /var/lib/grafana/dashboards
|
||||
9
logserver/config/grafana-datasources.yml
Normal file
9
logserver/config/grafana-datasources.yml
Normal file
@@ -0,0 +1,9 @@
|
||||
apiVersion: 1
|
||||
|
||||
datasources:
|
||||
- name: Loki
|
||||
type: loki
|
||||
access: proxy
|
||||
url: http://loki:3100
|
||||
isDefault: true
|
||||
editable: false
|
||||
@@ -1,142 +0,0 @@
|
||||
# Logstash Configuration for LogServer
|
||||
# Handles Beats input with API key authentication
|
||||
|
||||
input {
|
||||
# Beats input for Filebeat clients
|
||||
beats {
|
||||
port => 5044
|
||||
ssl => false # Set to true for production with proper certificates
|
||||
|
||||
# API key authentication handled via filter below
|
||||
}
|
||||
|
||||
# Optional: Syslog input for direct syslog shipping
|
||||
tcp {
|
||||
port => 514
|
||||
type => "syslog"
|
||||
}
|
||||
|
||||
udp {
|
||||
port => 514
|
||||
type => "syslog"
|
||||
}
|
||||
}
|
||||
|
||||
filter {
|
||||
# API Key validation - check if client provided a valid key
|
||||
# The API key should be in the [fields][api_key] field from Filebeat
|
||||
if [fields][api_key] {
|
||||
# Load and validate API key
|
||||
ruby {
|
||||
init => "
|
||||
require 'yaml'
|
||||
@api_keys = {}
|
||||
|
||||
# Load API keys from file
|
||||
begin
|
||||
if File.exist?('/usr/share/logstash/config/api-keys.yml')
|
||||
config = YAML.load_file('/usr/share/logstash/config/api-keys.yml')
|
||||
if config && config['api_keys']
|
||||
config['api_keys'].each do |hostname, key|
|
||||
@api_keys[key.to_s.strip] = hostname.to_s.strip
|
||||
end
|
||||
end
|
||||
end
|
||||
rescue => e
|
||||
@logger.error('Failed to load API keys', :error => e.message)
|
||||
end
|
||||
"
|
||||
code => "
|
||||
api_key = event.get('[fields][api_key]')
|
||||
|
||||
if api_key && @api_keys.has_key?(api_key)
|
||||
# Valid API key - add hostname to event
|
||||
event.set('[@metadata][client_hostname]', @api_keys[api_key])
|
||||
event.set('[@metadata][authenticated]', true)
|
||||
else
|
||||
# Invalid API key
|
||||
event.set('[@metadata][authenticated]', false)
|
||||
event.tag('_authfailure')
|
||||
end
|
||||
"
|
||||
}
|
||||
|
||||
# Drop unauthorized events
|
||||
if "_authfailure" in [tags] {
|
||||
drop { }
|
||||
}
|
||||
} else {
|
||||
# No API key provided - mark as unauthenticated
|
||||
# You can choose to drop these or allow them based on your security requirements
|
||||
mutate {
|
||||
add_tag => [ "no_api_key" ]
|
||||
}
|
||||
# Uncomment to require API keys for all connections:
|
||||
# drop { }
|
||||
}
|
||||
|
||||
# Parse Docker logs
|
||||
if [docker] {
|
||||
# Docker metadata is already parsed by Filebeat
|
||||
mutate {
|
||||
add_field => {
|
||||
"container_name" => "%{[docker][container][name]}"
|
||||
"container_id" => "%{[docker][container][id]}"
|
||||
"container_image" => "%{[docker][container][image]}"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
# Parse syslog
|
||||
if [type] == "syslog" {
|
||||
grok {
|
||||
match => {
|
||||
"message" => "%{SYSLOGLINE}"
|
||||
}
|
||||
}
|
||||
date {
|
||||
match => [ "timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss" ]
|
||||
}
|
||||
}
|
||||
|
||||
# Parse JSON logs if they exist
|
||||
if [message] =~ /^\{.*\}$/ {
|
||||
json {
|
||||
source => "message"
|
||||
target => "json_message"
|
||||
}
|
||||
}
|
||||
|
||||
# Add timestamp if not present
|
||||
if ![timestamp] {
|
||||
mutate {
|
||||
add_field => { "timestamp" => "%{@timestamp}" }
|
||||
}
|
||||
}
|
||||
|
||||
# Clean up metadata
|
||||
mutate {
|
||||
remove_field => [ "@version", "beat", "offset", "prospector" ]
|
||||
}
|
||||
}
|
||||
|
||||
output {
|
||||
# Send to Elasticsearch with authentication
|
||||
elasticsearch {
|
||||
hosts => ["elasticsearch:9200"]
|
||||
user => "elastic"
|
||||
password => "${ELASTIC_PASSWORD:changeme}"
|
||||
|
||||
# Use different indices based on input type
|
||||
index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"
|
||||
|
||||
# Manage index templates
|
||||
manage_template => true
|
||||
template_overwrite => true
|
||||
}
|
||||
|
||||
# Optional: Debug output (comment out in production)
|
||||
# stdout {
|
||||
# codec => rubydebug
|
||||
# }
|
||||
}
|
||||
@@ -1,29 +0,0 @@
|
||||
# Logstash Configuration Settings
|
||||
# This file contains Logstash settings (not pipeline configuration)
|
||||
|
||||
# Node name
|
||||
node.name: "logstash"
|
||||
|
||||
# Pipeline settings
|
||||
pipeline.workers: 2
|
||||
pipeline.batch.size: 125
|
||||
pipeline.batch.delay: 50
|
||||
pipeline.ecs_compatibility: disabled
|
||||
|
||||
# HTTP API settings
|
||||
http.host: "0.0.0.0"
|
||||
http.port: 9600
|
||||
|
||||
# Monitoring
|
||||
monitoring.enabled: false
|
||||
|
||||
# Queue settings
|
||||
queue.type: memory
|
||||
queue.max_bytes: 1gb
|
||||
|
||||
# Path settings are handled by Docker volumes
|
||||
# path.data: /usr/share/logstash/data
|
||||
# path.logs: /usr/share/logstash/logs
|
||||
|
||||
# Log level
|
||||
log.level: info
|
||||
51
logserver/config/loki.yaml
Normal file
51
logserver/config/loki.yaml
Normal file
@@ -0,0 +1,51 @@
|
||||
auth_enabled: false
|
||||
|
||||
server:
|
||||
http_listen_port: 3100
|
||||
grpc_listen_port: 9096
|
||||
|
||||
ingester:
|
||||
wal:
|
||||
enabled: true
|
||||
dir: /loki/wal
|
||||
lifecycler:
|
||||
address: 127.0.0.1
|
||||
ring:
|
||||
kvstore:
|
||||
store: inmemory
|
||||
replication_factor: 1
|
||||
|
||||
schema_config:
|
||||
configs:
|
||||
- from: 2020-10-24
|
||||
store: boltdb-shipper
|
||||
object_store: filesystem
|
||||
schema: v11
|
||||
index:
|
||||
prefix: index_
|
||||
period: 24h
|
||||
|
||||
storage_config:
|
||||
boltdb_shipper:
|
||||
active_index_directory: /loki/boltdb-shipper-active
|
||||
cache_location: /loki/boltdb-shipper-cache
|
||||
cache_ttl: 24h
|
||||
filesystem:
|
||||
directory: /loki/chunks
|
||||
|
||||
compactor:
|
||||
working_directory: /loki/boltdb-shipper-compactor
|
||||
|
||||
limits_config:
|
||||
enforce_metric_name: false
|
||||
reject_old_samples: true
|
||||
reject_old_samples_max_age: 168h
|
||||
max_entries_limit_per_query: 5000
|
||||
retention_period: 168h # 7 days
|
||||
|
||||
chunk_store_config:
|
||||
max_look_back_period: 168h
|
||||
|
||||
table_manager:
|
||||
retention_deletes_enabled: true
|
||||
retention_period: 168h
|
||||
30
logserver/config/nginx.conf
Normal file
30
logserver/config/nginx.conf
Normal file
@@ -0,0 +1,30 @@
|
||||
events {
|
||||
worker_connections 1024;
|
||||
}
|
||||
|
||||
http {
|
||||
upstream loki {
|
||||
server loki:3100;
|
||||
}
|
||||
|
||||
server {
|
||||
listen 80;
|
||||
|
||||
# Require authentication for all requests
|
||||
auth_basic "Loki Authentication";
|
||||
auth_basic_user_file /etc/nginx/.htpasswd;
|
||||
|
||||
# Proxy all requests to Loki
|
||||
location / {
|
||||
proxy_pass http://loki;
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_set_header X-Forwarded-Proto $scheme;
|
||||
|
||||
# Important for Loki push endpoint
|
||||
client_max_body_size 100M;
|
||||
client_body_buffer_size 10M;
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,46 +1,20 @@
|
||||
# Service identification
|
||||
# Log Server Configuration (Loki + Grafana)
|
||||
CONTAINER_NAME=logserver
|
||||
|
||||
# Server settings (REQUIRED by dropshell)
|
||||
SSH_USER="root"
|
||||
|
||||
# Elasticsearch settings
|
||||
ES_VERSION=7.17.23
|
||||
ES_HEAP_SIZE=2g
|
||||
ES_MAX_MAP_COUNT=262144
|
||||
|
||||
# Logstash settings
|
||||
LS_VERSION=7.17.23
|
||||
LS_HEAP_SIZE=1g
|
||||
LS_PIPELINE_WORKERS=2
|
||||
|
||||
# Kibana settings
|
||||
KIBANA_VERSION=7.17.23
|
||||
|
||||
# Authentication (IMPORTANT: Change these!)
|
||||
ELASTIC_PASSWORD=changeme # Password for 'elastic' superuser (internal use)
|
||||
KIBANA_USERNAME=admin # Your login username for Kibana
|
||||
KIBANA_USER_PASSWORD=changeme # Your login password for Kibana
|
||||
|
||||
# Ports
|
||||
KIBANA_PORT=5601
|
||||
LOGSTASH_BEATS_PORT=5044
|
||||
LOGSTASH_SYSLOG_PORT=514
|
||||
WEB_PORT=3000 # Grafana web UI
|
||||
LOKI_PORT=3100 # Loki API (for clients to send logs)
|
||||
|
||||
# Server configuration
|
||||
SERVER_PUBLICBASEURL=http://localhost:5601 # Change to your server's actual URL
|
||||
# Authentication for Grafana
|
||||
ADMIN_USER=admin
|
||||
ADMIN_PASSWORD=changeme
|
||||
|
||||
# Log retention
|
||||
LOG_RETENTION_DAYS=30
|
||||
LOG_MAX_SIZE_GB=50
|
||||
# Authentication for Loki (clients must use these)
|
||||
LOKI_USER=logclient
|
||||
LOKI_PASSWORD=changeme
|
||||
|
||||
# Authentication
|
||||
ENABLE_TLS=true
|
||||
API_KEYS_FILE=${CONFIG_PATH}/api-keys.yml
|
||||
|
||||
# Network Security
|
||||
ALLOWED_IPS="" # Comma-separated list, empty = all
|
||||
|
||||
# Resource limits
|
||||
MAX_CPU_PERCENT=80
|
||||
MAX_MEMORY=4GB
|
||||
# Log retention (days)
|
||||
LOG_RETENTION=7
|
||||
@@ -1,43 +0,0 @@
|
||||
# Ruby script for Logstash to validate API keys
|
||||
# This is a simplified validation - in production, use proper authentication
|
||||
|
||||
require 'yaml'
|
||||
|
||||
def register(params)
|
||||
@api_keys_file = params["api_keys_file"]
|
||||
end
|
||||
|
||||
def filter(event)
|
||||
# Get the API key from the event
|
||||
api_key = event.get("[api_key]") || event.get("[@metadata][api_key]")
|
||||
|
||||
# If no API key, pass through (for backwards compatibility)
|
||||
# In production, you should reject events without valid keys
|
||||
if api_key.nil? || api_key.empty?
|
||||
# For now, allow events without API keys
|
||||
# event.cancel # Uncomment to require API keys
|
||||
return [event]
|
||||
end
|
||||
|
||||
# Load API keys from file
|
||||
begin
|
||||
if File.exist?(@api_keys_file)
|
||||
config = YAML.load_file(@api_keys_file)
|
||||
valid_keys = config['api_keys'].values if config && config['api_keys']
|
||||
|
||||
# Check if the provided key is valid
|
||||
if valid_keys && valid_keys.include?(api_key)
|
||||
# Valid key - let the event through
|
||||
event.set("[@metadata][authenticated]", true)
|
||||
else
|
||||
# Invalid key - drop the event
|
||||
event.cancel
|
||||
end
|
||||
end
|
||||
rescue => e
|
||||
# Log error but don't crash
|
||||
event.set("[@metadata][auth_error]", e.message)
|
||||
end
|
||||
|
||||
return [event]
|
||||
end
|
||||
@@ -1,26 +0,0 @@
|
||||
#!/bin/bash
|
||||
source "${AGENT_PATH}/common.sh"
|
||||
source "$(dirname "${BASH_SOURCE[0]}")/_volumes.sh"
|
||||
|
||||
_check_required_env_vars "CONTAINER_NAME"
|
||||
|
||||
echo "WARNING: This will PERMANENTLY DELETE all ELK stack data!"
|
||||
echo "This includes all logs, dashboards, and configurations"
|
||||
echo "This action cannot be undone!"
|
||||
echo ""
|
||||
|
||||
# Since scripts must be non-interactive, only proceed if explicitly called
|
||||
# Stop all containers
|
||||
docker compose down || true
|
||||
|
||||
# Remove all volumes
|
||||
for volume in $(get_logserver_volumes); do
|
||||
volume_name=$(echo $volume | cut -d: -f3)
|
||||
echo "Removing volume: $volume_name"
|
||||
docker volume rm "$volume_name" 2>/dev/null || true
|
||||
done
|
||||
|
||||
# Also remove the Docker network if it exists
|
||||
docker network rm "${CONTAINER_NAME}_elk" 2>/dev/null || true
|
||||
|
||||
echo "ELK stack and all data destroyed"
|
||||
@@ -1,80 +1,57 @@
|
||||
version: '3.8'
|
||||
|
||||
services:
|
||||
elasticsearch:
|
||||
image: docker.elastic.co/elasticsearch/elasticsearch:${ES_VERSION:-7.17.23}
|
||||
container_name: ${CONTAINER_NAME}_elasticsearch
|
||||
# Grafana - Simple Web UI
|
||||
grafana:
|
||||
image: grafana/grafana:latest
|
||||
container_name: ${CONTAINER_NAME}_grafana
|
||||
environment:
|
||||
- discovery.type=single-node
|
||||
- "ES_JAVA_OPTS=-Xms${ES_HEAP_SIZE:-2g} -Xmx${ES_HEAP_SIZE:-2g}"
|
||||
- xpack.security.enabled=true
|
||||
- xpack.security.authc.api_key.enabled=true
|
||||
- ELASTIC_PASSWORD=${ELASTIC_PASSWORD:-${KIBANA_PASSWORD:-changeme}}
|
||||
- xpack.monitoring.enabled=false
|
||||
- cluster.routing.allocation.disk.threshold_enabled=false
|
||||
- GF_SECURITY_ADMIN_USER=${ADMIN_USER:-admin}
|
||||
- GF_SECURITY_ADMIN_PASSWORD=${ADMIN_PASSWORD:-changeme}
|
||||
- GF_USERS_ALLOW_SIGN_UP=false
|
||||
volumes:
|
||||
- elasticsearch_data:/usr/share/elasticsearch/data
|
||||
- grafana_data:/var/lib/grafana
|
||||
- ${CONFIG_PATH}/grafana-datasources.yml:/etc/grafana/provisioning/datasources/datasources.yaml:ro
|
||||
- ${CONFIG_PATH}/grafana-dashboard.yml:/etc/grafana/provisioning/dashboards/dashboard.yaml:ro
|
||||
- ${CONFIG_PATH}/dashboards:/var/lib/grafana/dashboards:ro
|
||||
ports:
|
||||
- "127.0.0.1:9200:9200"
|
||||
networks:
|
||||
- elk
|
||||
- "${WEB_PORT:-3000}:3000"
|
||||
restart: unless-stopped
|
||||
ulimits:
|
||||
memlock:
|
||||
soft: -1
|
||||
hard: -1
|
||||
nofile:
|
||||
soft: 65536
|
||||
hard: 65536
|
||||
|
||||
logstash:
|
||||
image: docker.elastic.co/logstash/logstash:${LS_VERSION:-7.17.23}
|
||||
container_name: ${CONTAINER_NAME}_logstash
|
||||
environment:
|
||||
- "LS_JAVA_OPTS=-Xms${LS_HEAP_SIZE:-1g} -Xmx${LS_HEAP_SIZE:-1g}"
|
||||
- "xpack.monitoring.enabled=false"
|
||||
- ELASTIC_PASSWORD=${ELASTIC_PASSWORD:-${KIBANA_PASSWORD:-changeme}}
|
||||
- CONTAINER_NAME=${CONTAINER_NAME}
|
||||
command: logstash -f /usr/share/logstash/config/logstash.conf
|
||||
volumes:
|
||||
- ${CONFIG_PATH}:/usr/share/logstash/config
|
||||
- logstash_data:/usr/share/logstash/data
|
||||
ports:
|
||||
- "${LOGSTASH_BEATS_PORT:-5044}:5044"
|
||||
- "${LOGSTASH_SYSLOG_PORT:-514}:514/tcp"
|
||||
- "${LOGSTASH_SYSLOG_PORT:-514}:514/udp"
|
||||
networks:
|
||||
- elk
|
||||
depends_on:
|
||||
- elasticsearch
|
||||
restart: unless-stopped
|
||||
|
||||
kibana:
|
||||
image: docker.elastic.co/kibana/kibana:${KIBANA_VERSION:-7.17.23}
|
||||
container_name: ${CONTAINER_NAME}_kibana
|
||||
environment:
|
||||
- ELASTICSEARCH_HOSTS=http://elasticsearch:9200
|
||||
- ELASTICSEARCH_USERNAME=elastic
|
||||
- ELASTICSEARCH_PASSWORD=${ELASTIC_PASSWORD:-${KIBANA_PASSWORD:-changeme}}
|
||||
- XPACK_SECURITY_ENABLED=true
|
||||
- NODE_OPTIONS=--openssl-legacy-provider
|
||||
- SERVER_PUBLICBASEURL=${SERVER_PUBLICBASEURL:-http://localhost:5601}
|
||||
volumes:
|
||||
- kibana_data:/usr/share/kibana/data
|
||||
ports:
|
||||
- "${KIBANA_PORT:-5601}:5601"
|
||||
- loki
|
||||
networks:
|
||||
- elk
|
||||
depends_on:
|
||||
- elasticsearch
|
||||
- loki-net
|
||||
|
||||
# Loki - Log storage (not exposed directly)
|
||||
loki:
|
||||
image: grafana/loki:latest
|
||||
container_name: ${CONTAINER_NAME}_loki
|
||||
volumes:
|
||||
- loki_data:/loki
|
||||
- ${CONFIG_PATH}/loki.yaml:/etc/loki/local-config.yaml:ro
|
||||
command: -config.file=/etc/loki/local-config.yaml
|
||||
restart: unless-stopped
|
||||
networks:
|
||||
- loki-net
|
||||
|
||||
# Nginx - Authentication proxy for Loki
|
||||
nginx:
|
||||
image: nginx:alpine
|
||||
container_name: ${CONTAINER_NAME}_nginx
|
||||
volumes:
|
||||
- ${CONFIG_PATH}/nginx.conf:/etc/nginx/nginx.conf:ro
|
||||
- ${CONFIG_PATH}/.htpasswd:/etc/nginx/.htpasswd:ro
|
||||
ports:
|
||||
- "${LOKI_PORT:-3100}:80"
|
||||
restart: unless-stopped
|
||||
depends_on:
|
||||
- loki
|
||||
networks:
|
||||
- loki-net
|
||||
|
||||
networks:
|
||||
elk:
|
||||
driver: bridge
|
||||
loki-net:
|
||||
|
||||
volumes:
|
||||
elasticsearch_data:
|
||||
name: ${CONTAINER_NAME}_elasticsearch_data
|
||||
logstash_data:
|
||||
name: ${CONTAINER_NAME}_logstash_data
|
||||
kibana_data:
|
||||
name: ${CONTAINER_NAME}_kibana_data
|
||||
grafana_data:
|
||||
loki_data:
|
||||
@@ -1,172 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# Interactive API Key Generation Script for LogServer
|
||||
# This script generates secure API keys and adds them to api-keys.yml
|
||||
|
||||
# Determine where to put the api-keys.yml file
|
||||
determine_api_keys_location() {
|
||||
# 1. If api-keys.yml already exists in current folder, use it
|
||||
if [ -f "./api-keys.yml" ]; then
|
||||
echo "./api-keys.yml"
|
||||
return 0
|
||||
fi
|
||||
|
||||
# 2. If service.env exists in current folder, put keys here
|
||||
if [ -f "./service.env" ]; then
|
||||
echo "./api-keys.yml"
|
||||
return 0
|
||||
fi
|
||||
|
||||
# 3. If config folder exists, put keys there
|
||||
if [ -d "./config" ]; then
|
||||
echo "./config/api-keys.yml"
|
||||
return 0
|
||||
fi
|
||||
|
||||
# No valid location found
|
||||
return 1
|
||||
}
|
||||
|
||||
# Try to determine location
|
||||
if API_KEYS_FILE=$(determine_api_keys_location); then
|
||||
: # Location found, continue
|
||||
else
|
||||
echo -e "${RED}Error: Cannot determine where to place api-keys.yml${NC}"
|
||||
echo ""
|
||||
echo "This script must be run from one of these locations:"
|
||||
echo " 1. A deployed service directory (contains service.env)"
|
||||
echo " 2. The logserver template directory (contains config/ folder)"
|
||||
echo " 3. A directory with existing api-keys.yml file"
|
||||
echo ""
|
||||
echo "Current directory: $(pwd)"
|
||||
echo "Contents: $(ls -la 2>/dev/null | head -5)"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Colors for output
|
||||
RED='\033[0;31m'
|
||||
GREEN='\033[0;32m'
|
||||
YELLOW='\033[1;33m'
|
||||
NC='\033[0m' # No Color
|
||||
|
||||
# Generate a secure random API key
|
||||
generate_key() {
|
||||
openssl rand -hex 32 2>/dev/null || cat /dev/urandom | tr -dc 'a-zA-Z0-9' | fold -w 64 | head -n 1
|
||||
}
|
||||
|
||||
# Initialize api-keys.yml if it doesn't exist
|
||||
init_api_keys_file() {
|
||||
if [ ! -f "$API_KEYS_FILE" ]; then
|
||||
# Create directory if needed
|
||||
local dir=$(dirname "$API_KEYS_FILE")
|
||||
if [ ! -d "$dir" ]; then
|
||||
mkdir -p "$dir"
|
||||
echo -e "${GREEN}Created directory: $dir${NC}"
|
||||
fi
|
||||
|
||||
echo "# API Keys for LogServer Authentication" > "$API_KEYS_FILE"
|
||||
echo "# Format: hostname:api_key" >> "$API_KEYS_FILE"
|
||||
echo "# Generated by generate-api-key.sh" >> "$API_KEYS_FILE"
|
||||
echo "" >> "$API_KEYS_FILE"
|
||||
echo "api_keys:" >> "$API_KEYS_FILE"
|
||||
echo -e "${GREEN}Created new api-keys.yml file at: $API_KEYS_FILE${NC}"
|
||||
else
|
||||
echo -e "${GREEN}Using existing api-keys.yml at: $API_KEYS_FILE${NC}"
|
||||
fi
|
||||
}
|
||||
|
||||
# Check if hostname already has a key
|
||||
check_existing_key() {
|
||||
local hostname=$1
|
||||
if grep -q "^ ${hostname}:" "$API_KEYS_FILE" 2>/dev/null; then
|
||||
return 0
|
||||
fi
|
||||
return 1
|
||||
}
|
||||
|
||||
# Add key to api-keys.yml
|
||||
add_key_to_file() {
|
||||
local hostname=$1
|
||||
local api_key=$2
|
||||
echo " ${hostname}: ${api_key}" >> "$API_KEYS_FILE"
|
||||
}
|
||||
|
||||
# Main script
|
||||
echo -e "${GREEN}=== LogServer API Key Generator ===${NC}"
|
||||
echo ""
|
||||
|
||||
# Initialize file if needed
|
||||
init_api_keys_file
|
||||
|
||||
# Interactive mode
|
||||
while true; do
|
||||
echo -e "${YELLOW}Enter hostname for the client (or 'done' to finish):${NC}"
|
||||
read -p "> " hostname
|
||||
|
||||
if [ "$hostname" = "done" ] || [ -z "$hostname" ]; then
|
||||
break
|
||||
fi
|
||||
|
||||
# Validate hostname - allow simple names, must start with alphanumeric
|
||||
if [[ ! "$hostname" =~ ^[a-zA-Z0-9]([a-zA-Z0-9._-]*[a-zA-Z0-9])?$ ]]; then
|
||||
echo -e "${RED}Invalid hostname format. Use only letters, numbers, dots, dashes, and underscores.${NC}"
|
||||
echo -e "${RED}Hostname must start and end with a letter or number.${NC}"
|
||||
continue
|
||||
fi
|
||||
|
||||
# Check if key already exists
|
||||
if check_existing_key "$hostname"; then
|
||||
echo -e "${YELLOW}Key already exists for ${hostname}${NC}"
|
||||
read -p "Generate new key? (y/n): " overwrite
|
||||
if [ "$overwrite" != "y" ]; then
|
||||
continue
|
||||
fi
|
||||
# Remove old key
|
||||
sed -i "/^ ${hostname}:/d" "$API_KEYS_FILE"
|
||||
fi
|
||||
|
||||
# Generate new key
|
||||
api_key=$(generate_key)
|
||||
|
||||
# Add to file
|
||||
add_key_to_file "$hostname" "$api_key"
|
||||
|
||||
echo -e "${GREEN}✓ Generated API key for ${hostname}${NC}"
|
||||
echo ""
|
||||
echo "Configuration for ${hostname}:"
|
||||
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
|
||||
echo "Add to client's service.env:"
|
||||
echo ""
|
||||
echo "LOGSERVER_HOST=$(hostname -I | awk '{print $1}')"
|
||||
echo "LOGSERVER_PORT=5044"
|
||||
echo "API_KEY=${api_key}"
|
||||
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
|
||||
echo ""
|
||||
|
||||
# Option to add more
|
||||
read -p "Add another client? (y/n): " add_more
|
||||
if [ "$add_more" != "y" ]; then
|
||||
break
|
||||
fi
|
||||
done
|
||||
|
||||
# Show summary
|
||||
echo ""
|
||||
echo -e "${GREEN}=== Summary ===${NC}"
|
||||
echo "API keys file: $API_KEYS_FILE"
|
||||
echo "Total clients configured: $(grep -c "^ " "$API_KEYS_FILE" 2>/dev/null || echo 0)"
|
||||
echo ""
|
||||
echo "To view all keys: cat $API_KEYS_FILE"
|
||||
echo "To revoke a key: Edit $API_KEYS_FILE and remove the line"
|
||||
echo ""
|
||||
|
||||
# Show location-specific restart instructions
|
||||
if [[ "$API_KEYS_FILE" == "./api-keys.yml" ]] && [ -f "./service.env" ]; then
|
||||
# We're in a deployed service directory
|
||||
echo -e "${YELLOW}Remember to restart the service to apply changes:${NC}"
|
||||
echo " dropshell restart logserver"
|
||||
else
|
||||
# We're in the template directory
|
||||
echo -e "${YELLOW}Note: Deploy this template to use these keys:${NC}"
|
||||
echo " dropshell install logserver"
|
||||
fi
|
||||
@@ -2,128 +2,48 @@
|
||||
source "${AGENT_PATH}/common.sh"
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
|
||||
# Check required environment variables
|
||||
_check_required_env_vars "CONTAINER_NAME" "ES_VERSION" "LS_VERSION" "KIBANA_VERSION"
|
||||
_check_required_env_vars "CONTAINER_NAME"
|
||||
|
||||
# Check Docker and Docker Compose are available
|
||||
# Check Docker
|
||||
_check_docker_installed || _die "Docker test failed"
|
||||
docker compose version >/dev/null 2>&1 || _die "Docker Compose is not installed (requires Docker Compose V2)"
|
||||
|
||||
# Check vm.max_map_count for Elasticsearch
|
||||
current_max_map_count=$(sysctl -n vm.max_map_count 2>/dev/null || echo 0)
|
||||
if [ "$current_max_map_count" -lt 262144 ]; then
|
||||
echo "WARNING: vm.max_map_count is too low ($current_max_map_count)"
|
||||
echo "Elasticsearch requires at least 262144"
|
||||
echo "Please run: sudo sysctl -w vm.max_map_count=262144"
|
||||
echo "And add to /etc/sysctl.conf to persist"
|
||||
_die "System configuration needs adjustment"
|
||||
fi
|
||||
|
||||
# Check available memory
|
||||
available_mem=$(free -m | awk '/^Mem:/{print $7}')
|
||||
if [ "$available_mem" -lt 3000 ]; then
|
||||
echo "WARNING: Low available memory (${available_mem}MB)"
|
||||
echo "ELK stack requires at least 3-4GB free memory for proper operation"
|
||||
echo "Services may take longer to start or fail to start"
|
||||
echo ""
|
||||
fi
|
||||
docker compose version >/dev/null 2>&1 || _die "Docker Compose V2 is required"
|
||||
|
||||
# Stop any existing containers
|
||||
bash ./stop.sh || true
|
||||
bash ./stop.sh 2>/dev/null || true
|
||||
|
||||
# Remove old containers
|
||||
docker compose down --remove-orphans 2>/dev/null || true
|
||||
# Create config directory
|
||||
mkdir -p "${CONFIG_PATH}/dashboards"
|
||||
|
||||
# Pull the Docker images
|
||||
echo "Pulling ELK stack images..."
|
||||
docker pull docker.elastic.co/elasticsearch/elasticsearch:${ES_VERSION} || _die "Failed to pull Elasticsearch"
|
||||
docker pull docker.elastic.co/logstash/logstash:${LS_VERSION} || _die "Failed to pull Logstash"
|
||||
docker pull docker.elastic.co/kibana/kibana:${KIBANA_VERSION} || _die "Failed to pull Kibana"
|
||||
# Copy configuration files
|
||||
cp "$SCRIPT_DIR/config/"*.yaml "$SCRIPT_DIR/config/"*.yml "$SCRIPT_DIR/config/"*.conf "${CONFIG_PATH}/" 2>/dev/null || true
|
||||
cp "$SCRIPT_DIR/config/dashboards/"*.json "${CONFIG_PATH}/dashboards/" 2>/dev/null || true
|
||||
|
||||
# Create volumes using common function
|
||||
source "$SCRIPT_DIR/_volumes.sh"
|
||||
echo "Creating volumes..."
|
||||
create_items $(get_logserver_volumes)
|
||||
|
||||
# Ensure config directory exists
|
||||
mkdir -p "${CONFIG_PATH}"
|
||||
|
||||
# Initialize API keys file if it doesn't exist
|
||||
if [ ! -f "${CONFIG_PATH}/api-keys.yml" ]; then
|
||||
echo "No API keys configured yet."
|
||||
echo "Run ./generate-api-key.sh to add client keys"
|
||||
echo "api_keys:" > "${CONFIG_PATH}/api-keys.yml"
|
||||
# Generate htpasswd file for Loki authentication
|
||||
echo "Generating authentication file..."
|
||||
# Use openssl to generate htpasswd (available on most systems)
|
||||
if command -v openssl >/dev/null 2>&1; then
|
||||
# Generate password hash
|
||||
PASS_HASH=$(openssl passwd -apr1 "${LOKI_PASSWORD:-changeme}")
|
||||
echo "${LOKI_USER:-logclient}:$PASS_HASH" > "${CONFIG_PATH}/.htpasswd"
|
||||
elif command -v htpasswd >/dev/null 2>&1; then
|
||||
# Use htpasswd if available
|
||||
htpasswd -cb "${CONFIG_PATH}/.htpasswd" "${LOKI_USER:-logclient}" "${LOKI_PASSWORD:-changeme}"
|
||||
else
|
||||
echo "WARNING: Cannot generate password file - no openssl or htpasswd found"
|
||||
echo "Using basic auth with plain text (NOT SECURE):"
|
||||
echo "${LOKI_USER:-logclient}:${LOKI_PASSWORD:-changeme}" > "${CONFIG_PATH}/.htpasswd"
|
||||
fi
|
||||
|
||||
# Copy Logstash configurations if they don't exist
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
# Start the stack
|
||||
echo "Starting Log Server..."
|
||||
docker compose up -d || _die "Failed to start"
|
||||
|
||||
if [ ! -f "${CONFIG_PATH}/logstash.conf" ]; then
|
||||
if [ -f "$SCRIPT_DIR/config/logstash.conf" ]; then
|
||||
cp "$SCRIPT_DIR/config/logstash.conf" "${CONFIG_PATH}/logstash.conf"
|
||||
echo "Copied Logstash pipeline configuration to ${CONFIG_PATH}"
|
||||
else
|
||||
echo "WARNING: logstash.conf not found in template"
|
||||
fi
|
||||
fi
|
||||
|
||||
if [ ! -f "${CONFIG_PATH}/logstash.yml" ]; then
|
||||
if [ -f "$SCRIPT_DIR/config/logstash.yml" ]; then
|
||||
cp "$SCRIPT_DIR/config/logstash.yml" "${CONFIG_PATH}/logstash.yml"
|
||||
echo "Copied Logstash settings to ${CONFIG_PATH}"
|
||||
else
|
||||
echo "WARNING: logstash.yml not found in template"
|
||||
fi
|
||||
fi
|
||||
|
||||
# Start the ELK stack
|
||||
echo "Starting ELK stack..."
|
||||
docker compose up -d --build || _die "Failed to start ELK stack"
|
||||
|
||||
# Wait for services to be ready with polling
|
||||
echo "Waiting for services to start (this can take 2-3 minutes on first run)..."
|
||||
MAX_WAIT=240 # Maximum 4 minutes
|
||||
# Wait for services
|
||||
echo -n "Waiting for services to start..."
|
||||
MAX_WAIT=60
|
||||
WAITED=0
|
||||
while [ $WAITED -lt $MAX_WAIT ]; do
|
||||
# Check if all services are running
|
||||
if bash ./status.sh 2>/dev/null | grep -q "Running"; then
|
||||
echo " All services are up!"
|
||||
break
|
||||
fi
|
||||
|
||||
# Show progress with time elapsed
|
||||
if [ $((WAITED % 10)) -eq 0 ] && [ $WAITED -gt 0 ]; then
|
||||
echo -n " [${WAITED}s]"
|
||||
else
|
||||
echo -n "."
|
||||
fi
|
||||
|
||||
sleep 2
|
||||
WAITED=$((WAITED + 2))
|
||||
done
|
||||
echo ""
|
||||
|
||||
if [ $WAITED -ge $MAX_WAIT ]; then
|
||||
echo "Warning: Services are still starting. This is normal on first run."
|
||||
echo "Current status:"
|
||||
bash ./status.sh || true
|
||||
echo ""
|
||||
echo "You can check the logs with:"
|
||||
echo " docker logs ${CONTAINER_NAME}_elasticsearch"
|
||||
echo " docker logs ${CONTAINER_NAME}_logstash"
|
||||
echo " docker logs ${CONTAINER_NAME}_kibana"
|
||||
echo ""
|
||||
echo "The services will continue starting in the background."
|
||||
fi
|
||||
|
||||
# Create custom user
|
||||
echo "Setting up custom user '${KIBANA_USERNAME:-admin}'..."
|
||||
echo -n "Waiting for Elasticsearch API..."
|
||||
|
||||
# First wait for Elasticsearch to be ready
|
||||
WAITED=0
|
||||
while [ $WAITED -lt 60 ]; do
|
||||
if docker exec ${CONTAINER_NAME}_elasticsearch curl -s -u elastic:${ELASTIC_PASSWORD} http://localhost:9200/_cluster/health 2>/dev/null | grep -q '"status":"yellow"\|"status":"green"'; then
|
||||
if curl -s "http://localhost:${WEB_PORT:-3000}/api/health" 2>/dev/null | grep -q "ok"; then
|
||||
echo " Ready!"
|
||||
break
|
||||
fi
|
||||
@@ -131,56 +51,28 @@ while [ $WAITED -lt 60 ]; do
|
||||
sleep 2
|
||||
WAITED=$((WAITED + 2))
|
||||
done
|
||||
echo ""
|
||||
|
||||
if [ $WAITED -lt 60 ]; then
|
||||
# Now create the user
|
||||
docker exec ${CONTAINER_NAME}_elasticsearch bash -c "
|
||||
|
||||
result=\$(curl -s -X POST -u elastic:${ELASTIC_PASSWORD} \
|
||||
-H 'Content-Type: application/json' \
|
||||
http://localhost:9200/_security/user/${KIBANA_USERNAME:-admin} \
|
||||
-d '{
|
||||
\"password\" : \"${KIBANA_USER_PASSWORD:-changeme}\",
|
||||
\"roles\" : [ \"superuser\" ],
|
||||
\"full_name\" : \"Admin User\"
|
||||
}' 2>/dev/null)
|
||||
|
||||
if echo \"\$result\" | grep -q '\"created\":true'; then
|
||||
echo \"User created successfully\"
|
||||
else
|
||||
echo \"User already exists (this is fine)\"
|
||||
fi
|
||||
"
|
||||
else
|
||||
echo "Warning: Elasticsearch API not ready after 60 seconds"
|
||||
fi
|
||||
|
||||
echo "Installation of ${CONTAINER_NAME} complete"
|
||||
echo ""
|
||||
echo "========================================="
|
||||
echo "Kibana UI: ${SERVER_PUBLICBASEURL:-http://$(hostname -I | awk '{print $1}'):${KIBANA_PORT}}"
|
||||
echo ""
|
||||
echo "Login with your custom user:"
|
||||
echo " Username: ${KIBANA_USERNAME:-admin}"
|
||||
echo " Password: ${KIBANA_USER_PASSWORD:-changeme}"
|
||||
echo ""
|
||||
echo "Or the superuser:"
|
||||
echo " Username: elastic"
|
||||
echo " Password: ${ELASTIC_PASSWORD:-changeme}"
|
||||
echo "Log Server Installed!"
|
||||
echo "========================================="
|
||||
echo ""
|
||||
# Only show reminder if using default values
|
||||
if [ "${ELASTIC_PASSWORD}" = "changeme" ] || [ -z "${SERVER_PUBLICBASEURL}" ] || [ "${SERVER_PUBLICBASEURL}" = "http://localhost:5601" ]; then
|
||||
echo "REMINDER: Update service.env with:"
|
||||
if [ "${ELASTIC_PASSWORD}" = "changeme" ]; then
|
||||
echo " - A secure password in ELASTIC_PASSWORD"
|
||||
fi
|
||||
if [ -z "${SERVER_PUBLICBASEURL}" ] || [ "${SERVER_PUBLICBASEURL}" = "http://localhost:5601" ]; then
|
||||
echo " - Your actual server IP/domain in SERVER_PUBLICBASEURL"
|
||||
fi
|
||||
echo ""
|
||||
fi
|
||||
echo "Logstash listening on port ${LOGSTASH_BEATS_PORT} for Filebeat clients"
|
||||
echo "Web UI: http://$(hostname -I | awk '{print $1}'):${WEB_PORT:-3000}"
|
||||
echo "Login: ${ADMIN_USER:-admin} / ${ADMIN_PASSWORD:-changeme}"
|
||||
echo ""
|
||||
echo "To add client authentication:"
|
||||
echo " ./generate-api-key.sh"
|
||||
echo "TO VIEW LOGS:"
|
||||
echo "1. Click 'Dashboards' (4 squares icon)"
|
||||
echo "2. Click 'Central Logs'"
|
||||
echo "3. See all logs from all servers!"
|
||||
echo ""
|
||||
echo "FOR CLIENTS TO SEND LOGS HERE:"
|
||||
echo "Server: $(hostname -I | awk '{print $1}')"
|
||||
echo "Port: ${LOKI_PORT:-3100}"
|
||||
echo "Username: ${LOKI_USER:-logclient}"
|
||||
echo "Password: ${LOKI_PASSWORD:-changeme}"
|
||||
echo ""
|
||||
echo "IMPORTANT: Change LOKI_PASSWORD in service.env!"
|
||||
echo ""
|
||||
echo "Install 'logclient' on other servers"
|
||||
echo "========================================="
|
||||
@@ -1,56 +0,0 @@
|
||||
#!/bin/bash
|
||||
source "${AGENT_PATH}/common.sh"
|
||||
source "$(dirname "${BASH_SOURCE[0]}")/_volumes.sh"
|
||||
|
||||
_check_required_env_vars "CONTAINER_NAME"
|
||||
|
||||
if [ -z "$1" ]; then
|
||||
echo "Usage: $0 <backup_directory>"
|
||||
echo "Available backups:"
|
||||
ls -la "${CONFIG_PATH}/backups/" 2>/dev/null || echo "No backups found"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
BACKUP_DIR="$1"
|
||||
|
||||
if [ ! -d "$BACKUP_DIR" ]; then
|
||||
_die "Backup directory not found: $BACKUP_DIR"
|
||||
fi
|
||||
|
||||
echo "Restoring ELK stack from backup: $BACKUP_DIR"
|
||||
echo "WARNING: This will overwrite all current data!"
|
||||
|
||||
# Stop the containers
|
||||
bash ./stop.sh || true
|
||||
|
||||
# Restore volumes
|
||||
for volume in $(get_logserver_volumes); do
|
||||
volume_name=$(echo $volume | cut -d: -f3)
|
||||
backup_file="$BACKUP_DIR/${volume_name}.tar.gz"
|
||||
|
||||
if [ -f "$backup_file" ]; then
|
||||
echo "Restoring volume: $volume_name"
|
||||
# Clear existing data and restore
|
||||
docker run --rm -v "$volume_name:/target" -v "$BACKUP_DIR:/backup:ro" alpine \
|
||||
sh -c "rm -rf /target/* && tar -xzf /backup/${volume_name}.tar.gz -C /target"
|
||||
else
|
||||
echo "Warning: Backup file not found for $volume_name"
|
||||
fi
|
||||
done
|
||||
|
||||
# Restore configuration if exists
|
||||
if [ -d "$BACKUP_DIR/config_backup" ]; then
|
||||
cp -r "$BACKUP_DIR/config_backup/"* "${CONFIG_PATH}/"
|
||||
fi
|
||||
|
||||
# Restore docker-compose.yml if exists
|
||||
if [ -f "$BACKUP_DIR/docker-compose.yml" ]; then
|
||||
cp "$BACKUP_DIR/docker-compose.yml" .
|
||||
fi
|
||||
|
||||
echo "Restore completed"
|
||||
|
||||
# Start the containers
|
||||
bash ./start.sh
|
||||
|
||||
echo "ELK stack restarted with restored data"
|
||||
@@ -1,33 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# Script to create a custom user for Kibana login
|
||||
# This runs after Elasticsearch is up and creates the user defined in service.env
|
||||
|
||||
# Wait for Elasticsearch to be ready
|
||||
echo "Waiting for Elasticsearch to be ready..."
|
||||
until curl -s -u elastic:${ELASTIC_PASSWORD} http://localhost:9200/_cluster/health | grep -q '"status":"yellow"\|"status":"green"'; do
|
||||
sleep 5
|
||||
done
|
||||
|
||||
echo "Creating user '${KIBANA_USERNAME}'..."
|
||||
|
||||
# Create the custom user with superuser role
|
||||
curl -X POST -u elastic:${ELASTIC_PASSWORD} \
|
||||
-H "Content-Type: application/json" \
|
||||
http://localhost:9200/_security/user/${KIBANA_USERNAME} \
|
||||
-d '{
|
||||
"password" : "'${KIBANA_USER_PASSWORD}'",
|
||||
"roles" : [ "superuser" ],
|
||||
"full_name" : "Admin User",
|
||||
"email" : "admin@example.com"
|
||||
}'
|
||||
|
||||
if [ $? -eq 0 ]; then
|
||||
echo ""
|
||||
echo "User '${KIBANA_USERNAME}' created successfully!"
|
||||
echo "You can now log in to Kibana with:"
|
||||
echo " Username: ${KIBANA_USERNAME}"
|
||||
echo " Password: ${KIBANA_USER_PASSWORD}"
|
||||
else
|
||||
echo "Note: User might already exist or there was an error"
|
||||
fi
|
||||
@@ -1,108 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# Setup Kibana for simple log viewing
|
||||
# This creates index patterns and saved searches for easy log access
|
||||
|
||||
source "${AGENT_PATH}/common.sh"
|
||||
_check_required_env_vars "CONTAINER_NAME" "ELASTIC_PASSWORD" "KIBANA_PORT"
|
||||
|
||||
KIBANA_URL="http://localhost:${KIBANA_PORT}"
|
||||
AUTH="elastic:${ELASTIC_PASSWORD}"
|
||||
|
||||
echo "Setting up Kibana for simple log viewing..."
|
||||
echo ""
|
||||
|
||||
# Wait for Kibana to be ready
|
||||
echo -n "Waiting for Kibana to be ready..."
|
||||
MAX_WAIT=60
|
||||
WAITED=0
|
||||
while [ $WAITED -lt $MAX_WAIT ]; do
|
||||
if docker exec ${CONTAINER_NAME}_kibana curl -s -u "$AUTH" "${KIBANA_URL}/api/status" 2>/dev/null | grep -q '"level":"available"'; then
|
||||
echo " Ready!"
|
||||
break
|
||||
fi
|
||||
echo -n "."
|
||||
sleep 2
|
||||
WAITED=$((WAITED + 2))
|
||||
done
|
||||
|
||||
if [ $WAITED -ge $MAX_WAIT ]; then
|
||||
echo ""
|
||||
echo "ERROR: Kibana is not ready after ${MAX_WAIT} seconds"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Create index pattern for Filebeat
|
||||
echo "Creating Filebeat index pattern..."
|
||||
docker exec ${CONTAINER_NAME}_kibana curl -s -X POST \
|
||||
-u "$AUTH" \
|
||||
-H "Content-Type: application/json" \
|
||||
-H "kbn-xsrf: true" \
|
||||
"${KIBANA_URL}/api/saved_objects/index-pattern/filebeat-*" \
|
||||
-d '{
|
||||
"attributes": {
|
||||
"title": "filebeat-*",
|
||||
"timeFieldName": "@timestamp",
|
||||
"fields": "[]"
|
||||
}
|
||||
}' > /dev/null 2>&1
|
||||
|
||||
# Set as default index pattern
|
||||
docker exec ${CONTAINER_NAME}_kibana curl -s -X POST \
|
||||
-u "$AUTH" \
|
||||
-H "Content-Type: application/json" \
|
||||
-H "kbn-xsrf: true" \
|
||||
"${KIBANA_URL}/api/kibana/settings" \
|
||||
-d '{
|
||||
"changes": {
|
||||
"defaultIndex": "filebeat-*"
|
||||
}
|
||||
}' > /dev/null 2>&1
|
||||
|
||||
# Create a simple saved search for Docker logs
|
||||
echo "Creating saved searches..."
|
||||
docker exec ${CONTAINER_NAME}_kibana curl -s -X POST \
|
||||
-u "$AUTH" \
|
||||
-H "Content-Type: application/json" \
|
||||
-H "kbn-xsrf: true" \
|
||||
"${KIBANA_URL}/api/saved_objects/search" \
|
||||
-d '{
|
||||
"attributes": {
|
||||
"title": "Docker Container Logs",
|
||||
"description": "View all Docker container logs",
|
||||
"columns": ["container_name", "message"],
|
||||
"sort": ["@timestamp", "desc"],
|
||||
"kibanaSavedObjectMeta": {
|
||||
"searchSourceJSON": "{\"index\":\"filebeat-*\",\"query\":{\"match_all\":{}},\"filter\":[{\"meta\":{\"index\":\"filebeat-*\",\"negate\":false,\"disabled\":false,\"alias\":null,\"type\":\"exists\",\"key\":\"container_name\",\"value\":\"exists\"},\"exists\":{\"field\":\"container_name\"}}]}"
|
||||
}
|
||||
}
|
||||
}' > /dev/null 2>&1
|
||||
|
||||
echo ""
|
||||
echo "========================================="
|
||||
echo "Kibana Setup Complete!"
|
||||
echo "========================================="
|
||||
echo ""
|
||||
echo "QUICK START GUIDE:"
|
||||
echo ""
|
||||
echo "1. Open Kibana: ${SERVER_PUBLICBASEURL:-http://$(hostname -I | awk '{print $1}'):${KIBANA_PORT}}"
|
||||
echo ""
|
||||
echo "2. Login with:"
|
||||
echo " Username: ${KIBANA_USERNAME:-elastic}"
|
||||
echo " Password: [your password]"
|
||||
echo ""
|
||||
echo "3. TO VIEW LOGS SIMPLY:"
|
||||
echo " a) Click 'Discover' in the left menu"
|
||||
echo " b) Time range is in top-right (set to 'Last 15 minutes' or 'Today')"
|
||||
echo " c) Your logs will appear below"
|
||||
echo ""
|
||||
echo "4. TO FILTER LOGS:"
|
||||
echo " - By container: Click '+' next to any 'container_name' value"
|
||||
echo " - By host: Click '+' next to any 'host.name' value"
|
||||
echo " - Search box: Type keywords to search all logs"
|
||||
echo ""
|
||||
echo "5. TO VIEW LIVE LOGS:"
|
||||
echo " - Click the 'Refresh' button in top-right"
|
||||
echo " - Set it to refresh every 5 seconds"
|
||||
echo ""
|
||||
echo "========================================="
|
||||
@@ -2,16 +2,5 @@
|
||||
source "${AGENT_PATH}/common.sh"
|
||||
_check_required_env_vars "CONTAINER_NAME"
|
||||
|
||||
echo "Starting ELK stack..."
|
||||
docker compose up -d || _die "Failed to start ELK stack"
|
||||
|
||||
# Wait for services to be ready
|
||||
echo "Waiting for services to start..."
|
||||
sleep 5
|
||||
|
||||
# Check if services are running
|
||||
if docker compose ps | grep -q "Up"; then
|
||||
echo "ELK stack started successfully"
|
||||
else
|
||||
_die "Failed to start ELK stack services"
|
||||
fi
|
||||
docker compose up -d || _die "Failed to start"
|
||||
echo "Log Server started"
|
||||
@@ -2,38 +2,11 @@
|
||||
source "${AGENT_PATH}/common.sh"
|
||||
_check_required_env_vars "CONTAINER_NAME"
|
||||
|
||||
# For verbose mode (when called directly, not from install script)
|
||||
if [ "$1" = "-v" ] || [ "$1" = "--verbose" ]; then
|
||||
VERBOSE=true
|
||||
else
|
||||
VERBOSE=false
|
||||
fi
|
||||
|
||||
# Check if containers exist
|
||||
if ! docker ps -a --format "{{.Names}}" | grep -q "^${CONTAINER_NAME}_"; then
|
||||
echo "Unknown"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
# Check individual service status using docker directly (more reliable)
|
||||
es_running=$(docker ps --format "{{.Names}}" | grep -c "^${CONTAINER_NAME}_elasticsearch$" || echo 0)
|
||||
ls_running=$(docker ps --format "{{.Names}}" | grep -c "^${CONTAINER_NAME}_logstash$" || echo 0)
|
||||
kb_running=$(docker ps --format "{{.Names}}" | grep -c "^${CONTAINER_NAME}_kibana$" || echo 0)
|
||||
|
||||
if [ "$VERBOSE" = "true" ]; then
|
||||
echo "Elasticsearch: $([ $es_running -eq 1 ] && echo "Running" || echo "Not running")"
|
||||
echo "Logstash: $([ $ls_running -eq 1 ] && echo "Running" || echo "Not running")"
|
||||
echo "Kibana: $([ $kb_running -eq 1 ] && echo "Running" || echo "Not running")"
|
||||
fi
|
||||
|
||||
# Return overall status
|
||||
if [ "$es_running" -eq 1 ] && [ "$ls_running" -eq 1 ] && [ "$kb_running" -eq 1 ]; then
|
||||
if docker ps | grep -q "${CONTAINER_NAME}_grafana"; then
|
||||
echo "Running"
|
||||
elif [ "$es_running" -eq 0 ] && [ "$ls_running" -eq 0 ] && [ "$kb_running" -eq 0 ]; then
|
||||
echo "Stopped"
|
||||
echo " Web UI: http://$(hostname -I | awk '{print $1}'):${WEB_PORT:-3000}"
|
||||
echo " Loki API: http://$(hostname -I | awk '{print $1}'):${LOKI_PORT:-3100}"
|
||||
docker ps --format "table {{.Names}}\t{{.Status}}" | grep "${CONTAINER_NAME}_"
|
||||
else
|
||||
# At least one service is having issues
|
||||
if [ "$VERBOSE" = "false" ]; then
|
||||
echo "Starting" # More accurate than "Error" during startup
|
||||
fi
|
||||
echo "Stopped"
|
||||
fi
|
||||
@@ -2,7 +2,5 @@
|
||||
source "${AGENT_PATH}/common.sh"
|
||||
_check_required_env_vars "CONTAINER_NAME"
|
||||
|
||||
echo "Stopping ELK stack..."
|
||||
docker compose stop || true
|
||||
|
||||
echo "ELK stack stopped"
|
||||
docker compose stop || _die "Failed to stop"
|
||||
echo "Log Server stopped"
|
||||
@@ -1,54 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# Test script to verify API key authentication is working
|
||||
# This sends test log messages with and without API keys to verify authentication
|
||||
|
||||
source "${AGENT_PATH}/common.sh"
|
||||
_check_required_env_vars "CONTAINER_NAME" "LOGSTASH_BEATS_PORT"
|
||||
|
||||
echo "=== Testing Logstash API Key Authentication ==="
|
||||
echo ""
|
||||
|
||||
# Check if service is running
|
||||
if ! bash ./status.sh | grep -q "Running"; then
|
||||
echo "Error: LogServer is not running. Please start it first."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Get server IP
|
||||
SERVER_IP=$(hostname -I | awk '{print $1}')
|
||||
|
||||
echo "Testing authentication on ${SERVER_IP}:${LOGSTASH_BEATS_PORT}"
|
||||
echo ""
|
||||
|
||||
# Test 1: Send without API key (should be tagged or dropped based on config)
|
||||
echo "Test 1: Sending log without API key..."
|
||||
echo '{"message":"Test log without API key","timestamp":"'$(date -Iseconds)'"}' | \
|
||||
nc -w 2 ${SERVER_IP} ${LOGSTASH_BEATS_PORT} 2>/dev/null || true
|
||||
|
||||
# Test 2: Send with invalid API key (should be dropped)
|
||||
echo "Test 2: Sending log with invalid API key..."
|
||||
echo '{"message":"Test log with invalid key","fields":{"api_key":"invalid-key-12345"},"timestamp":"'$(date -Iseconds)'"}' | \
|
||||
nc -w 2 ${SERVER_IP} ${LOGSTASH_BEATS_PORT} 2>/dev/null || true
|
||||
|
||||
# Test 3: Send with valid API key (if one exists)
|
||||
if [ -f "${CONFIG_PATH}/api-keys.yml" ]; then
|
||||
# Extract first API key from the file
|
||||
API_KEY=$(grep -E "^ [^:]+: " "${CONFIG_PATH}/api-keys.yml" | head -1 | awk '{print $2}')
|
||||
if [ -n "$API_KEY" ]; then
|
||||
echo "Test 3: Sending log with valid API key..."
|
||||
echo '{"message":"Test log with valid key","fields":{"api_key":"'${API_KEY}'"},"timestamp":"'$(date -Iseconds)'"}' | \
|
||||
nc -w 2 ${SERVER_IP} ${LOGSTASH_BEATS_PORT} 2>/dev/null || true
|
||||
else
|
||||
echo "Test 3: Skipped - no API keys found"
|
||||
fi
|
||||
else
|
||||
echo "Test 3: Skipped - api-keys.yml not found"
|
||||
fi
|
||||
|
||||
echo ""
|
||||
echo "Tests sent. Check Logstash logs to verify authentication:"
|
||||
echo " docker logs ${CONTAINER_NAME}_logstash --tail 20"
|
||||
echo ""
|
||||
echo "Valid logs should appear in Elasticsearch."
|
||||
echo "Invalid logs should be dropped."
|
||||
@@ -2,15 +2,5 @@
|
||||
source "${AGENT_PATH}/common.sh"
|
||||
_check_required_env_vars "CONTAINER_NAME"
|
||||
|
||||
# Stop the containers
|
||||
bash ./stop.sh || _die "Failed to stop containers"
|
||||
|
||||
# Remove the containers
|
||||
docker compose down --remove-orphans || _die "Failed to remove containers"
|
||||
|
||||
# CRITICAL: Never remove data volumes in uninstall.sh!
|
||||
# Data volumes must be preserved for potential reinstallation
|
||||
# Only destroy.sh should remove volumes, and it must be explicit
|
||||
|
||||
echo "Uninstallation of ${CONTAINER_NAME} complete"
|
||||
echo "Note: Data volumes have been preserved. To remove all data, use destroy.sh"
|
||||
docker compose down || true
|
||||
echo "Log Server uninstalled (data preserved)"
|
||||
Reference in New Issue
Block a user