448 lines
20 KiB
Markdown
448 lines
20 KiB
Markdown
# Architecture Patterns: Raspberry Pi-Based Web Monitoring Systems
|
|
|
|
**Project:** TCKRTUIYO - Rainfall Station RTU Web Interface
|
|
**Researched:** 2026-03-12
|
|
**Confidence:** HIGH
|
|
|
|
## Executive Summary
|
|
|
|
Raspberry Pi-based web monitoring systems typically follow a **three-tier architecture**:
|
|
1. **Frontend** — Web UI served via HTTP, displaying data with real-time updates
|
|
2. **Backend API** — Flask/FastAPI handling sensor data collection and serving JSON endpoints
|
|
3. **Data Layer** — Sensor interfaces, CSV logging, and network transmission
|
|
|
|
For this rainfall monitoring RTU, the architecture must accommodate:
|
|
- **Dual interfaces**: Kiosk mode (1024x600, port 8080) vs Remote access (Full HD, port 9090)
|
|
- **Low-powered hardware**: Pi Zero 2 W with ~512MB RAM, single-core equivalent
|
|
- **CSV-based data workflow**: Local storage → server transmission via FTP/SCP/SFTP/WebDAV
|
|
|
|
---
|
|
|
|
## Recommended Architecture
|
|
|
|
```
|
|
┌─────────────────────────────────────────────────────────────────────────────┐
|
|
│ RAINFALL MONITORING RTU │
|
|
├─────────────────────────────────────────────────────────────────────────────┤
|
|
│ │
|
|
│ ┌──────────────────────────────────────────────────────────────────────┐ │
|
|
│ │ WEB LAYER (Dual Port) │ │
|
|
│ │ ┌─────────────────────────┐ ┌─────────────────────────────┐ │ │
|
|
│ │ │ KIOSK INTERFACE │ │ REMOTE INTERFACE │ │ │
|
|
│ │ │ http://pi:8080 │ │ http://pi:9090 │ │ │
|
|
│ │ │ 1024x600 fixed │ │ Full HD responsive │ │ │
|
|
│ │ │ Chromium Kiosk Mode │ │ Auth required │ │ │
|
|
│ │ └───────────┬─────────────┘ └──────────────┬──────────────┘ │ │
|
|
│ │ │ │ │ │
|
|
│ │ └────────────┬───────────────────┘ │ │
|
|
│ │ │ │ │
|
|
│ │ ┌──────▼──────┐ │ │
|
|
│ │ │ STATIC │ │ │
|
|
│ │ │ FILES │ │ │
|
|
│ │ │ (HTML/CSS/ │ │ │
|
|
│ │ │ JS/VUE) │ │ │
|
|
│ │ └──────┬──────┘ │ │
|
|
│ └───────────────────────────│──────────────────────────────────────────┘ │
|
|
│ │ │
|
|
│ ┌───────────────────────────▼──────────────────────────────────────────┐ │
|
|
│ │ API LAYER (Flask/FastAPI) │ │
|
|
│ │ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │ │
|
|
│ │ │ /api/ │ │ /api/ │ │ /api/ │ │ /api/ │ │ │
|
|
│ │ │ status │ │ sensors │ │ settings │ │ files │ │ │
|
|
│ │ │ (GET) │ │ (GET) │ │ (GET/POST) │ │ (GET/POST) │ │ │
|
|
│ │ └──────┬──────┘ └──────┬──────┘ └──────┬──────┘ └──────┬──────┘ │ │
|
|
│ │ │ │ │ │ │ │
|
|
│ │ └────────────────┴────────┬────────┴─────────────────┘ │ │
|
|
│ │ │ │ │
|
|
│ │ ┌────────▼────────┐ │ │
|
|
│ │ │ BLUEPRINT │ │ │
|
|
│ │ │ ROUTING │ │ │
|
|
│ │ └────────┬────────┘ │ │
|
|
│ └───────────────────────────────│───────────────────────────────────────┘ │
|
|
│ │ │
|
|
│ ┌───────────────────────────────▼───────────────────────────────────────┐ │
|
|
│ │ SERVICE LAYER │ │
|
|
│ │ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │ │
|
|
│ │ │ SENSOR │ │ DATA │ │ NETWORK │ │ CONFIG │ │ │
|
|
│ │ │ SERVICE │ │ LOGGER │ │ SERVICE │ │ SERVICE │ │ │
|
|
│ │ │ (read │ │ (CSV │ │ (FTP/ │ │ (read/ │ │ │
|
|
│ │ │ GPIO/ │ │ write) │ │ SFTP) │ │ write) │ │ │
|
|
│ │ │ ADC) │ │ │ │ │ │ │ │ │
|
|
│ │ └─────────────┘ └─────────────┘ └─────────────┘ └─────────────┘ │ │
|
|
│ │ │ │ │ │ │ │
|
|
│ └─────────┼────────────────┼─────────────────┼─────────────────┼──────────┘ │
|
|
│ │ │ │ │ │
|
|
│ ──────────┼────────────────┼─────────────────┼─────────────────┼──────────────│
|
|
│ │ │ │ │ │
|
|
│ ┌─────────▼────────────────▼─────────────────▼─────────────────▼──────────┐ │
|
|
│ │ DATA LAYER │ │
|
|
│ │ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │ │
|
|
│ │ │ HARDWARE │ │ CSV │ │ NETWORK │ │ CONFIG │ │ │
|
|
│ │ │ INTERFACE │ │ FILES │ │ INTERFACE │ │ FILES │ │ │
|
|
│ │ │ (GPIO/ │ │ (local │ │ (modem/ │ │ (JSON/ │ │ │
|
|
│ │ │ ADC/ │ │ storage) │ │ ethernet) │ │ YAML) │ │ │
|
|
│ │ │ Serial) │ │ │ │ │ │ │ │ │
|
|
│ │ └─────────────┘ └─────────────┘ └─────────────┘ └─────────────┘ │ │
|
|
│ └──────────────────────────────────────────────────────────────────────────┘ │
|
|
│ │
|
|
└─────────────────────────────────────────────────────────────────────────────┘
|
|
```
|
|
|
|
---
|
|
|
|
## Component Boundaries
|
|
|
|
### 1. Web Layer (HTTP Servers)
|
|
|
|
| Component | Responsibility | Boundary |
|
|
|-----------|---------------|----------|
|
|
| **Kiosk Server** (port 8080) | Serves fixed 1024x600 UI for touchscreen | Reads from API layer only |
|
|
| **Remote Server** (port 9090) | Serves responsive Full HD UI with authentication | Reads/Writes via API layer |
|
|
|
|
**Communication:**
|
|
- Both serve static files (HTML/CSS/JavaScript)
|
|
- Both make AJAX/Fetch requests to API layer
|
|
- Single API backend shared by both ports
|
|
|
|
### 2. API Layer (Flask/FastAPI)
|
|
|
|
| Endpoint | Method | Purpose | Writes To |
|
|
|----------|--------|---------|-----------|
|
|
| `/api/status` | GET | Station ID, time, comm status, version | None |
|
|
| `/api/sensors` | GET | Rainfall (today/hourly/MAR/yearly), solar, battery | None |
|
|
| `/api/settings` | GET/POST | All configuration (ADC, network, calibration) | Config files |
|
|
| `/api/files` | GET/POST/DELETE | CSV file management | CSV storage |
|
|
| `/api/calibration` | GET | Live sensor readings | None |
|
|
|
|
**Communication:**
|
|
- JSON responses to Web Layer
|
|
- Delegates to Service Layer for operations
|
|
|
|
### 3. Service Layer
|
|
|
|
| Service | Responsibility | Communicates With |
|
|
|---------|---------------|-------------------|
|
|
| **SensorService** | Polls GPIO/ADC/Serial for rainfall, water level, voltage | Hardware Interface |
|
|
| **DataLogger** | Writes sensor readings to CSV with timestamps | CSV Files |
|
|
| **NetworkService** | Transmits CSV via FTP/SFTP/SCP/WebDAV | Remote Server |
|
|
| **ConfigService** | Reads/writes JSON configuration files | Config Files |
|
|
|
|
### 4. Data Layer
|
|
|
|
| Component | Location | Purpose |
|
|
|-----------|----------|---------|
|
|
| **Hardware Interface** | GPIO pins, ADC (MCP3008/3208), Serial | Physical sensor connections |
|
|
| **CSV Storage** | `/data/*.csv` | Time-series rainfall data |
|
|
| **Network Interface** | Ethernet/USB Modem | Server communication |
|
|
| **Config Storage** | `/config/*.json` | Station settings |
|
|
|
|
---
|
|
|
|
## Data Flow
|
|
|
|
### Primary Flow: Sensor Reading → Display
|
|
|
|
```
|
|
1. TIMER (every N seconds)
|
|
│
|
|
▼
|
|
2. SENSOR SERVICE reads from hardware
|
|
- Rainfall tips (GPIO interrupt)
|
|
- Water level (ADC via SPI)
|
|
- Solar/Battery voltage (ADC)
|
|
│
|
|
▼
|
|
3. UPDATE IN-MEMORY STATE
|
|
- today_rainfall, hourly_rainfall, etc.
|
|
- battery_voltage, solar_voltage
|
|
│
|
|
▼
|
|
4. DATA LOGGER writes to CSV
|
|
- Append timestamp + readings
|
|
│
|
|
▼
|
|
5. API LAYER receives GET /api/sensors
|
|
- Returns JSON with current state
|
|
│
|
|
▼
|
|
6. WEB LAYER displays data
|
|
- Polling every 1-5 seconds (kiosk)
|
|
- Socket.IO push (if used)
|
|
```
|
|
|
|
### Secondary Flow: Settings Change
|
|
|
|
```
|
|
1. USER taps setting on touchscreen
|
|
2. WEB LAYER sends POST /api/settings
|
|
3. API LAYER validates payload
|
|
4. CONFIG SERVICE writes to JSON file
|
|
5. SENSOR SERVICE reloads config
|
|
6. ACK response to UI
|
|
```
|
|
|
|
### Tertiary Flow: CSV Transmission
|
|
|
|
```
|
|
1. SCHEDULED TASK (cron or timer)
|
|
│
|
|
▼
|
|
2. NETWORK SERVICE checks queue
|
|
│
|
|
▼
|
|
3. READ pending CSV files
|
|
│
|
|
▼
|
|
4. TRANSMIT via selected protocol
|
|
- FTP / SFTP / SCP / WebDAV
|
|
│
|
|
▼
|
|
5. LOG transmission result
|
|
- Success: mark as sent
|
|
- Failure: retry queue
|
|
```
|
|
|
|
---
|
|
|
|
## Build Order & Dependencies
|
|
|
|
Based on research of similar systems and component dependencies:
|
|
|
|
### Phase 1: Foundation (Weeks 1-2)
|
|
**Goal:** Get basic sensor reading and display working
|
|
|
|
| Order | Component | Depends On | Rationale |
|
|
|-------|-----------|------------|-----------|
|
|
| 1 | Hardware Interface | None | Must read sensors before anything else |
|
|
| 2 | Sensor Service | Hardware Interface | Abstracts reading logic |
|
|
| 3 | Basic API (status, sensors) | Sensor Service | Data must exist to serve |
|
|
| 4 | Minimal Web UI (dashboard) | API | Shows readings on screen |
|
|
|
|
**Why:** This follows the "vertical slice" approach — each layer works end-to-end before adding features.
|
|
|
|
### Phase 2: Local Interface (Weeks 3-4)
|
|
**Goal:** Complete kiosk-mode interface
|
|
|
|
| Order | Component | Depends On |
|
|
|-------|-----------|------------|
|
|
| 5 | Settings UI | Phase 1 API |
|
|
| 6 | Calibration UI | Phase 1 API |
|
|
| 7 | File Management UI | Phase 1 API |
|
|
| 8 | Port 8080 Kiosk Server | All above |
|
|
|
|
### Phase 3: Data Persistence (Weeks 5-6)
|
|
**Goal:** CSV logging and transmission
|
|
|
|
| Order | Component | Depends On |
|
|
|-------|-----------|------------|
|
|
| 9 | Data Logger Service | Sensor Service |
|
|
| 10 | CSV Storage Layer | Data Logger |
|
|
| 11 | Network Transmission Service | CSV Storage |
|
|
|
|
### Phase 4: Remote Access (Weeks 7-8)
|
|
**Goal:** Full HD remote interface
|
|
|
|
| Order | Component | Depends On |
|
|
|-------|-----------|------------|
|
|
| 12 | Authentication Layer | Phase 2 UI |
|
|
| 13 | Responsive Templates | Phase 2 UI |
|
|
| 14 | Port 9090 Remote Server | All above |
|
|
|
|
---
|
|
|
|
## Architecture Patterns to Follow
|
|
|
|
### Pattern 1: Service-Oriented Architecture (SOA)
|
|
|
|
Each service (Sensor, Network, Config) is a Python class/module with clear interface:
|
|
|
|
```python
|
|
class SensorService:
|
|
def read_all(self) -> SensorReading:
|
|
"""Read all sensors and return unified reading"""
|
|
|
|
def read_rainfall(self) -> float:
|
|
"""Read rainfall counter"""
|
|
|
|
def read_voltage(self, channel: int) -> float:
|
|
"""Read ADC channel voltage"""
|
|
```
|
|
|
|
**Why:** Isolates hardware access, easy to mock for testing, clear boundaries.
|
|
|
|
### Pattern 2: Blueprint-Based Routing
|
|
|
|
Flask blueprints separate concerns:
|
|
|
|
```python
|
|
# routes/status.py
|
|
status_bp = Blueprint('status', __name__)
|
|
@status_bp.route('/api/status')
|
|
def get_status(): ...
|
|
|
|
# routes/sensors.py
|
|
sensors_bp = Blueprint('sensors', __name__)
|
|
@sensors_bp.route('/api/sensors')
|
|
def get_sensors(): ...
|
|
|
|
# routes/settings.py
|
|
settings_bp = Blueprint('settings', __name__)
|
|
settings_bp.route('/api/settings', methods=['GET', 'POST'])
|
|
```
|
|
|
|
**Why:** Clean separation, easier maintenance, modular.
|
|
|
|
### Pattern 3: Configuration-Driven Design
|
|
|
|
Store all settings in JSON files, not hardcoded:
|
|
|
|
```json
|
|
{
|
|
"station": {
|
|
"id": "D007",
|
|
"name": "Station A"
|
|
},
|
|
"network": {
|
|
"protocol": "FTP",
|
|
"server": "myvscada.example.com",
|
|
"port": 21
|
|
},
|
|
"sensors": {
|
|
"rainfall": {
|
|
"id": "123456RF",
|
|
"danger_mm": 30.0,
|
|
"danger_window_min": 60
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
**Why:** Allows runtime changes without code modification, matches existing RTU behavior.
|
|
|
|
### Pattern 4: Polling with Fallback
|
|
|
|
For real-time updates on Pi Zero 2 W:
|
|
|
|
```javascript
|
|
// Primary: Polling (simple, reliable)
|
|
setInterval(fetchSensorData, 2000); // Every 2 seconds
|
|
|
|
// Fallback: Only add Socket.IO if latency becomes issue
|
|
// Don't optimize until measured
|
|
```
|
|
|
|
**Why:** Socket.IO adds overhead. On Pi Zero 2 W with limited resources, polling every 1-2 seconds is acceptable and simpler to debug.
|
|
|
|
---
|
|
|
|
## Anti-Patterns to Avoid
|
|
|
|
### Anti-Pattern 1: Monolithic API Handler
|
|
|
|
**Bad:**
|
|
```python
|
|
@app.route('/api/<any>')
|
|
def handle_all():
|
|
# 500 lines of if/else for different endpoints
|
|
```
|
|
|
|
**Why:** Unmaintainable, hard to test, violates single responsibility.
|
|
|
|
### Anti-Pattern 2: Direct Hardware Calls in Routes
|
|
|
|
**Bad:**
|
|
```python
|
|
@app.route('/api/sensors')
|
|
def get_sensors():
|
|
# Direct GPIO/ADC calls inline
|
|
rainfall = read_rainfall_gpio() # Bad!
|
|
```
|
|
|
|
**Why:** Hard to test, mixes HTTP handling with hardware access.
|
|
|
|
### Anti-Pattern 3: Storing State in Global Variables
|
|
|
|
**Bad:**
|
|
```python
|
|
current_rainfall = 0 # Global state!
|
|
|
|
@app.route('/api/sensors')
|
|
def get_sensors():
|
|
return {'rainfall': current_rainfall}
|
|
```
|
|
|
|
**Why:** Race conditions, hard to reason about state, difficult to persist across restarts.
|
|
|
|
---
|
|
|
|
## Scalability Considerations
|
|
|
|
| Concern | At Deployment (1 station) | At 10 stations | At 100 stations |
|
|
|---------|-------------------------|----------------|-----------------|
|
|
| **Data Storage** | CSV files, ~10MB/year | Same per station | Consider SQLite per station |
|
|
| **Network** | Single FTP push | Same | Batch uploads, queue system |
|
|
| **UI Performance** | Polling 2s acceptable | N/A | N/A |
|
|
| **CPU Load** | ~10-15% idle | Same | Same |
|
|
|
|
**Key insight:** This is an edge device, not a server. Architecture focuses on **reliability** and **low resource usage** rather than horizontal scaling.
|
|
|
|
---
|
|
|
|
## Dual Interface Implementation
|
|
|
|
### Shared API, Separate Static Roots
|
|
|
|
```
|
|
app.py
|
|
├── /static_kiosk/ (1024x600 optimized)
|
|
│ ├── css/
|
|
│ ├── js/
|
|
│ └── index.html
|
|
├── /static_remote/ (Full HD responsive)
|
|
│ ├── css/
|
|
│ ├── js/
|
|
│ └── index.html
|
|
```
|
|
|
|
### Route Configuration
|
|
|
|
```python
|
|
# Kiosk mode - no auth, fixed resolution
|
|
@app.route('/', defaults={'path': ''}, subdomain='kiosk')
|
|
@app.route('/<path>', defaults={'path': ''}, subdomain='kiosk')
|
|
def serve_kiosk(path):
|
|
return send_from_directory('static_kiosk', 'index.html')
|
|
|
|
# Remote mode - auth required, responsive
|
|
@app.route('/')
|
|
@app.route('/<path>')
|
|
@login_required
|
|
def serve_remote(path):
|
|
return send_from_directory('static_remote', 'index.html')
|
|
```
|
|
|
|
**Alternative (simpler):** Single static folder with CSS media queries adapting to screen size, two separate ports.
|
|
|
|
---
|
|
|
|
## Confidence Assessment
|
|
|
|
| Area | Level | Reason |
|
|
|------|-------|--------|
|
|
| **Architecture Pattern** | HIGH | Matches proven IoT/SCADA patterns from research |
|
|
| **Component Boundaries** | HIGH | Clear separation matches Flask best practices |
|
|
| **Data Flow** | HIGH | Follows standard sensor→API→UI pattern |
|
|
| **Build Order** | MEDIUM | Suggested based on typical patterns; verify during implementation |
|
|
| **Pi Zero 2 W Constraints** | HIGH | Performance considerations documented |
|
|
|
|
---
|
|
|
|
## Sources
|
|
|
|
- Szymon Pulut, "The Architecture Behind the IoT Dashboard" (2024) — IoT dashboard architecture with Raspberry Pi
|
|
- GitHub g1forfun/pi-monitor — Flask + Socket.IO real-time monitoring example
|
|
- Random Nerd Tutorials — Raspberry Pi sensor data logging patterns
|
|
- ResearchGate — RTU modular architecture for industrial monitoring
|
|
- Sensors and Materials (2025) — Django + Raspberry Pi web monitoring
|
|
- Raspberry Pi Forums — Pi Zero 2 W performance discussions
|