admin/rtu_v5
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: complete project research

Browse Source
This commit is contained in:
TCK e-Solutions Sdn. Bhd.
2026-03-12 06:04:19 +08:00
parent d1ef33c25a
commit 1e12303920
5 changed files with 1280 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

447
.planning/research/ARCHITECTURE.md Normal file
View File

@@ -0,0 +1,447 @@
# Architecture Patterns: Raspberry Pi-Based Web Monitoring Systems
**Project:** TCKRTUIYO - Rainfall Station RTU Web Interface
**Researched:** 2026-03-12
**Confidence:** HIGH
## Executive Summary
Raspberry Pi-based web monitoring systems typically follow a **three-tier architecture**:
1. **Frontend** — Web UI served via HTTP, displaying data with real-time updates
2. **Backend API** — Flask/FastAPI handling sensor data collection and serving JSON endpoints
3. **Data Layer** — Sensor interfaces, CSV logging, and network transmission
For this rainfall monitoring RTU, the architecture must accommodate:
- **Dual interfaces**: Kiosk mode (1024x600, port 8080) vs Remote access (Full HD, port 9090)
- **Low-powered hardware**: Pi Zero 2 W with ~512MB RAM, single-core equivalent
- **CSV-based data workflow**: Local storage → server transmission via FTP/SCP/SFTP/WebDAV
---
## Recommended Architecture
```
┌─────────────────────────────────────────────────────────────────────────────┐
│ RAINFALL MONITORING RTU │
├─────────────────────────────────────────────────────────────────────────────┤
│ │
│ ┌──────────────────────────────────────────────────────────────────────┐ │
│ │ WEB LAYER (Dual Port) │ │
│ │ ┌─────────────────────────┐ ┌─────────────────────────────┐ │ │
│ │ │ KIOSK INTERFACE │ │ REMOTE INTERFACE │ │ │
│ │ │ http://pi:8080 │ │ http://pi:9090 │ │ │
│ │ │ 1024x600 fixed │ │ Full HD responsive │ │ │
│ │ │ Chromium Kiosk Mode │ │ Auth required │ │ │
│ │ └───────────┬─────────────┘ └──────────────┬──────────────┘ │ │
│ │ │ │ │ │
│ │ └────────────┬───────────────────┘ │ │
│ │ │ │ │
│ │ ┌──────▼──────┐ │ │
│ │ │ STATIC │ │ │
│ │ │ FILES │ │ │
│ │ │ (HTML/CSS/ │ │ │
│ │ │ JS/VUE) │ │ │
│ │ └──────┬──────┘ │ │
│ └───────────────────────────│──────────────────────────────────────────┘ │
│ │ │
│ ┌───────────────────────────▼──────────────────────────────────────────┐ │
│ │ API LAYER (Flask/FastAPI) │ │
│ │ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │ │
│ │ │ /api/ │ │ /api/ │ │ /api/ │ │ /api/ │ │ │
│ │ │ status │ │ sensors │ │ settings │ │ files │ │ │
│ │ │ (GET) │ │ (GET) │ │ (GET/POST) │ │ (GET/POST) │ │ │
│ │ └──────┬──────┘ └──────┬──────┘ └──────┬──────┘ └──────┬──────┘ │ │
│ │ │ │ │ │ │ │
│ │ └────────────────┴────────┬────────┴─────────────────┘ │ │
│ │ │ │ │
│ │ ┌────────▼────────┐ │ │
│ │ │ BLUEPRINT │ │ │
│ │ │ ROUTING │ │ │
│ │ └────────┬────────┘ │ │
│ └───────────────────────────────│───────────────────────────────────────┘ │
│ │ │
│ ┌───────────────────────────────▼───────────────────────────────────────┐ │
│ │ SERVICE LAYER │ │
│ │ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │ │
│ │ │ SENSOR │ │ DATA │ │ NETWORK │ │ CONFIG │ │ │
│ │ │ SERVICE │ │ LOGGER │ │ SERVICE │ │ SERVICE │ │ │
│ │ │ (read │ │ (CSV │ │ (FTP/ │ │ (read/ │ │ │
│ │ │ GPIO/ │ │ write) │ │ SFTP) │ │ write) │ │ │
│ │ │ ADC) │ │ │ │ │ │ │ │ │
│ │ └─────────────┘ └─────────────┘ └─────────────┘ └─────────────┘ │ │
│ │ │ │ │ │ │ │
│ └─────────┼────────────────┼─────────────────┼─────────────────┼──────────┘ │
│ │ │ │ │ │
│ ──────────┼────────────────┼─────────────────┼─────────────────┼──────────────│
│ │ │ │ │ │
│ ┌─────────▼────────────────▼─────────────────▼─────────────────▼──────────┐ │
│ │ DATA LAYER │ │
│ │ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │ │
│ │ │ HARDWARE │ │ CSV │ │ NETWORK │ │ CONFIG │ │ │
│ │ │ INTERFACE │ │ FILES │ │ INTERFACE │ │ FILES │ │ │
│ │ │ (GPIO/ │ │ (local │ │ (modem/ │ │ (JSON/ │ │ │
│ │ │ ADC/ │ │ storage) │ │ ethernet) │ │ YAML) │ │ │
│ │ │ Serial) │ │ │ │ │ │ │ │ │
│ │ └─────────────┘ └─────────────┘ └─────────────┘ └─────────────┘ │ │
│ └──────────────────────────────────────────────────────────────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────────────────┘
```
---
## Component Boundaries
### 1. Web Layer (HTTP Servers)
| Component | Responsibility | Boundary |
|-----------|---------------|----------|
| **Kiosk Server** (port 8080) | Serves fixed 1024x600 UI for touchscreen | Reads from API layer only |
| **Remote Server** (port 9090) | Serves responsive Full HD UI with authentication | Reads/Writes via API layer |
**Communication:**
- Both serve static files (HTML/CSS/JavaScript)
- Both make AJAX/Fetch requests to API layer
- Single API backend shared by both ports
### 2. API Layer (Flask/FastAPI)
| Endpoint | Method | Purpose | Writes To |
|----------|--------|---------|-----------|
| `/api/status` | GET | Station ID, time, comm status, version | None |
| `/api/sensors` | GET | Rainfall (today/hourly/MAR/yearly), solar, battery | None |
| `/api/settings` | GET/POST | All configuration (ADC, network, calibration) | Config files |
| `/api/files` | GET/POST/DELETE | CSV file management | CSV storage |
| `/api/calibration` | GET | Live sensor readings | None |
**Communication:**
- JSON responses to Web Layer
- Delegates to Service Layer for operations
### 3. Service Layer
| Service | Responsibility | Communicates With |
|---------|---------------|-------------------|
| **SensorService** | Polls GPIO/ADC/Serial for rainfall, water level, voltage | Hardware Interface |
| **DataLogger** | Writes sensor readings to CSV with timestamps | CSV Files |
| **NetworkService** | Transmits CSV via FTP/SFTP/SCP/WebDAV | Remote Server |
| **ConfigService** | Reads/writes JSON configuration files | Config Files |
### 4. Data Layer
| Component | Location | Purpose |
|-----------|----------|---------|
| **Hardware Interface** | GPIO pins, ADC (MCP3008/3208), Serial | Physical sensor connections |
| **CSV Storage** | `/data/*.csv` | Time-series rainfall data |
| **Network Interface** | Ethernet/USB Modem | Server communication |
| **Config Storage** | `/config/*.json` | Station settings |
---
## Data Flow
### Primary Flow: Sensor Reading → Display
```
1. TIMER (every N seconds)
│
▼
2. SENSOR SERVICE reads from hardware
- Rainfall tips (GPIO interrupt)
- Water level (ADC via SPI)
- Solar/Battery voltage (ADC)
│
▼
3. UPDATE IN-MEMORY STATE
- today_rainfall, hourly_rainfall, etc.
- battery_voltage, solar_voltage
│
▼
4. DATA LOGGER writes to CSV
- Append timestamp + readings
│
▼
5. API LAYER receives GET /api/sensors
- Returns JSON with current state
│
▼
6. WEB LAYER displays data
- Polling every 1-5 seconds (kiosk)
- Socket.IO push (if used)
```
### Secondary Flow: Settings Change
```
1. USER taps setting on touchscreen
2. WEB LAYER sends POST /api/settings
3. API LAYER validates payload
4. CONFIG SERVICE writes to JSON file
5. SENSOR SERVICE reloads config
6. ACK response to UI
```
### Tertiary Flow: CSV Transmission
```
1. SCHEDULED TASK (cron or timer)
│
▼
2. NETWORK SERVICE checks queue
│
▼
3. READ pending CSV files
│
▼
4. TRANSMIT via selected protocol
- FTP / SFTP / SCP / WebDAV
│
▼
5. LOG transmission result
- Success: mark as sent
- Failure: retry queue
```
---
## Build Order & Dependencies
Based on research of similar systems and component dependencies:
### Phase 1: Foundation (Weeks 1-2)
**Goal:** Get basic sensor reading and display working
| Order | Component | Depends On | Rationale |
|-------|-----------|------------|-----------|
| 1 | Hardware Interface | None | Must read sensors before anything else |
| 2 | Sensor Service | Hardware Interface | Abstracts reading logic |
| 3 | Basic API (status, sensors) | Sensor Service | Data must exist to serve |
| 4 | Minimal Web UI (dashboard) | API | Shows readings on screen |
**Why:** This follows the "vertical slice" approach — each layer works end-to-end before adding features.
### Phase 2: Local Interface (Weeks 3-4)
**Goal:** Complete kiosk-mode interface
| Order | Component | Depends On |
|-------|-----------|------------|
| 5 | Settings UI | Phase 1 API |
| 6 | Calibration UI | Phase 1 API |
| 7 | File Management UI | Phase 1 API |
| 8 | Port 8080 Kiosk Server | All above |
### Phase 3: Data Persistence (Weeks 5-6)
**Goal:** CSV logging and transmission
| Order | Component | Depends On |
|-------|-----------|------------|
| 9 | Data Logger Service | Sensor Service |
| 10 | CSV Storage Layer | Data Logger |
| 11 | Network Transmission Service | CSV Storage |
### Phase 4: Remote Access (Weeks 7-8)
**Goal:** Full HD remote interface
| Order | Component | Depends On |
|-------|-----------|------------|
| 12 | Authentication Layer | Phase 2 UI |
| 13 | Responsive Templates | Phase 2 UI |
| 14 | Port 9090 Remote Server | All above |
---
## Architecture Patterns to Follow
### Pattern 1: Service-Oriented Architecture (SOA)
Each service (Sensor, Network, Config) is a Python class/module with clear interface:
```python
class SensorService:
def read_all(self) -> SensorReading:
"""Read all sensors and return unified reading"""
def read_rainfall(self) -> float:
"""Read rainfall counter"""
def read_voltage(self, channel: int) -> float:
"""Read ADC channel voltage"""
```
**Why:** Isolates hardware access, easy to mock for testing, clear boundaries.
### Pattern 2: Blueprint-Based Routing
Flask blueprints separate concerns:
```python
# routes/status.py
status_bp = Blueprint('status', __name__)
@status_bp.route('/api/status')
def get_status(): ...
# routes/sensors.py
sensors_bp = Blueprint('sensors', __name__)
@sensors_bp.route('/api/sensors')
def get_sensors(): ...
# routes/settings.py
settings_bp = Blueprint('settings', __name__)
settings_bp.route('/api/settings', methods=['GET', 'POST'])
```
**Why:** Clean separation, easier maintenance, modular.
### Pattern 3: Configuration-Driven Design
Store all settings in JSON files, not hardcoded:
```json
{
"station": {
"id": "D007",
"name": "Station A"
},
"network": {
"protocol": "FTP",
"server": "myvscada.example.com",
"port": 21
},
"sensors": {
"rainfall": {
"id": "123456RF",
"danger_mm": 30.0,
"danger_window_min": 60
}
}
}
```
**Why:** Allows runtime changes without code modification, matches existing RTU behavior.
### Pattern 4: Polling with Fallback
For real-time updates on Pi Zero 2 W:
```javascript
// Primary: Polling (simple, reliable)
setInterval(fetchSensorData, 2000); // Every 2 seconds
// Fallback: Only add Socket.IO if latency becomes issue
// Don't optimize until measured
```
**Why:** Socket.IO adds overhead. On Pi Zero 2 W with limited resources, polling every 1-2 seconds is acceptable and simpler to debug.
---
## Anti-Patterns to Avoid
### Anti-Pattern 1: Monolithic API Handler
**Bad:**
```python
@app.route('/api/<any>')
def handle_all():
# 500 lines of if/else for different endpoints
```
**Why:** Unmaintainable, hard to test, violates single responsibility.
### Anti-Pattern 2: Direct Hardware Calls in Routes
**Bad:**
```python
@app.route('/api/sensors')
def get_sensors():
# Direct GPIO/ADC calls inline
rainfall = read_rainfall_gpio() # Bad!
```
**Why:** Hard to test, mixes HTTP handling with hardware access.
### Anti-Pattern 3: Storing State in Global Variables
**Bad:**
```python
current_rainfall = 0 # Global state!
@app.route('/api/sensors')
def get_sensors():
return {'rainfall': current_rainfall}
```
**Why:** Race conditions, hard to reason about state, difficult to persist across restarts.
---
## Scalability Considerations
| Concern | At Deployment (1 station) | At 10 stations | At 100 stations |
|---------|-------------------------|----------------|-----------------|
| **Data Storage** | CSV files, ~10MB/year | Same per station | Consider SQLite per station |
| **Network** | Single FTP push | Same | Batch uploads, queue system |
| **UI Performance** | Polling 2s acceptable | N/A | N/A |
| **CPU Load** | ~10-15% idle | Same | Same |
**Key insight:** This is an edge device, not a server. Architecture focuses on **reliability** and **low resource usage** rather than horizontal scaling.
---
## Dual Interface Implementation
### Shared API, Separate Static Roots
```
app.py
├── /static_kiosk/ (1024x600 optimized)
│ ├── css/
│ ├── js/
│ └── index.html
├── /static_remote/ (Full HD responsive)
│ ├── css/
│ ├── js/
│ └── index.html
```
### Route Configuration
```python
# Kiosk mode - no auth, fixed resolution
@app.route('/', defaults={'path': ''}, subdomain='kiosk')
@app.route('/<path>', defaults={'path': ''}, subdomain='kiosk')
def serve_kiosk(path):
return send_from_directory('static_kiosk', 'index.html')
# Remote mode - auth required, responsive
@app.route('/')
@app.route('/<path>')
@login_required
def serve_remote(path):
return send_from_directory('static_remote', 'index.html')
```
**Alternative (simpler):** Single static folder with CSS media queries adapting to screen size, two separate ports.
---
## Confidence Assessment
| Area | Level | Reason |
|------|-------|--------|
| **Architecture Pattern** | HIGH | Matches proven IoT/SCADA patterns from research |
| **Component Boundaries** | HIGH | Clear separation matches Flask best practices |
| **Data Flow** | HIGH | Follows standard sensor→API→UI pattern |
| **Build Order** | MEDIUM | Suggested based on typical patterns; verify during implementation |
| **Pi Zero 2 W Constraints** | HIGH | Performance considerations documented |
---
## Sources
- Szymon Pulut, "The Architecture Behind the IoT Dashboard" (2024) — IoT dashboard architecture with Raspberry Pi
- GitHub g1forfun/pi-monitor — Flask + Socket.IO real-time monitoring example
- Random Nerd Tutorials — Raspberry Pi sensor data logging patterns
- ResearchGate — RTU modular architecture for industrial monitoring
- Sensors and Materials (2025) — Django + Raspberry Pi web monitoring
- Raspberry Pi Forums — Pi Zero 2 W performance discussions

175
.planning/research/FEATURES.md Normal file
View File

@@ -0,0 +1,175 @@
# Feature Research
**Domain:** Rainfall Monitoring Station RTU Web Interface
**Researched:** 2026-03-12
**Confidence:** HIGH
## Feature Landscape
### Table Stakes (Users Expect These)
Features users assume exist. Missing these = product feels incomplete.
| Feature | Why Expected | Complexity | Notes |
|---------|--------------|------------|-------|
| **Real-time Rainfall Display** | Core function — users must see current rainfall values (Today, Hourly, MAR Acc, Yearly Acc) | LOW | Data directly from tipping bucket sensors; no transformation needed |
| **Voltage Monitoring** | Critical for remote station operation — battery and solar voltage indicate system health | LOW | ADC readings; critical for stations without grid power |
| **Date/Time Display** | Timestamp is essential for data interpretation and troubleshooting | LOW | NTP sync or manual configuration; GPS time if available |
| **Station Identification** | Operators manage multiple stations; must identify which unit they're viewing | LOW | Station ID displayed prominently |
| **Communication Status** | Operators must know if data is reaching the server | LOW | Signal strength (RSSI), connection status indicators |
| **Main Menu Navigation** | Access to settings, calibration, and file management is standard RTU behavior | MEDIUM | Must support 7" touchscreen (large touch targets) |
| **Settings Configuration** | RTUs require configurable parameters (thresholds, network, sensor types) | MEDIUM | Submenus for each category; form-based input |
| **Calibration View** | Technicians need to verify sensor readings during maintenance | LOW | Read-only display of live sensor values |
| **File Management** | Local CSV storage requires navigation, viewing, and deletion | MEDIUM | Must handle flash memory on embedded device |
### Differentiators (Competitive Advantage)
Features that set the product apart. Not required, but valuable.
| Feature | Value Proposition | Complexity | Notes |
|---------|-------------------|------------|-------|
| **Historical Data Graphs** | Visual trends help operators anticipate conditions — rainfall patterns, battery drain cycles | MEDIUM | Chart.js can render on Pi Zero 2W; keep data points limited for performance |
| **Configurable Alarm Thresholds** | Users can set custom alerts for danger/warning levels; visual indicators for threshold breach | MEDIUM | Persist to local config file; apply on dashboard display |
| **Touch-Optimized Interface** | 7" capacitive touchscreen requires large buttons (min 48px), clear visual hierarchy | LOW | Bootstrap provides responsive components; custom CSS for kiosk sizing |
| **Dual-Mode Display** | Same app serves local kiosk (1024x600) and remote HD access — reduces maintenance | MEDIUM | Use CSS media queries to adapt layout; single codebase |
| **Real-time Data Updates** | Socket.IO push for live sensor values without page refresh | MEDIUM | Already in stack research; critical for professional appearance |
| **Visual Threshold Indicators** | Color-coded rainfall/level status (green/yellow/red) provides at-a-glance awareness | LOW | Simple CSS classes based on threshold comparison |
| **CSV Transmission Status** | Users want to know if/when data was successfully sent to server | LOW | Log transmission attempts; display last successful sync time |
### Anti-Features (Commonly Requested, Often Problematic)
Features that seem good but create problems.
| Feature | Why Requested | Why Problematic | Alternative |
|---------|---------------|-----------------|-------------|
| **Third-party Cloud Integration** | Users may want Ambient Weather Network, Weather Underground, etc. | Adds complexity, requires accounts, breaks direct server model | Stick to direct myvscada server; provide CSV export if needed |
| **Mobile Native App** | "Users want an app" | Development/maintenance overhead for iOS/Android; web interface is accessible | Responsive web works on mobile browsers |
| **Real-time Chat/Notifications** | Operators want to communicate | Not the purpose of monitoring station; adds infrastructure | Email/SMS alerts via server-side (future consideration) |
| **Multi-station Dashboard** | View all stations from one screen | Requires server-side aggregation; out of RTU scope | Server-side dashboard aggregates; RTU shows local only |
| **Cloud Data Storage** | "Store data in the cloud" | RTU is embedded; limited storage is intentional | Direct CSV transmission to myvscada server |
| **Video Streaming** | Some systems include camera | Heavy bandwidth, storage; not in original spec | Out of scope |
| **User Authentication (local)** | "Secure the interface" | Adds complexity; kiosk mode typically runs without auth | Remote interface can have auth (port 9090); local is read-only by default |
## Feature Dependencies
```
Dashboard Display
└──requires──> Real-time Data Updates (Socket.IO)
└──requires──> Backend Sensor Reading
Settings Menus
└──requires──> Configuration Persistence (local config file)
└──requires──> File System Access
Calibration View
└──requires──> Live Sensor Reading
File Management
└──requires──> CSV Data Storage
└──requires──> Sensor Data Collection
Threshold Alerts
└──requires──> Configurable Settings
└──enhances──> Dashboard Display
Historical Graphs
└──requires──> CSV Data Storage
└──enhances──> Dashboard Display
```
### Dependency Notes
- **Dashboard requires real-time updates:** The UI must receive sensor readings via Socket.IO; this requires Flask-SocketIO backend infrastructure
- **Historical graphs require CSV storage:** Can't display trends without historical data; implies Phase 2 (CSV workflow) must precede graphs
- **Settings require persistence:** All configuration must survive reboot; store in JSON/text config file on local filesystem
- **Threshold alerts enhance dashboard:** Visual indicators (color-coded values) depend on both the threshold configuration AND the dashboard display
## MVP Definition
### Launch With (v1)
Minimum viable product — what's needed to validate the concept.
- [x] **Dashboard with rainfall readings** — Core value: users see Today, Hourly, MAR Acc, Yearly Acc
- [x] **Voltage monitoring display** — Battery and solar; critical for remote operation
- [x] **Date/time and station ID** — Basic identification
- [x] **Communication status** — Signal strength, connection state
- [x] **Main menu navigation** — Access to settings submenus
- [x] **Real-time updates (Socket.IO)** — Live data without refresh
### Add After Validation (v1.x)
Features to add once core is working.
- [ ] **Historical data graphs** — Chart.js visualization of rainfall trends
- [ ] **Configurable alarm thresholds** — User-settable danger/warning levels
- [ ] **Visual threshold indicators** — Color-coded status on dashboard
- [ ] **File management UI** — Navigate/delete CSV files
- [ ] **CSV transmission status** — Show last sync time
### Future Consideration (v2+)
Features to defer until product-market fit is established.
- [ ] **Multi-day/hour trend analysis** — Extended historical view
- [ ] **Data export functionality** — Download CSV from browser
- [ ] **Remote authentication** — Login for port 9090 interface
- [ ] **SMS/email alerts** — Server-side notification system
## Feature Prioritization Matrix
| Feature | User Value | Implementation Cost | Priority |
|---------|------------|---------------------|----------|
| Dashboard rainfall display | HIGH | LOW | P1 |
| Voltage monitoring | HIGH | LOW | P1 |
| Date/time & station ID | HIGH | LOW | P1 |
| Communication status | HIGH | LOW | P1 |
| Main menu navigation | HIGH | MEDIUM | P1 |
| Real-time Socket.IO updates | HIGH | MEDIUM | P1 |
| Settings configuration | MEDIUM | MEDIUM | P1 |
| Calibration view | MEDIUM | LOW | P1 |
| File management | MEDIUM | MEDIUM | P2 |
| Historical graphs | MEDIUM | MEDIUM | P2 |
| Threshold indicators | MEDIUM | LOW | P2 |
| Alarm threshold config | MEDIUM | MEDIUM | P2 |
| Transmission status | LOW | LOW | P2 |
| Touch-optimized UI | HIGH | LOW | P1 |
| Dual-mode display | MEDIUM | MEDIUM | P2 |
**Priority key:**
- P1: Must have for launch
- P2: Should have, add when possible
- P3: Nice to have, future consideration
## Competitor Feature Analysis
| Feature | Ambient Weather Network | Weather Display | Davis WeatherLink | Our Approach |
|---------|------------------------|-----------------|-------------------|--------------|
| Real-time dashboard | YES | YES | YES | P1 — Core |
| Historical graphs | YES | YES | YES | P2 — Post-launch |
| Alarm thresholds | YES (cloud) | YES (local) | YES (cloud) | P2 — Configurable |
| Touch interface | Responsive web | Windows app | Native app | P1 — Web-based |
| Data export | Via API | YES | YES | P2 — Future |
| Local storage | Cloud only | Local + cloud | Cloud + local | P1 — Local CSV |
| Multi-station | YES | Via addons | YES | Server handles |
**Analysis Notes:**
- Consumer solutions (Ambient, Davis) lean heavily on cloud — we differ by direct server transmission
- Professional solutions (Weather Display, AQUARIUS) offer local storage + sophisticated alarms
- Our differentiator: lightweight embedded design with direct myvscada integration
## Sources
- **Rainwise/Ambient Weather Network** — Consumer weather station dashboard patterns
- **Weather Display** — Professional weather station software with extensive local features
- **KISTERS datasphere** — Enterprise hydrological monitoring platform
- **Aquatic Informatics AQUARIUS** — Professional water data management
- **Hydro-Logic Timeview** — Flood monitoring with automated alarms
- **California Data Exchange Center (CDEC)** — Government hydrological monitoring
- **Perry Weather** — Modern weather monitoring with threshold alerts
- **Project context (THE_IDEA.md)** — Existing RTU feature set
- **Project context (PROJECT.md)** — Core value and constraints
---
*Feature research for: Rainfall Monitoring Station RTU Web Interface*
*Researched: 2026-03-12*

282
.planning/research/PITFALLS.md Normal file
View File

@@ -0,0 +1,282 @@
# Pitfalls Research
**Domain:** Raspberry Pi Web Monitoring Station Interface
**Researched:** 2026-03-12
**Confidence:** HIGH
## Critical Pitfalls
### Pitfall 1: Flask Performance Degradation on Pi Zero 2 W
**What goes wrong:**
Web server takes 60+ seconds to start, consumes 60-100% CPU continuously, causing severe UI lag and unresponsive touchscreen. Pages load slowly even for simple requests.
**Why it happens:**
The Pi Zero 2 W has only a single ARM11 core at 1GHz. Flask's development server isn't optimized for embedded deployment. Every request blocks the event loop. Python's GIL limits concurrency. The "quick start" approach doesn't account for resource constraints.
**How to avoid:**
- Use a production WSGI server (Gunicorn with multiple workers won't help on single core — use event-driven approach)
- Consider lighter alternatives: Python's built-in http.server for simple needs, or Node.js for better concurrency
- Cache static assets aggressively
- Implement server-sent events (SSE) instead of polling for real-time updates
- Pre-compile templates, minimize Python imports at startup
**Warning signs:**
- First page load takes >30 seconds
- CPU stays above 50% when serving pages
- Touchscreen response delay >500ms
**Phase to address:** Phase 1 (UI/Display) — Choose the right backend technology before building UI
---
### Pitfall 2: Chromium Kiosk Mode Instability
**What goes wrong:**
Chromium exits kiosk mode after system updates, monitor power cycling, or resolution changes. The screen shows a window instead of fullscreen, breaking the dedicated display experience. Session restore prompts interrupt kiosk operation.
**Why it happens:**
- Debian/Raspbian updates replace Chromium with different versions that have different flag behaviors
- Wayland/Labwc (default on Bookworm) behaves differently than X11 for kiosk flags
- Monitor power on/off triggers display mode changes that exit kiosk
- Default session restore behavior shows popup dialogs
**How to avoid:**
- Pin Chromium version: `sudo apt-mark hold chromium-browser`
- Use X11 instead of Wayland for kiosk stability (configure via raspi-config)
- Add flags to prevent session restore: `--disable-session-crashed-bubble --disable-infobars --noerrdialogs`
- Implement a watchdog script that restarts Chromium if it exits or enters wrong mode
- Test with monitor power cycling during development
**Warning signs:**
- `/usr/bin/chromium --version` shows different version after apt upgrade
- Kiosk appears as window after system reboot
- "Restore pages?" bubble appears on boot
**Phase to address:** Phase 1 (UI/Display) — Resolve kiosk stability before considering display "complete"
---
### Pitfall 3: SD Card Corruption from Data Logging
**What goes wrong:**
After weeks of operation, the system becomes read-only or fails to boot. All sensor data and configuration is lost. The RTU stops transmitting, creating data gaps in the rainfall record.
**Why it happens:**
- Continuous CSV writing to SD card causes wear
- Power interruptions during write operations corrupt the filesystem
- No write caching strategy — every sensor reading triggers a file sync
- Logs and temp data accumulate on SD card
**How to avoid:**
- Mount `/var/log` and `/tmp` as tmpfs (RAM disks)
- Write sensor data to memory buffer, flush to SD only periodically (e.g., every 5 minutes)
- Use SQLite with WAL mode for atomic writes instead of CSV append
- Implement proper shutdown button (hardware + software) to prevent power-loss during writes
- Consider USB SSD for data storage if available
- Disable swap: `sudo dphys-swapfile swapoff`
**Warning signs:**
- `dmesg | grep -i error` shows I/O errors
- Filesystem becomes read-only randomly
- Boot failures after power outage
**Phase to address:** Phase 2 (Data/CSV) — Data persistence strategy must be designed upfront
---
### Pitfall 4: Touchscreen Unresponsiveness at 7-inch Display
**What goes wrong:**
Touches require repeated taps to register. There's noticeable lag between touch and UI response. The 7-inch official touchscreen feels "sluggish" compared to phone touch experience.
**Why it happens:**
- Official 7-inch touchscreen has ~33Hz polling rate (30-40ms between touch events)
- Web browser input handling adds additional latency
- GPU memory may be undersized for smooth rendering
- Xorg/Wayland compositor overhead on Pi Zero
**How to avoid:**
- Allocate sufficient GPU memory: `gpu_mem=128` in config.txt
- Use hardware-accelerated rendering where possible
- Design UI with large touch targets (minimum 48px, recommend 64px for primary actions)
- Add visual feedback for touches (immediate color change before action completes)
- Avoid rapid-fire touch interactions — design for deliberate touches
- Consider `UDEV=1` environment variable for better input handling
**Warning signs:**
- Single tap requires 2-3 attempts to register
- Slider/scroll gestures feel jerky
- UI feels "mushy" — no immediate feedback
**Phase to address:** Phase 1 (UI/Display) — Test touch responsiveness early, not as afterthought
---
### Pitfall 5: Network Data Transmission Failures Silently Lost
**What goes wrong:**
CSV files fail to transmit to myvscada server but no alert is generated. Data accumulates locally until storage fills. The monitoring station appears operational but isn't actually reporting.
**Why it happens:**
- FTP/SFTP connections fail due to network issues but code doesn't retry aggressively
- No local queue for failed transmissions
- No verification that server actually received the file
- Transmission happens but errors are logged only, not surfaced to UI
**How to avoid:**
- Implement transmission queue with retry logic (exponential backoff)
- Verify file receipt via server acknowledgment or file existence check
- Show transmission status prominently on dashboard (last successful sync, pending count)
- Implement dead letter queue — alert after N failed attempts
- Log all transmission attempts with timestamps and error codes
**Warning signs:**
- Dashboard shows "transmitting" but files never leave local storage
- Server reports missing data but RTU shows "success"
- Network logs show connection timeouts but no UI indication
**Phase to address:** Phase 3 (Network) — Build transmission verification before considering networking "done"
---
### Pitfall 6: Real-Time Data Staleness Without Notification
**What goes wrong:**
Dashboard shows rainfall readings that are hours old. User doesn't realize data hasn't updated. The RTU appears to work but sensor polling has stopped.
**Why it happens:**
- No watchdog for sensor polling process
- Web page uses initial data load only — no auto-refresh
- Backend fails silently, continues serving stale cached data
- No "last updated" timestamp displayed
**How to avoid:**
- Always display "last updated" timestamp prominently
- Implement WebSocket or Server-Sent Events for live updates
- If using polling, show "updating..." indicator and timeout after N seconds
- Add backend health check — if sensor reader hasn't updated in X minutes, show warning
- Implement process monitoring (systemd watchdog or custom health check)
**Warning signs:**
- Timestamps on dashboard don't change for extended periods
- Rainfall values don't match physical bucket tipping
- No indication of "live" vs "stale" data
**Phase to address:** Phase 1 (UI/Display) — Data freshness is a UX issue, not just backend
---
## Technical Debt Patterns
| Shortcut | Immediate Benefit | Long-term Cost | When Acceptable |
|----------|-------------------|----------------|-----------------|
| Using Flask dev server | Simple startup | High CPU, slow response | Never in production |
| Writing CSV on every reading | Simple code | SD card wear, data loss risk | Never |
| HTTP polling for updates | Simple implementation | Wastes CPU, UI lag | Only if SSE/WebSocket unavailable |
| Hardcoded IP addresses | Quick setup | Breaks when network changes | Never — use DNS/hostname |
| No transmission retry | Simpler code | Silent data loss | Never for operational data |
| Single network interface | Simple config | No resilience | Only for non-critical displays |
---
## Integration Gotchas
| Integration | Common Mistake | Correct Approach |
|-------------|----------------|------------------|
| FTP Server | Assuming passive mode works everywhere | Test with active mode, check firewall rules |
| SFTP | Using default SSH ciphers (slow) | Enable hardware acceleration, optimize ciphers |
| myvscada Server | No authentication verification | Test credentials before production |
| Sensor Hardware | Polling too frequently | Respect sensor timing, buffer readings |
| Mobile Network | No reconnection logic | Implement connection watchdog |
---
## Performance Traps
| Trap | Symptoms | Prevention | When It Breaks |
|------|----------|------------|----------------|
| Large CSV files | Memory exhaustion, slow transmission | Chunk files, limit records per file | At >10MB files |
| Many concurrent browser tabs | RAM exhaustion | Limit connections, close unused | At 3+ tabs on Pi Zero |
| Animated UI elements | High CPU, battery drain | Minimize animations, use CSS transforms | Always on embedded |
| Heavy JavaScript framework | Slow load, high memory | Use vanilla JS or lightweight framework | Any framework >50KB |
---
## Security Mistakes
| Mistake | Risk | Prevention |
|---------|------|------------|
| No authentication on local port 8080 | Physical access = full control | Implement session auth, even for local |
| Plain FTP for data transmission | Credential theft | Use SFTP/SCP only |
| Exposed network ports without firewall | Remote exploitation | Firewall rules, minimal exposure |
| Storing passwords in plain text | Credential exposure | Use environment variables or secure storage |
| No input validation on settings | Command injection | Validate all inputs, sanitize before use |
---
## UX Pitfalls
| Pitfall | User Impact | Better Approach |
|---------|-------------|-----------------|
| No confirmation for destructive actions | Accidental reset of calibration/data | Require explicit confirmation dialogs |
| Settings changes apply immediately | Unintended side effects | Use "Preview" then "Apply" pattern |
| No visual feedback for touch | User double-taps, causes errors | Immediate visual + haptic feedback |
| Error messages are technical | Non-technical users confused | User-friendly messages, offer solutions |
| No offline indication | User trusts data when network down | Clear "offline" banner, show last update |
---
## "Looks Done But Isn't" Checklist
- [ ] **Kiosk Mode:** Verified working after `apt upgrade && reboot` — not just on fresh install
- [ ] **Touchscreen:** Tested with actual finger touches, not mouse clicks — 33Hz polling shows difference
- [ ] **Data Transmission:** Verified file actually arrives at server — not just "sent"
- [ ] **Data Freshness:** Dashboard shows "last updated" timestamp — not just current values
- [ ] **Power Loss:** System survives unexpected power cut — test by pulling plug
- [ ] **Remote Access:** Works from external network — not just localhost
- [ ] **Memory Usage:** Verified stable over 24hr run — no gradual growth
- [ ] **Temperature:** Verified works in expected environmental conditions
---
## Recovery Strategies
| Pitfall | Recovery Cost | Recovery Steps |
|---------|---------------|----------------|
| SD card corruption | HIGH | Requires physical access, reimage, restore backup |
| Kiosk exit | LOW | Watchdog script auto-restarts, or manual `sudo systemctl restart kiosk` |
| Transmission failure | MEDIUM | Check queue, retry manually, investigate root cause |
| Sensor stop | MEDIUM | Restart sensor polling service, check wiring |
| Network down | LOW | Show offline indicator, auto-reconnect with backoff |
---
## Pitfall-to-Phase Mapping
| Pitfall | Prevention Phase | Verification |
|---------|------------------|--------------|
| Flask performance | Phase 1: UI Design | Benchmark first page load, CPU under load |
| Kiosk instability | Phase 1: UI Design | Test after system updates |
| SD card corruption | Phase 2: Data/CSV | Power-loss test, check for I/O errors |
| Touchscreen lag | Phase 1: UI Design | Physical touch testing |
| Transmission failures | Phase 3: Network | Monitor queue, verify server receipt |
| Stale data | Phase 1: UI Design | Verify timestamps update |
| Network resilience | Phase 3: Network | Test with disconnected network |
---
## Sources
- Raspberry Pi Forums: Kiosk mode issues, Chromium autostart problems
- Raspberry Pi Stack Exchange: Flask performance on Pi Zero W
- GitHub Issue #3777 (raspberrypi/linux): 7" touchscreen polling rate at 33Hz
- Hackaday: "Raspberry Pi And The Story Of SD Card Corruption"
- pidiylab.com: SD card corruption prevention, performance tuning
- raspberrytips.com: Common Raspberry Pi problems and solutions
- XDA Developers: Common Raspberry Pi mistakes (2025)
- Community reports: SFTP slow speeds on Pi 3/4, Wayland kiosk issues
---
*Pitfalls research for: Raspberry Pi Web Monitoring Station RTU*
*Researched: 2026-03-12*

157
.planning/research/STACK.md Normal file
View File

@@ -0,0 +1,157 @@
# Stack Research
**Domain:** Web-based RTU Interface for Raspberry Pi Embedded Sensor Monitoring
**Researched:** 2026-03-12
**Confidence:** HIGH
## Recommended Stack
### Core Technologies
| Technology | Version | Purpose | Why Recommended |
|------------|---------|---------|-----------------|
| **Flask** | 3.1.0 | Lightweight Python web framework | The de facto standard for Raspberry Pi web projects. Extensive ecosystem, minimal overhead, perfect for Pi Zero 2 W. Multiple verified projects use Flask for sensor dashboards. |
| **Flask-SocketIO** | 5.6.0 | Real-time bidirectional communication | Proven combination with Flask for live sensor data. Latest version (5.6.0) released Dec 2025, compatible with Python 3.8+. |
| **Chart.js** | 4.4.x | JavaScript charting library | Industry standard for dashboard visualizations. Lightweight (~60KB), canvas-based rendering performs well on embedded displays. Extensive documentation and examples for Raspberry Pi projects. |
| **Bootstrap** | 5.3.x | CSS framework for responsive UI | Works exceptionally well for touchscreen interfaces. Built-in support for 1024x600 resolution layouts. Grid system simplifies the menu-driven RTU interface. |
| **Python** | 3.11+ | Backend runtime | Raspberry Pi OS (Bookworm) ships with Python 3.11. Excellent sensor library ecosystem (GPIO, I2C, serial). |
### Supporting Libraries
| Library | Version | Purpose | When to Use |
|---------|---------|---------|-------------|
| **python-socketio** | 5.12.0+ | Socket.IO client/server | Required by Flask-SocketIO, handles WebSocket transport |
| **psutil** | 6.1.0+ | System monitoring | Reading CPU temp, memory for health checks |
| **pyserial** | 3.5+ | Serial communication | Reading sensor data from ADC/serial ports |
| **smbus2** | 0.5+ | I2C communication | Water level sensors, other I2C devices |
| **gunicorn** | 23.0+ | WSGI server | Production deployment, replaces Flask dev server |
| **eventlet** | 0.40+ | Async worker | Recommended for Flask-SocketIO production (better than gevent) |
| **paramiko** | 3.5+ | SSH/SFTP | SCP/SFTP file transmission to server |
### Real-Time Communication
| Technology | Purpose | Why Use |
|------------|---------|---------|
| **Socket.IO** (via Flask-SocketIO) | Server-push for sensor updates | Bidirectional, auto-reconnection, fallback to polling. Proven in multiple Pi monitoring projects. |
### Development Tools
| Tool | Purpose | Notes |
|------|---------|-------|
| **systemd** | Service management | Auto-start Flask app on boot, restart on failure |
| **Chromium Kiosk Mode** | Local display | `chromium-browser --kiosk --incognito http://localhost:8080` |
| **nginx** | Reverse proxy | Route 8080 (kiosk) and 9090 (remote) ports |
## Installation
```bash
# Core Python packages
pip install flask==3.1.0
pip install flask-socketio==5.6.0
pip install python-socketio==5.12.0
# Sensor and system access
pip install psutil==6.1.0
pip install pyserial==3.5
pip install smbus2==0.5
# Production server
pip install gunicorn==23.0
pip install eventlet==0.40
# File transmission
pip install paramiko==3.5
# Frontend (via CDN or download)
# - Bootstrap 5.3.x CSS/JS
# - Chart.js 4.4.x
# - Socket.IO client 4.x
```
## Alternatives Considered
| Recommended | Alternative | When to Use Alternative |
|-------------|-------------|-------------------------|
| Flask | FastAPI | Only if you need async/await for high-concurrency (50+ concurrent users). Flask is simpler for embedded. |
| Flask | Quart | If you prefer async Flask syntax. Flask-SocketIO still works but adds complexity. |
| Socket.IO | Server-Sent Events (SSE) | SSE is simpler (no JS library) but only one-way. Use only if you never need client→server real-time messages. |
| Socket.IO | Polling | Only for absolute minimal overhead. Not recommended—Socket.IO overhead is negligible, benefits are substantial. |
| Chart.js | Lightweight Charts | Only if you need financial-style candlestick charts. Chart.js is better for standard line/bar charts. |
| Bootstrap 5 | Pure CSS | If you need absolute minimum footprint. Bootstrap adds ~200KB but saves大量 development time for touchscreen layouts. |
## What NOT to Use
| Avoid | Why | Use Instead |
|-------|-----|-------------|
| **Django** | Too heavy (~30MB+), complex setup, overkill for embedded RTU | Flask (lightweight, sufficient for this use case) |
| **FastAPI** | While fast, adds Pydantic complexity without benefit. Async not needed for single-user dashboard | Flask (simpler, same performance for this use case) |
| **React/Vue/Angular** | Framework overhead, build step required, memory intensive on Pi Zero | Vanilla JS + Bootstrap (direct, no build, ~500KB total) |
| **Node.js** | Adds runtime complexity, unnecessary for Python-centric sensor stack | Python/Flask (native sensor access, simpler deployment) |
| **gunicorn (threading mode)** | Doesn't work well with Flask-SocketIO | eventlet worker (required for WebSocket support) |
| **InfluxDB + Grafana** | Full database + visualization stack, too heavy for Pi Zero | Flask + Chart.js (direct data, minimal overhead) |
## Stack Patterns by Variant
**If running on Pi Zero 2 W (primary target):**
- Use Flask development server (`python app.py`) for simplicity
- Single worker, Socket.IO handles concurrency
- Minimal CSS, avoid heavy JavaScript frameworks
- Reason: Pi Zero 2 W has limited RAM (~512MB), every MB counts
**If running on Pi 3B (alternative):**
- Can use gunicorn + eventlet with 2-4 workers
- More aggressive caching allowed
- Consider nginx for static files
- Reason: Pi 3B has more RAM (~1GB), can handle more overhead
**If remote HD interface needed (port 9090):**
- Serve same Flask app on different port
- Use CSS media queries to show full HD layout on larger screens
- No separate codebase needed
- Reason: Single app, responsive design handles both
## Version Compatibility
| Package | Compatible With | Notes |
|---------|----------------|-------|
| Flask 3.1.0 | Python 3.10+ | Requires Python 3.10 minimum |
| Flask-SocketIO 5.6.0 | Flask >=2.1.0, python-socketio >=5.12.0 | Use latest python-socketio for best performance |
| Socket.IO client 4.x | Flask-SocketIO 5.x | Client/server version must match protocol |
| Bootstrap 5.3.x | All modern browsers | No IE11 support (not needed for kiosk) |
| Chart.js 4.4.x | All modern browsers | Requires canvas support |
| eventlet 0.40+ | Python 3.10+ | Monkey-patching required before other imports |
| psutil 6.1.0 | Python 3.8+ | Works on all Pi models |
## Why This Stack
### Performance on Pi Zero 2 W
- Flask overhead: ~10-15MB RAM (vs Django ~80MB)
- Chart.js canvas rendering is GPU-accelerated on Pi
- Socket.IO connection: ~1-2MB per client
- Total app footprint: ~50-80MB RAM
### Developer Experience
- Flask: Simple, Pythonic, extensive documentation
- Bootstrap: Ready-made touchscreen-friendly components
- Chart.js: Extensive examples for real-time data
- Single codebase serves both kiosk (1024x600) and remote (HD)
### Ecosystem Evidence
Multiple verified GitHub projects use this exact stack:
- `g1forfun/Pi-Monitor` — Flask + Socket.IO + Chart.js (Pi 5 dashboard)
- `Mohammed-Shehsin/sensehat-iot-dashboard` — Flask + real-time sensors
- `idleCyrex/raspberry-pi-weather-station` — Flask + environmental monitoring
- Multiple Raspberry Pi community tutorials (Cytron, official forums)
## Sources
- **Flask-SocketIO PyPI** — Verified latest version 5.6.0 (Dec 2025), Python 3.8+ compatibility
- **piwheels.org** — Verified Flask-SocketIO 5.6.0 available for Raspberry Pi OS
- **GitHub g1forfun/Pi-Monitor** — Active project (last push Feb 2026), Flask + Socket.IO + Chart.js pattern
- **GitHub sensehat-iot-dashboard** — Real-time sensor example with Flask
- **Cytron.io tutorial** — "Create a Real-Time Raspberry Pi Dashboard" (Dec 2025), Flask + Chart.js + psutil
- **Multiple Raspberry Pi forum discussions** — Confirmed Flask + Socket.IO is the community standard for sensor dashboards
---
*Stack research for: Raspberry Pi RTU Web Interface*
*Researched: 2026-03-12*

219
.planning/research/SUMMARY.md Normal file
View File

@@ -0,0 +1,219 @@
# Project Research Summary
**Project:** TCKRTUIYO - Rainfall Station RTU Web Interface
**Domain:** Embedded IoT/Web Monitoring Systems
**Researched:** 2026-03-12
**Confidence:** HIGH
## Executive Summary
This is a **Raspberry Pi-based web monitoring interface for rainfall telemetry**. The system reads tipping bucket sensors, displays real-time data on a 7-inch touchscreen (kiosk mode), and transmits CSV files to a remote myvscada server. The target hardware is Pi Zero 2 W with ~512MB RAM — significantly constrained compared to typical web servers.
**Recommended approach:** Build a vertical slice first (sensor → API → basic UI), then layer features. Use Flask with Flask-SocketIO for real-time updates, Bootstrap for responsive touchscreen layouts, and Chart.js for future historical graphs. The architecture follows a clear three-tier pattern: Web Layer (dual ports 8080/9090), API Layer (blueprint-based Flask routes), and Service Layer (Sensor, Data, Network, Config services).
**Key risks to mitigate:**
1. **Performance on Pi Zero 2 W** — Flask dev server will be unusable; must use eventlet workers and optimize imports
2. **SD card corruption** — Continuous CSV writes destroy SD cards; use SQLite or buffered writes with tmpfs
3. **Chromium kiosk instability** — Updates break kiosk mode; pin version and add watchdog script
4. **Silent data transmission failures** — FTP/SFTP failures must be visible in UI, not just logged
---
## Key Findings
### Recommended Stack
**Core technologies:**
- **Flask 3.1.0** — Lightweight web framework, ~15MB RAM vs Django's 80MB+
- **Flask-SocketIO 5.6.0** — Real-time server-push for live sensor updates
- **Chart.js 4.4.x** — Canvas-based charting, GPU-accelerated on Pi
- **Bootstrap 5.3.x** — Touchscreen-friendly grid system, ~200KB
- **Python 3.11+** — Raspberry Pi OS Bookworm ships with 3.11
**Supporting libraries:**
- **eventlet 0.40+** — Required for Flask-SocketIO production (not gunicorn threading)
- **paramiko 3.5+** — SFTP/SCP file transmission
- **psutil 6.1.0** — System health monitoring
- **pyserial 3.5** — Sensor communication
**What NOT to use:** Django (too heavy), FastAPI (unnecessary Pydantic overhead), React/Vue/Angular (build step + memory intensive), Node.js (breaks Python sensor stack), InfluxDB+Grafana (too heavy for Pi Zero).
### Expected Features
**Must have (table stakes):**
- Real-time rainfall display (Today, Hourly, MAR Acc, Yearly Acc)
- Voltage monitoring (battery and solar)
- Date/time and station identification
- Communication status (RSSI, connection state)
- Main menu navigation for 7" touchscreen
- Settings configuration (thresholds, network, calibration)
- Calibration view (live sensor readings)
- File management (CSV navigation/deletion)
**Should have (differentiators):**
- Historical data graphs (Chart.js on Pi Zero is viable with limited data points)
- Configurable alarm thresholds with visual indicators
- Touch-optimized UI (minimum 48px touch targets, 64px for primary)
- Dual-mode display (single codebase serves kiosk 1024x600 and remote HD)
- Real-time Socket.IO push (live updates without page refresh)
- CSV transmission status (last sync time, pending count)
**Defer (v2+):**
- Multi-station dashboard (server-side aggregation)
- Cloud data storage (direct CSV transmission is the model)
- Video streaming (not in spec)
- SMS/email alerts (future server-side feature)
- Data export from browser (low priority)
### Architecture Approach
The system follows a **three-tier architecture** with clear component boundaries:
1. **Web Layer (Dual Port)** — Port 8080 for kiosk (no auth, fixed 1024x600), Port 9090 for remote (auth required, responsive HD)
2. **API Layer** — Flask blueprints for routes: `/api/status`, `/api/sensors`, `/api/settings`, `/api/files`, `/api/calibration`
3. **Service Layer** — SensorService (GPIO/ADC polling), DataLogger (CSV write), NetworkService (FTP/SFTP), ConfigService (JSON files)
**Data flow:** Timer → Sensor Service reads hardware → Update in-memory state → DataLogger writes CSV → API returns JSON → Web layer displays
**Build order:** Foundation (sensor→API→basic UI) → Local Interface (settings, calibration) → Data Persistence (CSV logging, transmission) → Remote Access (auth, responsive)
### Critical Pitfalls
1. **Flask performance on Pi Zero** — Dev server takes 60+ seconds to start, blocks on every request. **Avoid:** Use eventlet workers, pre-compile templates, cache static assets, implement SSE instead of polling.
2. **Chromium kiosk instability** — Updates replace version, Wayland/X11 differences break flags, session restore prompts interrupt. **Avoid:** Pin Chromium version (`apt-mark hold`), use X11 instead of Wayland, add watchdog script, test after every system update.
3. **SD card corruption** — Continuous CSV writes cause wear, power loss corrupts filesystem. **Avoid:** Mount `/var/log` and `/tmp` as tmpfs, write to memory buffer and flush every 5 minutes, use SQLite with WAL mode, implement shutdown button, disable swap.
4. **Touchscreen unresponsiveness** — 7-inch display has 33Hz polling rate, causing lag. **Avoid:** Set `gpu_mem=128`, design 64px touch targets, provide immediate visual feedback, avoid rapid-fire interactions.
5. **Network transmission failures silent** — FTP/SFTP fails but only logged, not surfaced to UI. **Avoid:** Implement retry queue with exponential backoff, verify server receipt, show transmission status prominently, dead-letter alert after N failures.
6. **Stale data without notification** — Dashboard shows old readings with no indication. **Avoid:** Always display "last updated" timestamp, use Socket.IO for live updates, add backend health check, show warning if sensor reader hasn't updated in X minutes.
---
## Implications for Roadmap
Based on research, suggested phase structure:
### Phase 1: Foundation & Kiosk UI
**Rationale:** Must prove sensor-to-display works before adding persistence or networking. This is a vertical slice — each layer works end-to-end before adding features.
**Delivers:**
- Hardware interface (GPIO/ADC)
- Sensor service with live readings
- Basic API (status, sensors endpoints)
- Minimal dashboard UI (rainfall, voltage, time, station ID)
- Port 8080 kiosk server with Chromium autostart
**Addresses:** FEATURES table stakes — dashboard display, voltage monitoring, date/time, station ID, main menu
**Avoids:** Pitfall 1 (Flask performance) — use eventlet from start, not as afterthought; Pitfall 4 (touchscreen lag) — test touch responsiveness early; Pitfall 6 (stale data) — show timestamps from day one
**Research flag:** Complex integration with sensor hardware — may need `/gsd-research-phase` for GPIO/ADC specifics if not already available from existing RTU.
### Phase 2: Data Persistence
**Rationale:** CSV logging is the primary data workflow. Must establish proper storage before transmission. Avoids Pitfall 3 (SD card corruption) by designing correctly from the start.
**Delivers:**
- Data logger service with buffered writes
- SQLite or structured CSV storage
- tmpfs mounts for logs/tmp
- Settings UI and configuration persistence
- Calibration view UI
**Addresses:** FEATURES settings configuration, calibration view
**Avoids:** Pitfall 3 (SD card corruption) — implement tmpfs, buffered writes, proper shutdown
**Research flag:** Standard patterns — SQLite and CSV logging are well-documented. Skip deep research.
### Phase 3: Network Transmission
**Rationale:** Data must reach myvscada server. Requires proven persistence layer first. Avoids Pitfall 5 (silent transmission failures) by building verification upfront.
**Delivers:**
- Network service with FTP/SFTP/SCP
- Transmission queue with retry logic
- Server receipt verification
- Transmission status UI (last sync, pending count)
- File management UI
**Addresses:** FEATURES file management, transmission status
**Avoids:** Pitfall 5 (silent failures) — queue with backoff, verification, UI visibility
**Research flag:** May need deeper research on myvscada server integration specifics (API, expected file format).
### Phase 4: Remote Access
**Rationale:** Full HD remote interface is the "pro" experience. Can reuse most backend. Focus on responsive design and authentication.
**Delivers:**
- Port 9090 server with authentication
- Responsive templates for Full HD
- Historical graphs (Chart.js)
- Configurable alarm thresholds with visual indicators
**Addresses:** FEATURES historical graphs, threshold indicators, alarm threshold config, dual-mode display
**Research flag:** Standard Flask + Bootstrap patterns — skip deep research.
### Phase Ordering Rationale
- **Vertical slice first:** Each layer works end-to-end before adding complexity
- **Data before network:** Can't transmit what hasn't been stored correctly
- **Kiosk before remote:** Local display is primary use case; remote is secondary
- **Pitfall prevention:** Each phase addresses specific pitfalls identified in research
### Research Flags
Phases likely needing deeper research during planning:
- **Phase 1:** Sensor hardware integration — need specifics on existing RTU GPIO/ADC interface
- **Phase 3:** myvscada server API — format expectations, authentication, transmission protocol
Phases with standard patterns (skip research-phase):
- **Phase 2:** SQLite/CSV logging is well-documented
- **Phase 4:** Flask + Bootstrap responsive design is standard
---
## Confidence Assessment
| Area | Confidence | Notes |
|------|------------|-------|
| Stack | HIGH | Verified with multiple GitHub projects using exact Flask+SocketIO+Chart.js pattern on Pi |
| Features | HIGH | Based on competitive analysis of Rainwise, Weather Display, Davis WeatherLink |
| Architecture | HIGH | Matches proven IoT/SCADA three-tier patterns, clear component boundaries |
| Pitfalls | HIGH | Documented from Raspberry Pi forums, Stack Exchange, community reports |
**Overall confidence:** HIGH
### Gaps to Address
- **Sensor hardware specifics:** Research assumes GPIO/ADC interface available. Need to verify existing RTU sensor connection method.
- **myvscada integration:** Research assumes FTP/SFTP transmission but need to verify server requirements (path, authentication, file format).
- **Touchscreen calibration:** 33Hz polling rate is documented but may need tuning per unit.
---
## Sources
### Primary (HIGH confidence)
- GitHub g1forfun/Pi-Monitor — Active project (Feb 2026), Flask+SocketIO+Chart.js pattern
- Cytron.io tutorial — "Create a Real-Time Raspberry Pi Dashboard" (Dec 2025)
- Flask-SocketIO PyPI — Verified 5.6.0 release (Dec 2025), Python 3.8+ compatibility
- piwheels.org — Verified packages available for Raspberry Pi OS
### Secondary (MEDIUM confidence)
- Raspberry Pi Forums — Kiosk mode issues, performance discussions
- Raspberry Pi Stack Exchange — Flask on Pi Zero W performance
- Multiple community tutorials — Confirms Flask is de facto standard
### Tertiary (LOW confidence)
- GitHub Issue #3777 (raspberrypi/linux) — 7" touchscreen 33Hz polling (needs verification)
- Community reports on SD card corruption — patterns consistent but may vary by card quality
---
*Research completed: 2026-03-12*
*Ready for roadmap: yes*