# Project Research Summary

**Project:** TCKRTUIYO - Rainfall Station RTU Web Interface
**Domain:** Embedded IoT/Web Monitoring Systems
**Researched:** 2026-03-12
**Confidence:** HIGH

## Executive Summary

This is a **Raspberry Pi-based web monitoring interface for rainfall telemetry**. The system reads tipping bucket sensors, displays real-time data on a 7-inch touchscreen (kiosk mode), and transmits CSV files to a remote myvscada server. The target hardware is Pi Zero 2 W with ~512MB RAM — significantly constrained compared to typical web servers.

**Recommended approach:** Build a vertical slice first (sensor → API → basic UI), then layer features. Use Flask with Flask-SocketIO for real-time updates, Bootstrap for responsive touchscreen layouts, and Chart.js for future historical graphs. The architecture follows a clear three-tier pattern: Web Layer (dual ports 8080/9090), API Layer (blueprint-based Flask routes), and Service Layer (Sensor, Data, Network, Config services).

**Key risks to mitigate:**
1. **Performance on Pi Zero 2 W** — Flask dev server will be unusable; must use eventlet workers and optimize imports
2. **SD card corruption** — Continuous CSV writes destroy SD cards; use SQLite or buffered writes with tmpfs
3. **Chromium kiosk instability** — Updates break kiosk mode; pin version and add watchdog script
4. **Silent data transmission failures** — FTP/SFTP failures must be visible in UI, not just logged

---

## Key Findings

### Recommended Stack

**Core technologies:**
- **Flask 3.1.0** — Lightweight web framework, ~15MB RAM vs Django's 80MB+
- **Flask-SocketIO 5.6.0** — Real-time server-push for live sensor updates
- **Chart.js 4.4.x** — Canvas-based charting, GPU-accelerated on Pi
- **Bootstrap 5.3.x** — Touchscreen-friendly grid system, ~200KB
- **Python 3.11+** — Raspberry Pi OS Bookworm ships with 3.11

**Supporting libraries:**
- **eventlet 0.40+** — Required for Flask-SocketIO production (not gunicorn threading)
- **paramiko 3.5+** — SFTP/SCP file transmission
- **psutil 6.1.0** — System health monitoring
- **pyserial 3.5** — Sensor communication

**What NOT to use:** Django (too heavy), FastAPI (unnecessary Pydantic overhead), React/Vue/Angular (build step + memory intensive), Node.js (breaks Python sensor stack), InfluxDB+Grafana (too heavy for Pi Zero).

### Expected Features

**Must have (table stakes):**
- Real-time rainfall display (Today, Hourly, MAR Acc, Yearly Acc)
- Voltage monitoring (battery and solar)
- Date/time and station identification
- Communication status (RSSI, connection state)
- Main menu navigation for 7" touchscreen
- Settings configuration (thresholds, network, calibration)
- Calibration view (live sensor readings)
- File management (CSV navigation/deletion)

**Should have (differentiators):**
- Historical data graphs (Chart.js on Pi Zero is viable with limited data points)
- Configurable alarm thresholds with visual indicators
- Touch-optimized UI (minimum 48px touch targets, 64px for primary)
- Dual-mode display (single codebase serves kiosk 1024x600 and remote HD)
- Real-time Socket.IO push (live updates without page refresh)
- CSV transmission status (last sync time, pending count)

**Defer (v2+):**
- Multi-station dashboard (server-side aggregation)
- Cloud data storage (direct CSV transmission is the model)
- Video streaming (not in spec)
- SMS/email alerts (future server-side feature)
- Data export from browser (low priority)

### Architecture Approach

The system follows a **three-tier architecture** with clear component boundaries:

1. **Web Layer (Dual Port)** — Port 8080 for kiosk (no auth, fixed 1024x600), Port 9090 for remote (auth required, responsive HD)
2. **API Layer** — Flask blueprints for routes: `/api/status`, `/api/sensors`, `/api/settings`, `/api/files`, `/api/calibration`
3. **Service Layer** — SensorService (GPIO/ADC polling), DataLogger (CSV write), NetworkService (FTP/SFTP), ConfigService (JSON files)

**Data flow:** Timer → Sensor Service reads hardware → Update in-memory state → DataLogger writes CSV → API returns JSON → Web layer displays

**Build order:** Foundation (sensor→API→basic UI) → Local Interface (settings, calibration) → Data Persistence (CSV logging, transmission) → Remote Access (auth, responsive)

### Critical Pitfalls

1. **Flask performance on Pi Zero** — Dev server takes 60+ seconds to start, blocks on every request. **Avoid:** Use eventlet workers, pre-compile templates, cache static assets, implement SSE instead of polling.

2. **Chromium kiosk instability** — Updates replace version, Wayland/X11 differences break flags, session restore prompts interrupt. **Avoid:** Pin Chromium version (`apt-mark hold`), use X11 instead of Wayland, add watchdog script, test after every system update.

3. **SD card corruption** — Continuous CSV writes cause wear, power loss corrupts filesystem. **Avoid:** Mount `/var/log` and `/tmp` as tmpfs, write to memory buffer and flush every 5 minutes, use SQLite with WAL mode, implement shutdown button, disable swap.

4. **Touchscreen unresponsiveness** — 7-inch display has 33Hz polling rate, causing lag. **Avoid:** Set `gpu_mem=128`, design 64px touch targets, provide immediate visual feedback, avoid rapid-fire interactions.

5. **Network transmission failures silent** — FTP/SFTP fails but only logged, not surfaced to UI. **Avoid:** Implement retry queue with exponential backoff, verify server receipt, show transmission status prominently, dead-letter alert after N failures.

6. **Stale data without notification** — Dashboard shows old readings with no indication. **Avoid:** Always display "last updated" timestamp, use Socket.IO for live updates, add backend health check, show warning if sensor reader hasn't updated in X minutes.

---

## Implications for Roadmap

Based on research, suggested phase structure:

### Phase 1: Foundation & Kiosk UI
**Rationale:** Must prove sensor-to-display works before adding persistence or networking. This is a vertical slice — each layer works end-to-end before adding features.

**Delivers:** 
- Hardware interface (GPIO/ADC)
- Sensor service with live readings
- Basic API (status, sensors endpoints)
- Minimal dashboard UI (rainfall, voltage, time, station ID)
- Port 8080 kiosk server with Chromium autostart

**Addresses:** FEATURES table stakes — dashboard display, voltage monitoring, date/time, station ID, main menu

**Avoids:** Pitfall 1 (Flask performance) — use eventlet from start, not as afterthought; Pitfall 4 (touchscreen lag) — test touch responsiveness early; Pitfall 6 (stale data) — show timestamps from day one

**Research flag:** Complex integration with sensor hardware — may need `/gsd-research-phase` for GPIO/ADC specifics if not already available from existing RTU.

### Phase 2: Data Persistence
**Rationale:** CSV logging is the primary data workflow. Must establish proper storage before transmission. Avoids Pitfall 3 (SD card corruption) by designing correctly from the start.

**Delivers:**
- Data logger service with buffered writes
- SQLite or structured CSV storage
- tmpfs mounts for logs/tmp
- Settings UI and configuration persistence
- Calibration view UI

**Addresses:** FEATURES settings configuration, calibration view

**Avoids:** Pitfall 3 (SD card corruption) — implement tmpfs, buffered writes, proper shutdown

**Research flag:** Standard patterns — SQLite and CSV logging are well-documented. Skip deep research.

### Phase 3: Network Transmission
**Rationale:** Data must reach myvscada server. Requires proven persistence layer first. Avoids Pitfall 5 (silent transmission failures) by building verification upfront.

**Delivers:**
- Network service with FTP/SFTP/SCP
- Transmission queue with retry logic
- Server receipt verification
- Transmission status UI (last sync, pending count)
- File management UI

**Addresses:** FEATURES file management, transmission status

**Avoids:** Pitfall 5 (silent failures) — queue with backoff, verification, UI visibility

**Research flag:** May need deeper research on myvscada server integration specifics (API, expected file format).

### Phase 4: Remote Access
**Rationale:** Full HD remote interface is the "pro" experience. Can reuse most backend. Focus on responsive design and authentication.

**Delivers:**
- Port 9090 server with authentication
- Responsive templates for Full HD
- Historical graphs (Chart.js)
- Configurable alarm thresholds with visual indicators

**Addresses:** FEATURES historical graphs, threshold indicators, alarm threshold config, dual-mode display

**Research flag:** Standard Flask + Bootstrap patterns — skip deep research.

### Phase Ordering Rationale

- **Vertical slice first:** Each layer works end-to-end before adding complexity
- **Data before network:** Can't transmit what hasn't been stored correctly
- **Kiosk before remote:** Local display is primary use case; remote is secondary
- **Pitfall prevention:** Each phase addresses specific pitfalls identified in research

### Research Flags

Phases likely needing deeper research during planning:
- **Phase 1:** Sensor hardware integration — need specifics on existing RTU GPIO/ADC interface
- **Phase 3:** myvscada server API — format expectations, authentication, transmission protocol

Phases with standard patterns (skip research-phase):
- **Phase 2:** SQLite/CSV logging is well-documented
- **Phase 4:** Flask + Bootstrap responsive design is standard

---

## Confidence Assessment

| Area | Confidence | Notes |
|------|------------|-------|
| Stack | HIGH | Verified with multiple GitHub projects using exact Flask+SocketIO+Chart.js pattern on Pi |
| Features | HIGH | Based on competitive analysis of Rainwise, Weather Display, Davis WeatherLink |
| Architecture | HIGH | Matches proven IoT/SCADA three-tier patterns, clear component boundaries |
| Pitfalls | HIGH | Documented from Raspberry Pi forums, Stack Exchange, community reports |

**Overall confidence:** HIGH

### Gaps to Address

- **Sensor hardware specifics:** Research assumes GPIO/ADC interface available. Need to verify existing RTU sensor connection method.
- **myvscada integration:** Research assumes FTP/SFTP transmission but need to verify server requirements (path, authentication, file format).
- **Touchscreen calibration:** 33Hz polling rate is documented but may need tuning per unit.

---

## Sources

### Primary (HIGH confidence)
- GitHub g1forfun/Pi-Monitor — Active project (Feb 2026), Flask+SocketIO+Chart.js pattern
- Cytron.io tutorial — "Create a Real-Time Raspberry Pi Dashboard" (Dec 2025)
- Flask-SocketIO PyPI — Verified 5.6.0 release (Dec 2025), Python 3.8+ compatibility
- piwheels.org — Verified packages available for Raspberry Pi OS

### Secondary (MEDIUM confidence)
- Raspberry Pi Forums — Kiosk mode issues, performance discussions
- Raspberry Pi Stack Exchange — Flask on Pi Zero W performance
- Multiple community tutorials — Confirms Flask is de facto standard

### Tertiary (LOW confidence)
- GitHub Issue #3777 (raspberrypi/linux) — 7" touchscreen 33Hz polling (needs verification)
- Community reports on SD card corruption — patterns consistent but may vary by card quality

---

*Research completed: 2026-03-12*
*Ready for roadmap: yes*