From 6863f39a249c6e5260e25b4ef79f687c7f9ad77f Mon Sep 17 00:00:00 2001 From: root Date: Thu, 21 May 2026 02:59:32 +0800 Subject: [PATCH] docs: rewrite all documentation to reflect current state - Remove adminer references (service was removed) - Remove mermaid diagrams (ASCII only) - Remove hardcoded credentials (use env var references) - Update all Docker references to 4-container setup (app, postgres, web, pgadmin) - Document env-based admin credentials (ADMIN_EMAIL/ADMIN_PASSWORD) - Document parameterized queries (SQL injection fixed) - Document FCM topic routing by stationtype+level - Document siren stationtype=3 fix in sidesdecode.py - Document idempotent seeder (firstOrCreate) - Document reverse proxy setup in deployment guide - Remove Makefile references (Docker Compose only) --- docs/01-OVERVIEW.md | 154 ++++++++++----- docs/02-ARCHITECTURE.md | 401 +++++++++++++++++++++++++-------------- docs/04-DATA-PIPELINE.md | 250 +++++++++++++++++------- docs/05-FEATURES.md | 265 +++++++++++--------------- docs/06-DATABASE.md | 357 +++++++++++++++++++++++----------- docs/07-API.md | 115 +++++++---- docs/08-CONFIGURATION.md | 232 +++++++++++++--------- 7 files changed, 1116 insertions(+), 658 deletions(-) diff --git a/docs/01-OVERVIEW.md b/docs/01-OVERVIEW.md index 5a4b8828..6d4f0c1c 100644 --- a/docs/01-OVERVIEW.md +++ b/docs/01-OVERVIEW.md @@ -1,71 +1,123 @@ + + # SIDES - Project Overview -## What Is SIDES? +## What Is SIDES -**Sabo Integrated Debris Flow Monitoring and Early Warning System (SIDES)** is a flood and debris flow early warning system for **Sungai Kupang**, Malaysia. It is operated under **Jabatan Pengairan dan Saliran (JPS)** — the Malaysian Department of Irrigation and Drainage. +SIDES (Sabo Integrated Debris Flow Monitoring and Early Warning System) is a web application that monitors rainfall and water level data from sensor stations in the Sungai Kupang area, Kedah, Malaysia. It is operated by JPS (Jabatan Pengairan dan Saliran / Department of Irrigation and Drainage). The system ingests real-time telemetry data, displays it on a map dashboard, and pushes Firebase Cloud Messaging (FCM) alerts when readings cross warning or danger thresholds. -The system monitors rainfall, water levels, and siren activations at remote telemetry stations and presents real-time data on a web dashboard with an interactive map. +## Technology Stack -## Purpose +| Layer | Technology | +|--------------|--------------------------------| +| Backend | Laravel 12 on PHP 8.2 | +| Database | PostgreSQL 16 | +| Frontend | Blade templates + Vite build | +| Auth | Laravel Breeze (session-based) | +| Notifications| Firebase Cloud Messaging (FCM) | +| Deployment | Docker Compose | +| PDF Export | barryvdh/laravel-dompdf | +| Excel Export | maatwebsite/excel | -SIDES provides: +## Architecture -- **Real-time monitoring** of hydrological and meteorological data from field stations -- **Early warning alerts** when rainfall or water levels exceed danger thresholds -- **Siren status tracking** for stations equipped with warning sirens -- **Historical data analysis** with exportable reports (Excel, PDF) -- **Push notifications** to mobile devices via Firebase Cloud Messaging (FCM) -- **Admin management** of stations, users, and access control +``` + Internet + | + Reverse Proxy + (TLS termination) + | + +--------------+--------------+ + | Docker Network | + | | + +----+----+ +---------+ +-------+-----+ +---------+ + | nginx | | php-fpm | | PostgreSQL | | pgAdmin | + | :80 +--->| :9000 +--->| :5432 | | :5050 | + +---------+ +----+----+ +-------+------+ +---------+ + | ^ + v | + Laravel 12 +-----+------+ + Application | pgdata | + | | volume | + +----+----+ +------------+ + | FCM | + | API | + +---------+ +``` -## Domain Context +All four services run as Docker containers on a single host: -- **Location**: Sungai Kupang, Malaysia (Kedah region based on station IDs) -- **Operator**: JPS (Department of Irrigation and Drainage), Malaysia -- **Data source**: Remote telemetry stations transmit CSV data via FTP to a central server -- **Data pipeline**: Python script (`autoscript/sidesdecode.py`) fetches FTP files, parses CSV, inserts into PostgreSQL, and triggers push notifications +| Container | Image | Purpose | +|-------------|-------------------------|--------------------------------------| +| tckdev-app | php:8.2-fpm (custom) | Laravel application via PHP-FPM | +| tckdev-web | nginx:stable-alpine | Serves static assets, proxies PHP | +| tckdev-db | postgres:16 | Primary database | +| tckdev-pgAdmin | dpage/pgadmin4 | Database administration UI | + +A reverse proxy (outside Docker) terminates TLS and forwards traffic to the nginx container on port 80. The production URL is `https://sides.tck.com.my`. + +## Data Pipeline + +1. A Python script running outside Docker fetches CSV files from an external FTP server. +2. The script parses rainfall and water level readings and inserts them into the PostgreSQL database. +3. When a reading crosses a threshold, the script calls the `/api/alert` endpoint. +4. `AlertController` resolves the appropriate FCM topic (e.g., `rainfall_danger`, `waterlevel_alert`) and pushes a notification to subscribed devices via `FcmService`. + +FCM topics are configured through environment variables: + +- `FCM_TOPIC_RAINFALL_WARNING` +- `FCM_TOPIC_RAINFALL_DANGER` +- `FCM_TOPIC_WATERLEVEL_ALERT` +- `FCM_TOPIC_WATERLEVEL_DANGER` + +## Core Data Model + +| Table | Purpose | +|---------------|------------------------------------------------------------| +| `station` | Sensor stations (ID, name, district, coordinates, type flags) | +| `rainfall` | Rainfall readings per station with timestamp, hourly/daily/cumulative values | +| `waterlevel` | Water level readings per station with alert/warning/danger thresholds | +| `notification`| Alert history per station, type, and severity level | +| `siren` | Siren activation events linked to stations | +| `users` | User accounts with role-based access levels | ## User Roles -| Role | Access Level | Description | -|------|-------------|-------------| -| **Public** | Unauthenticated | View dashboard map with station data | -| **User** | `access_level = 2` | View all monitoring pages, historical data, export reports | -| **Admin** | `access_level = 1` | Full access including station management and user management | +| Role | access_level | Access | +|--------|-------------|---------------------------------------------------------| +| Admin | 1 | Station management, user management, all data views | +| User | 2 | Rainfall/water level/siren/notification views, profile | +| Public | (none) | Public dashboard and station map only | -## High-Level Architecture +Admin credentials are set through environment variables (`ADMIN_EMAIL`, `ADMIN_PASSWORD`) and seeded on first deployment via `DatabaseSeeder`. -``` -[Telemetry Stations] --CSV/FTP--> [FTP Server (myvscada.com)] - | - [Python Script (cron)] - | - v -[PostgreSQL Database] <--data-- [sidesdecode.py] --alert--> [Laravel API] - | - v - [Firebase FCM] - | - v - [Mobile Devices] - -[Laravel Web App] <--reads-- [PostgreSQL Database] - | - v -[Web Browser] <--Leaflet Map, Charts, Tables--> -``` +## Localization -## Application Name +The application supports English (`en`) and Bahasa Malaysia (`bm`). Language files are stored in `src/lang/en/` and `src/lang/bm/`. Users can switch language through the `/locale/{locale}` route. -The application is registered as **SIDES** in Laravel config (`APP_NAME=SIDES`). +## Key Application Features -## Bilingual Support +- **Map Dashboard** -- Live station data (rainfall, water level, siren status) displayed on an interactive map. +- **Rainfall Monitoring** -- Current readings, historical data, hourly graphs, and IDF threshold analysis with Excel export. +- **Water Level Monitoring** -- Current readings, historical data, graphs, and Excel export. +- **Siren Management** -- Siren activation tracking with PDF export of siren history. +- **Notification Center** -- Separate views for rainfall, water level, and siren alerts with PDF export. +- **Station Management** -- Admin CRUD for sensor stations. +- **User Management** -- Admin CRUD for user accounts. +- **CCTV Integration** -- CCTV feed links per station. -The application supports: -- **English** (en) — default -- **Bahasa Malaysia** (bm) +## API Endpoints -Language files are stored in `src/lang/{en,bm}/` and toggled via `LocaleController` with session-based persistence. +The application exposes a REST API (prefix `/api/`) consumed by the data pipeline and external integrations: -## Project Name Origin - -The repository is named `tckdev` (likely **TCK Development**), referencing the organization running the SIDES system. +| Method | Path | Description | +|--------|-----------------------------|------------------------------------------| +| GET | /api/station/current | Latest readings across all stations | +| GET | /api/station/rainfall | Current rainfall data | +| GET | /api/station/waterlevel | Current water level data | +| GET | /api/station/notification | Today's notifications | +| GET | /api/station/history | Notification history (last 3 days) | +| GET | /api/station/siren | Current siren status | +| GET | /api/station/siren/history | Siren history (last 3 days) | +| POST | /api/login | API authentication | +| POST | /api/alert | Trigger FCM push notification | diff --git a/docs/02-ARCHITECTURE.md b/docs/02-ARCHITECTURE.md index 2eaf62b1..38784723 100644 --- a/docs/02-ARCHITECTURE.md +++ b/docs/02-ARCHITECTURE.md @@ -1,165 +1,286 @@ + + # Architecture -## Technology Stack +## System Overview -| Layer | Technology | Version | -|-------|-----------|---------| -| Backend | Laravel (PHP) | 12.x | -| PHP | PHP-FPM | 8.2 | -| Frontend | Blade Templates + Alpine.js | 3.x | -| CSS | Tailwind CSS + Bootstrap 5.3 | 3.x / 5.3 | -| Charts | Custom JS (Chart.js via `graph.js`) | — | -| Maps | Leaflet.js | 1.9.4 | -| Build Tool | Vite | 7.x | -| Database | PostgreSQL | 15 | -| Cache/Session | Database (via Laravel) | — | -| Queue | Database (via Laravel) | — | -| Containerization | Docker + Docker Compose | 3.9 | -| Web Server | Nginx | stable-alpine | -| PDF Generation | barryvdh/laravel-dompdf | 3.1 | -| Excel Export | maatwebsite/excel | 3.1 | -| Push Notifications | Firebase Cloud Messaging (FCM) via Google Auth | 1.49 | -| Date Picker | Flatpickr | 4.6 | -| Auth Scaffold | Laravel Breeze | 2.3 | -| Data Pipeline | Python (psycopg2, ftplib) | 3.x | +SIDES (Sistem Informasi Data dan Early Warning System) is a Laravel 12 web application that collects, stores, and displays real-time environmental monitoring data from rainfall, water level, and siren stations. It provides a dashboard with a map view of station status, historical data with export capabilities (Excel, PDF), and push notifications via Firebase Cloud Messaging (FCM). The system runs entirely in Docker containers using nginx as a reverse proxy, PHP-FPM for application code, and PostgreSQL 16 for persistence. External IoT stations submit readings through a public REST API; authenticated admin users manage stations and user accounts through a Blade-based web UI. ## Container Architecture -The application runs in 5 Docker containers defined in `docker-compose.yml`: - ``` -┌──────────────────────────────────────────────────────────┐ -│ Docker Network: tckdev_net │ -│ │ -│ ┌─────────┐ ┌─────────┐ ┌──────────┐ │ -│ │ web │ │ app │ │ postgres │ │ -│ │ (nginx) │──>│ (PHP-FPM│──>│ (DB) │ │ -│ │ :80 │ │ :9000) │ │ :5432 │ │ -│ └─────────┘ └─────────┘ └──────────┘ │ -│ │ -│ ┌─────────┐ ┌─────────┐ │ -│ │ pgAdmin │ │ adminer │ │ -│ │ :5050 │ │ :6060 │ (DB management tools) │ -│ └─────────┘ └─────────┘ │ -└──────────────────────────────────────────────────────────┘ + +------------------+ + | User Browser | + +--------+---------+ + | + | :80 + v + +--------+---------+ + | web (nginx) | + | nginx:stable- | + | alpine | + +--------+---------+ + | + fastcgi :9000 + | + +--------+---------+ + ./src -----> | app (php-fpm) | + (bind mount) | php:8.2-fpm | + +--------+---------+ + | + pgsql :5432 + | + +--------+---------+ +------------------+ + | postgres | | pgadmin | + | postgres:16 | | dpage/pgadmin4 | + +------------------+ +------------------+ + volume: pgdata volume: pgadmin_data ``` -### Container Details +### Container Summary -| Container | Image | Port | Purpose | -|-----------|-------|------|---------| -| `tckdev-app` | Custom (PHP 8.2-FPM + Composer 2.3 + Node.js) | 9000 (internal) | Laravel application | -| `tckdev-web` | nginx:stable-alpine | 80 | Reverse proxy & static files | -| `tckdev-db` | postgres:15 | 5432 | PostgreSQL database | -| `tckdev-pgAdmin` | dpage/pgadmin4 | 5050 | Database management UI | -| `tckdev-adminer` | adminer | 6060 | Lightweight DB management UI | +| Container | Image | Port | Purpose | +|------------|---------------------|--------|--------------------------------------------| +| tckdev-app | php:8.2-fpm (custom)| 9000 | Laravel application (PHP-FPM) | +| tckdev-db | postgres:16 | 5432 | PostgreSQL database | +| tckdev-web | nginx:stable-alpine | 80 | Reverse proxy, serves static assets | +| tckdev-pgAdmin | dpage/pgadmin4 | 5050 | Database administration UI | + +### Networking and Volumes + +- All containers share the `tckdev_net` bridge network. +- `src/` is bind-mounted into both `app` and `web` at `/var/www/html`, so code changes are reflected without rebuilding. +- `pgdata` and `pgadmin_data` are Docker named volumes -- not bind mounts. +- The nginx config is bind-mounted from `docker/nginx/default.conf`. +- `docker/backup` is bind-mounted into pgadmin at `/backups`. + +### Entrypoint Behaviour + +The `app` container runs `docker/entrypoint.sh` before starting PHP-FPM. This script: + +1. Copies `.env.example` to `.env` if no `.env` exists. +2. Runs `composer install` if `vendor/` is missing. +3. Runs `npm install` if `node_modules/` is missing. +4. Runs `npm run build` (Vite) if `public/build/` is missing. +5. Runs `php artisan migrate --force` when `RUN_MIGRATIONS=true`. +6. Runs `php artisan db:seed --force` when `RUN_SEEDER=true`. +7. Caches config, routes, and views for production performance. ## Request Flow -``` -Browser --> Nginx (:80) --> PHP-FPM (:9000) --> Laravel Router - | - ┌─────────────┼─────────────────┐ - v v v - Web Routes API Routes Auth Routes - (web.php) (api.php) (auth.php) - | | | - v v v - Controllers Api/Controllers Breeze Controllers - | | - v v - DB::raw() SQL DB::raw() SQL - | | - v v - PostgreSQL PostgreSQL - | - v - Blade Views --> HTML Response -``` - -## Middleware Stack - -| Middleware | Route Group | Purpose | -|-----------|------------|---------| -| `auth` | Web (protected) | Require authenticated session | -| `admin` | Admin routes | Require `access_level = 1` | -| `LocalizationMiddleware` | All web routes | Set locale from session (`en`/`bm`) | -| `guest` | Auth routes (login/register) | Redirect away if already authenticated | - -## Database Design - -### Core Tables +### Web UI Request (Blade pages) ``` -┌─────────────┐ ┌──────────────┐ -│ station │ │ users │ -│─────────────│ │──────────────│ -│ stationid* │ │ id* │ -│ name │ │ name │ -│ district │ │ email │ -│ lng, lat │ │ password │ -│ mainriverbasin│ │ access_level │ -│ subriverbasin│ │ is_blocked │ -│ rainfall │ │ login_attempts│ -│ waterlevel │ └──────────────┘ -│ siren │ -│ cctv_link │ -└──────┬──────┘ - │ - │ (stationid FK via application logic, not DB constraint) - │ - ┌────┼────────────┬──────────────┐ - v v v v -┌──────────┐ ┌────────────┐ ┌───────────┐ -│ rainfall │ │ waterlevel │ │ siren │ -│──────────│ │────────────│ │───────────│ -│ id* │ │ id* │ │ id* │ -│ stationid│ │ stationid │ │ stationid │ -│ timestamp│ │ datetime │ │ stationtype│ -│ anncum │ │ waterlevel │ │ active_time│ -│ daily │ │ alert │ │ level │ -│ hourly │ │ warning │ └───────────┘ -│ currentrf│ │ danger │ -│ battery │ └────────────┘ -└──────────┘ - │ - v -┌──────────────┐ -│ notification │ -│──────────────│ -│ id* │ -│ stationid │ -│ timestamp │ -│ stationtype │ (1=rainfall, 2=waterlevel, 3=siren) -│ level │ (Normal, Alert, Warning, Danger) -│ active_time │ -└──────────────┘ +Browser --:80--> nginx + | + +-- static asset? --> /var/www/html/public/{file} + | + +-- *.php? ---------> fastcgi://app:9000 + | + Laravel Router + | + +-- auth middleware --> Breeze session auth + +-- admin middleware --> access_level === 1 check + | + Controller + | + DB::table() / DB::select() + | + v + PostgreSQL + | + Blade view / JSON response ``` -### Station Types +1. Browser hits port 80 on the `web` container. +2. nginx serves static files from `/var/www/html/public` directly. +3. PHP requests are forwarded via FastCGI to `app:9000`. +4. Laravel routes the request through middleware (`auth`, `admin`, or none). +5. Controllers query PostgreSQL using `DB::table()` or `DB::select()` with parameterized bindings. +6. Responses are rendered Blade views (HTML) or JSON for AJAX endpoints. -The `station` table uses boolean flags to indicate which monitoring types each station supports: +### API Request (IoT station data) -- `rainfall = 1` — Station has rainfall monitoring -- `waterlevel = 1` — Station has water level monitoring -- `siren = 1` — Station has a warning siren +``` +IoT Device --:80/api/...--> nginx ---> app:9000 + | + Laravel API Router (no auth middleware on data endpoints) + | + Controller (Api\*) + | + DB::table() / DB::select() + | + v + PostgreSQL +``` -### Foreign Key Note +API routes under `/api/` have no middleware. The `POST /api/login` endpoint validates credentials against the `users` table using `DB::select()` with bound parameters and `Hash::check()`. There are no API tokens -- authentication is a single request/response returning user data on success. -There are **no database-level foreign keys** between `station` and the data tables. Relationships are maintained at the application level via `stationid` column matching. +## Data Model -### Laravel Standard Tables +The application uses six database tables. There are no foreign key constraints; relationships are enforced at the application level. `User` is the only Eloquent model. All other data access uses the `DB` query builder with parameterized queries. -The application also uses standard Laravel tables: -- `users` — Authentication and authorization -- `password_reset_tokens` — Password reset flow -- `sessions` — Database-backed sessions -- `cache`, `cache_locks` — Database cache store -- `jobs`, `job_batches`, `failed_jobs` — Database queue +``` ++----------------+ +----------------+ +----------------+ +| station | | rainfall | | waterlevel | +|----------------| |----------------| |----------------| +| stationid (PK) |<--+---| id (PK) | +----| id (PK) | +| name | | | stationid | | | stationid | +| district | | | timestamp | | | datetime | +| lng | | | anncum | | | waterlevel | +| lat | | | daily | | | alert | +| mainriverbasin | | | hourly | | | warning | +| subriverbasin | | | currentrf | | | danger | +| rainfall (flag)| | | battery | | +----------------+ +| waterlevel (f) | | +----------------+ | +| siren (flag) | | | +| cctv_link | | +----------------+ | +----------------+ ++----------------+ | | siren | | | notification | + +---| id (PK) | +---| id (PK) | + | stationid | | stationid | + | stationtype | | timestamp | + | active_time | | stationtype | + | level | | level | + +----------------+ | active_time | + +----------------+ ++----------------+ +| users | +|----------------| +| id (PK) | +| name | +| email | +| access_level | +| password | +| login_attempts | +| is_blocked | ++----------------+ +``` -## Authentication & Authorization +### Table Descriptions -- **Web auth**: Laravel Breeze (session-based, Blade views) -- **API auth**: Custom token-less login via `Api\AuthController` (returns user info, no token generation) -- **Admin access**: Controlled by `AdminMiddleware` checking `access_level === 1` -- **User blocking**: Users can be blocked via `is_blocked` flag (though no middleware enforces it currently) +| Table | Purpose | +|---------------|--------------------------------------------------------| +| `station` | Master list of monitoring stations with location and capability flags | +| `rainfall` | Timestamped rainfall readings per station | +| `waterlevel` | Timestamped water level readings with alert/warning/danger thresholds | +| `siren` | Siren activation events with level (N/D/W/A) | +| `notification`| Notification log entries linking stations to alert levels | +| `users` | Application users; `access_level=1` for admin | + +## Application Layer + +### Controllers + +**Web controllers** (render Blade views): + +| Controller | Responsibility | +|-------------------------|-------------------------------------------------------| +| `MapController` | Dashboard/map view; joins station with latest readings | +| `RainfallController` | Rainfall display, graphs, historical data, Excel export | +| `WaterLevelController` | Water level display, graphs, historical data, Excel export | +| `SirenController` | Siren status display, history, PDF export | +| `NotificationController`| Rainfall/water level/siren notification views and PDF exports | +| `AdminController` | Station and user CRUD (admin-only) | +| `cctvController` | CCTV feed display | +| `LocaleController` | Language switching | +| `ProfileController` | User profile management (Breeze) | +| Auth controllers | Login, register, password reset (Breeze scaffolding) | + +**API controllers** (return JSON): + +| Controller | Responsibility | +|-------------------------|-------------------------------------------------------| +| `Api\StationController` | Current data, rainfall, water level, notifications, siren status for IoT/external consumption | +| `Api\AuthController` | Token-less login; validates username/password, returns user JSON | +| `Api\AlertController` | Routes FCM push notifications to the correct topic based on stationtype and level | + +### Middleware + +| Middleware | Behavior | +|-------------------------|-------------------------------------------------------| +| `auth` | Redirects unauthenticated users to login (Breeze) | +| `admin` | Checks `access_level === 1`; redirects non-admins to dashboard | +| `LocalizationMiddleware`| Sets application locale from session | + +### Services + +| Service | Responsibility | +|---------------|--------------------------------------------------------------| +| `FcmService` | Obtains Google OAuth2 access token from Firebase service account credentials, sends FCM v1 API messages to topics | + +### FCM Topic Routing + +`AlertController::send()` determines the FCM topic from the request's `stationtype` and `level`: + +| stationtype | Level | FCM Topic | +|-------------|----------|---------------------------| +| 1 | Warning | `FCM_TOPIC_RAINFALL_WARNING` | +| 1 | Danger | `FCM_TOPIC_RAINFALL_DANGER` | +| 2 | Warning | `FCM_TOPIC_WATERLEVEL_ALERT` | +| 2 | Danger | `FCM_TOPIC_WATERLEVEL_DANGER` | +| 3 | (any) | `FCM_TOPIC_RAINFALL_WARNING` | + +## Authentication + +Two separate authentication mechanisms coexist: + +- **Web UI**: Laravel Breeze provides session-based authentication (login, register, password reset, email verification). The `auth` middleware guards protected routes. +- **API**: `POST /api/login` accepts `username` and `password`, validates against the `users` table, and returns user data as JSON. There is no token or session persisted for API consumers -- each API data endpoint is stateless and unauthenticated. + +## Frontend Stack + +- Blade templates with Tailwind CSS (via `tailwind.config.js`). +- Vite handles asset bundling (`vite.config.js`, `postcss.config.js`). +- Map rendering uses Leaflet with station markers. +- Client-side JavaScript in `public/js/` handles graph rendering and AJAX calls. + +## Database Configuration + +- **Connection**: `pgsql` (PostgreSQL 16) -- configured in `.env` as `DB_CONNECTION=pgsql`. +- **Host**: `postgres` (Docker service name). +- **Session driver**: `database` (Laravel's database session storage). +- **Cache store**: `database`. +- **Queue connection**: `database`. + +All session, cache, and queue data lives in PostgreSQL tables created by Laravel's default migrations (`cache`, `jobs`, `sessions` tables). + +## Directory Structure + +``` +tckdev/ + docker/ Docker configuration + entrypoint.sh App container startup script + nginx/ + default.conf nginx reverse proxy config + src/ Laravel application (bind-mounted into containers) + app/ + Http/ + Controllers/ Web controllers + Controllers/Api/ API controllers (StationController, AuthController, AlertController) + Middleware/ AdminMiddleware, LocalizationMiddleware + Requests/ Form request validation + Exports/ Maatwebsite Excel export classes + Models/ User (only Eloquent model) + Notifications/ ResetPasswordNotification + Services/ FcmService + Providers/ AppServiceProvider + View/Components/ AppLayout, GuestLayout (Blade components) + bootstrap/ + config/ Laravel config files + database/ + migrations/ Schema migrations (13 files) + seeders/ Database seeders + factories/ Model factories + public/ Document root (served by nginx) + resources/ + views/ Blade templates + routes/ + web.php Web routes (authenticated + admin + public) + api.php API routes (stateless, no auth middleware) + auth.php Breeze auth routes + console.php Artisan commands + storage/ App storage (logs, firebase credentials, uploads) + tests/ PHPUnit tests + docker-compose.yml Service definitions (Compose v2, no version key) + Dockerfile Custom php:8.2-fpm image +``` diff --git a/docs/04-DATA-PIPELINE.md b/docs/04-DATA-PIPELINE.md index 05af5470..c18ed3d6 100644 --- a/docs/04-DATA-PIPELINE.md +++ b/docs/04-DATA-PIPELINE.md @@ -1,48 +1,62 @@ + # Data Pipeline: Python Autoscript ## Overview -The file `autoscript/sidesdecode.py` is the data ingestion pipeline that: +`autoscript/sidesdecode.py` is the data ingestion pipeline that: 1. Connects to an FTP server where telemetry stations upload CSV data 2. Downloads and parses CSV files for the current day 3. Inserts rainfall, water level, and siren data into PostgreSQL 4. Triggers push notifications when thresholds are exceeded -## How It Runs +The script is designed to run on a schedule (e.g., cron job), processing new files uploaded by remote telemetry stations throughout the day. -The script is designed to be run on a **schedule** (likely cron job), processing new data files uploaded by remote telemetry stations throughout the day. +## Environment Variables + +All credentials come from environment variables with defaults: + +| Variable | Default | Description | +|----------|---------|-------------| +| `FTP_SERVER` | `myvscada.com` | FTP server hostname | +| `FTP_USERNAME` | `tck` | FTP login username | +| `FTP_PASSWORD` | *(empty)* | FTP login password | +| `PG_HOST` | `postgres` | PostgreSQL host (`postgres` on Docker network, `localhost` on Docker host) | +| `PG_DATABASE` | `sides_db` | PostgreSQL database name | +| `PG_USER` | `tck` | PostgreSQL username | +| `PG_PASSWORD` | *(empty)* | PostgreSQL password | ## FTP Connection +The script connects to the configured FTP server and navigates to today's date folder: + ``` -Server: myvscada.com -Username: tck -Password: tck6789 Path: files/SIDES/SUCCESS/{year}/{month}/{day}/ ``` -The script navigates to today's date folder and lists all files. +Example path for 21 May 2026: `files/SIDES/SUCCESS/2026/05/21/` ### File Filtering - Skips files containing "rf" in the filename (Tideda format files) -- Only processes files with today's date (`yymmdd` format) in the filename +- Only processes files with today's date in `yymmdd` format in the filename ## CSV Format -Each line in the CSV file contains 37+ comma-separated columns. Key columns extracted: +Each line in the CSV file contains 37+ comma-separated columns. The script requires at least 25 columns per line; shorter lines are skipped. -| Column Index | Field | Description | -|-------------|-------|-------------| -| 1 | `station_id` | Station identifier (e.g., KBLG0026) | +Key columns extracted: + +| Column Index | Variable | Description | +|-------------|----------|-------------| +| 1 | `station_id` | Station identifier (e.g., `KBLG0026`) | | 4 | `timestamp` | Timestamp in `yymmddHHMMSS` format | | 6 | `battery` | Battery voltage | | 15 | `wlalert` | Water level alert threshold | | 16 | `wlwarn` | Water level warning threshold | | 17 | `wldgr` | Water level danger threshold | | 18 | `sirenid` | Siren identifier | -| 19 | `siren` | Siren status (`H`=Danger/High, `L`=Warning/Low, `N`=Normal) | +| 19 | `siren` | Siren status: `H`=Danger, `L`=Warning, `N`=Normal | | 21 | `anncumm` | Annual cumulative rainfall | | 22 | `dailycumm` | Daily cumulative rainfall | | 23 | `hourlycumm` | Hourly rainfall | @@ -51,39 +65,129 @@ Each line in the CSV file contains 37+ comma-separated columns. Key columns extr ## Data Processing Logic +The `process_line()` function handles each CSV line. All database operations use `psycopg2` parameterized queries to prevent SQL injection. + ### Rainfall Data -1. Check if `dailycumm` or `hourlycumm` is not null -2. Check if record already exists for this station+timestamp -3. If new, INSERT into `rainfall` table -4. **Threshold check**: If `hourlycumm >= 30`: - - `30 <= hourly < 60` → **Warning** level - - `hourly >= 60` → **Danger** level - - INSERT into `notification` table - - Send push notification via Laravel API +``` +CSV line + | + v +[dailycumm or hourlycumm not null?] + |-- No --> skip + |-- Yes + v + [record exists for station+timestamp?] + |-- Yes --> skip insert + |-- No --> INSERT INTO rainfall (stationid, timestamp, anncum, daily, hourly, currentrf, battery) + | + v + [hourlycumm >= 30?] + |-- No --> done + |-- Yes + | + v + Determine level: + 30 <= hourly < 60 --> Warning + hourly >= 60 --> Danger + | + v + [notification exists for station+timestamp+stationtype=1?] + |-- Yes --> skip insert + |-- No --> INSERT INTO notification (stationid, timestamp, stationtype=1, level, active_time) + | + v + send_alert_to_laravel(station_id, level, 1) +``` ### Water Level Data -1. Check if `waterlevel` is not null -2. Check if record already exists for this station+datetime -3. If new, INSERT into `waterlevel` table (with alert/warning/danger thresholds) -4. **Threshold check**: If `waterlevel >= alert`: - - `alert <= wl < warning` → **Alert** level - - `warning <= wl < danger` → **Warning** level - - `wl >= danger` → **Danger** level - - INSERT into `notification` table - - Send push notification via Laravel API +``` +CSV line + | + v +[waterlevel not null?] + |-- No --> skip + |-- Yes + v + [record exists for station+datetime?] + |-- Yes --> skip insert + |-- No --> INSERT INTO waterlevel (stationid, datetime, waterlevel, alert, warning, danger) + | + v + [waterlevel >= wlalert?] + |-- No --> done + |-- Yes + | + v + Determine level: + alert <= wl < warning --> Alert + warning <= wl < danger --> Warning + wl >= danger --> Danger + | + v + [notification exists for station+timestamp+stationtype=2?] + |-- Yes --> skip insert + |-- No --> INSERT INTO notification (stationid, timestamp, stationtype=2, level, active_time) + | + v + send_alert_to_laravel(station_id, level, 2) +``` ### Siren Data -1. Check if `sirenid` is not null -2. Check if record already exists for this station+active_time -3. Determine level from siren status: - - `H` → **Danger** - - `L` → **Warning** - - `N` → **Normal** -4. INSERT into `siren` table -5. If level is not Normal, send push notification via Laravel API +``` +CSV line + | + v +[sirenid not null?] + |-- No --> skip + |-- Yes + v + Determine level from siren status: + H --> Danger + L --> Warning + N --> Normal + | + v + [record exists for station+active_time?] + |-- Yes --> skip insert + |-- No --> INSERT INTO siren (stationid, stationtype=3, active_time, level) + | + v + [level != Normal?] + |-- No --> done + |-- Yes --> send_alert_to_laravel(station_id, level, 3) +``` + +## Station Types + +The `stationtype` integer identifies the data source in notifications and alerts: + +| stationtype | Data Source | +|------------|-------------| +| 1 | Rainfall | +| 2 | Water Level | +| 3 | Siren | + +## Threshold Summary + +### Rainfall Thresholds + +| Condition | Level | +|-----------|-------| +| `hourlycumm >= 30` and `< 60` | Warning | +| `hourlycumm >= 60` | Danger | + +### Water Level Thresholds + +Thresholds are per-station values from CSV columns 15-17. + +| Condition | Level | +|-----------|-------| +| `waterlevel >= wlalert` and `< wlwarn` | Alert | +| `waterlevel >= wlwarn` and `< wldgr` | Warning | +| `waterlevel >= wldgr` | Danger | ## Alert Notification Flow @@ -99,45 +203,57 @@ def send_alert_to_laravel(stationid, level, stationtype): response = requests.post("https://sides.tck.com.my/api/alert", json=payload, timeout=5) ``` -This hits the Laravel `AlertController` which: -1. Builds notification title/body based on station type and level -2. Calls `FcmService::sendToTopic()` which: - - Reads Firebase service account credentials - - Gets an OAuth2 access token from Google - - Sends FCM message to topic (e.g., `rainfall_warning`) - - Push notification arrives on subscribed mobile devices + -## PostgreSQL Connection +The full notification chain: -The script connects directly to PostgreSQL: - -```python -pg_host = "192.168.0.211" -pg_database = "sides_db" -pg_user = "tck" -pg_password = "projectdev##1" +``` +sidesdecode.py + | POST /api/alert {stationid, level, stationtype} + v +AlertController (Laravel) + | Builds notification title/body from station type and level + v +FcmService::sendToTopic() + | Routes to FCM topic by stationtype and level + | (e.g., rainfall_warning, rainfall_danger, waterlevel_alert, waterlevel_danger) + v +Firebase Cloud Messaging + | Push notification delivered to subscribed mobile devices + v +Mobile App ``` -**Note**: This is a hardcoded external IP, not using the Docker container. The database name is `sides_db` (different from the Docker `.env` which uses `tckdev`). +## Deduplication + +Before every INSERT, the script checks for an existing record: + +- **Rainfall**: `SELECT COUNT(*) FROM rainfall WHERE stationid = %s AND timestamp = %s` +- **Water Level**: `SELECT COUNT(*) FROM waterlevel WHERE stationid = %s AND datetime = %s` +- **Siren**: `SELECT COUNT(*) FROM siren WHERE stationid = %s AND active_time = %s` +- **Notification**: `SELECT COUNT(*) FROM notification WHERE stationid = %s AND timestamp = %s AND stationtype = %s` + +If a record exists, the INSERT is skipped. This makes the script safe to re-run for the same time period. ## File Management (Commented Out) -The script contains (commented out) functions for: -- `move_to_error_folder()` — Move malformed files to an FTP error folder -- `move_to_success_folder()` — Move processed files to a success archive folder +Two file management functions are defined but currently commented out: -These are currently disabled — files remain in the source folder after processing. +- `move_to_error_folder()` -- Move malformed files to an FTP error subfolder +- `move_to_success_folder()` -- Move processed files to a success archive subfolder -## Log Files +When active, these functions create the target FTP directory if it does not exist, upload the file, and delete the original. Currently disabled -- processed files remain in the source FTP folder after processing. -- `autoscript/sidesdecode.log` — Processing output -- `autoscript/sidesdecode_error.log` — Error output +## Error Handling + +- Malformed lines (fewer than 25 columns) are skipped with a log message +- Any exception during line processing triggers `conn.rollback()` to prevent partial inserts +- Alert sending failures are caught and logged but do not halt processing +- The script closes both FTP and database connections on exit ## Known Issues -1. **Hardcoded credentials** — FTP and PostgreSQL credentials are embedded in the script -2. **No deduplication beyond same-timestamp** — If the script runs twice, it skips exact duplicates but has no broader deduplication -3. **Commented out file management** — Processed files are not moved/archived -4. **Water level alert sends `stationtype=1`** instead of `2` (likely a bug at line 378) -5. **No error recovery** — If the script crashes mid-processing, some data may be partially inserted -6. **No connection pooling** — New FTP and database connections each run +1. **No file archiving** -- `move_to_error_folder` and `move_to_success_folder` are commented out, so files are never moved after processing +2. **No broader deduplication** -- Deduplication only checks exact station+timestamp matches; no handling for near-duplicate records +3. **No connection retry** -- If FTP or PostgreSQL is unreachable, the script fails immediately with no retry logic +4. **Partial processing risk** -- If the script crashes mid-file, lines already processed are committed but remaining lines are lost until the next run diff --git a/docs/05-FEATURES.md b/docs/05-FEATURES.md index 5c889126..08944d99 100644 --- a/docs/05-FEATURES.md +++ b/docs/05-FEATURES.md @@ -1,203 +1,150 @@ + # Features & User Guide ## Feature Overview -SIDES provides the following feature areas: +SIDES provides a web-based dashboard for monitoring rainfall, water levels, and siren activations at telemetry stations in the Sungai Kupang catchment. Features are grouped by access level: public pages (no login), authenticated user pages, admin-only management, and a REST API. -### 1. Public Dashboard (Map View) +--- -**Route**: `/` or `/dashboard` -**Access**: Public (no auth required) -**Controller**: `MapController::getCurrentData()` +## Public Pages -Displays an interactive Leaflet.js map showing all monitoring stations with their current readings: -- Station markers color-coded by status -- Popup showing latest rainfall, water level, and siren data -- Real-time station data from PostgreSQL +These pages are accessible without authentication. -### 2. Rainfall Monitoring +### Dashboard (Map View) -**Route**: `/rainfall` -**Access**: Authenticated users -**Controller**: `RainfallController::index()` +| Route | Description | +|---|---| +| `GET /` or `GET /dashboard` | Interactive map powered by Leaflet.js. Displays all telemetry stations with their latest sensor readings. Served by `MapController`. | +| `GET /stations` | Returns a JSON list of stations and current readings. Used by the map to populate markers. | +| `GET /locale/{locale}` | Switches the application language. Supports `en` (English) and `bm` (Bahasa Malaysia). | -- Table showing rainfall data for all rainfall-enabled stations -- Filter by station and date -- Displays: hourly, daily, and 7-day historical values -- Last data update timestamp +--- -### 3. Rainfall Graph +## Authenticated User Pages -**Route**: `/rainfall/graph/{stationid}` -**Access**: Authenticated users -**Controller**: `RainfallController::rainfallGraph()` +All routes below require a logged-in session (Laravel `auth` middleware). -- Interactive chart showing hourly rainfall for the selected station today -- X-axis: time (HH:MM), Y-axis: rainfall value +### General -### 4. Early Warning Threshold (IDF) +| Route | Description | +|---|---| +| `GET /home` | Home page shown after login. | -**Route**: `/threshold` -**Access**: Authenticated users -**Controller**: `RainfallController::rainfallSum()` +### Rainfall Monitoring -- 6-hour cumulative rainfall data for early warning -- Shows hourly values and timestamps for each of the last 6 hours -- Filter by station and date -- Threshold graph data available via API +| Route | Description | +|---|---| +| `GET /rainfall` | Tabular view of rainfall data with filters for station and date range. | +| `GET /rainfall/graph/{stationid}` | Hourly rainfall chart for a specific station. | +| `GET /rainfall/historical` | Historical hourly rainfall data (24 readings per day) with export option. | +| `GET /rainfall/historical/export` | Exports historical rainfall data to an Excel file. | -### 5. Historical Rainfall +### Water Level Monitoring -**Route**: `/rainfall/historical` -**Access**: Authenticated users -**Controller**: `RainfallController::historicalRainfall()` +| Route | Description | +|---|---| +| `GET /waterlevel` | Current water level readings in a filterable table. | +| `GET /waterlevel/graph/{stationid}` | Water level chart for a specific station. | +| `GET /waterlevel/historical` | Historical water level data with threshold lines and export option. | +| `GET /waterlevel/historical/export` | Exports historical water level data to an Excel file. | -- Detailed hourly rainfall breakdown (24 hours per row) -- Filter by station and date range -- Shows total 24h rainfall per day -- **Export to Excel**: `/rainfall/historical/export` +### Threshold & Early Warning -### 6. Water Level Monitoring +| Route | Description | +|---|---| +| `GET /threshold` | 6-hour cumulative rainfall summary used for early warning (IDF-based). | +| `GET /threshold/graphData/{stationid}/{dates}` | Returns threshold graph data for a station and date range. | -**Route**: `/waterlevel` -**Access**: Authenticated users -**Controller**: `WaterLevelController::index()` +### Siren Status -- Table showing current water level readings -- Displays: water level value, alert/warning/danger thresholds -- Filter by station and date -- Interactive water level graph +| Route | Description | +|---|---| +| `GET /siren` | Current siren status across all stations (last 3 days). | +| `GET /sirenhistory` | Paginated siren history, excluding "Normal" status, with PDF export option. | +| `GET /export/siren-history/pdf` | Exports siren history to a PDF file. | -### 7. Historical Water Level +### Notifications -**Route**: `/waterlevel/historical` -**Access**: Authenticated users -**Controller**: `WaterLevelController::wlHistory()` +| Route | Description | +|---|---| +| `GET /notificationrf` | Today's rainfall notifications. | +| `GET /notificationwl` | Today's water level notifications. | +| `GET /notificationsiren` | Today's siren notifications. | +| `GET /historyrf` | Historical rainfall notifications with PDF export. | +| `GET /historywl` | Historical water level notifications with PDF export. | -- Hourly water level data for a selected station and date -- Shows threshold levels alongside actual readings -- **Export to Excel**: `/waterlevel/historical/export` +### CCTV & Profile -### 8. Siren Monitoring +| Route | Description | +|---|---| +| `GET /cctv` | CCTV camera links for water level stations. | +| `GET /profile` | User profile management (Laravel Breeze). | -**Route**: `/siren` -**Access**: Authenticated users -**Controller**: `SirenController::index()` +--- -- Current siren status for all siren-equipped stations -- Shows last 3 days of siren activations -- Siren levels: Normal (N), Warning (L), Danger (H) +## Admin Pages -### 9. Siren History +These routes require both authentication and admin privileges (`admin` middleware). -**Route**: `/sirenhistory` -**Access**: Authenticated users -**Controller**: `SirenController::SirenHistory()` +### Station Management -- Paginated history of all siren activations (excluding Normal) -- **Export to PDF**: `/export/siren-history/pdf` +| Route | Description | +|---|---| +| `GET /stationmanagement` | List all stations (paginated, 5 per page). | +| `POST /stationmanagement/store` | Add a new station. | +| `POST /stationmanagement/{stationid}/update` | Edit an existing station. | +| `DELETE /stationmanagement/{stationid}/delete` | Delete a station. | -### 10. Notifications +### User Management -**Rainfall Notifications**: `/notificationrf` -**Water Level Notifications**: `/notificationwl` -**Siren Notifications**: `/notificationsiren` -**Access**: Authenticated users -**Controller**: `NotificationController` +| Route | Description | +|---|---| +| `GET /usermgmt` | List all users (paginated, 5 per page). | +| `POST /usermgmt/store` | Create a new user. | +| `POST /usermgmt/{userid}/update` | Edit user details. | +| `POST /usermgmt/{userid}/updatePassword` | Reset a user's password. | +| `DELETE /usermgmt/{userid}/delete` | Delete a user. | -- Shows today's threshold-triggered notifications -- Grouped by type (rainfall, water level, siren) +--- -### 11. Notification History +## Authentication -**Rainfall History**: `/historyrf` -**Water Level History**: `/historywl` -**Access**: Authenticated users -**Controller**: `NotificationController` +Provided by Laravel Breeze with Blade views. -- Paginated history of all notifications -- **Export to PDF**: - - `/export/rainfall-history/pdf` - - `/export/waterlevel-history/pdf` +| Route | Description | +|---|---| +| `GET /login` | Login form. | +| `POST /login` | Authenticate credentials. | +| `GET /register` | Registration form. | +| `POST /register` | Create a new account. | +| `GET /forgot-password` | Request a password reset link. | +| `POST /forgot-password` | Send the reset link email. | +| `GET /reset-password` | Password reset form. | +| `POST /reset-password` | Submit new password. | +| `POST /logout` | End the session. | -### 12. CCTV Links +--- -**Route**: `/cctv` -**Access**: Authenticated users -**Controller**: `cctvController::index()` +## API Endpoints -- Lists stations with CCTV links (where `cctv_link` is not null and `waterlevel = 1`) -- Links open external CCTV feeds +All API endpoints are unauthenticated except `/api/login` which validates credentials. -### 13. Admin: Station Management +### Data Retrieval -**Route**: `/stationmanagement` -**Access**: Admin only (`access_level = 1`) -**Controller**: `AdminController` +| Method | Endpoint | Description | +|---|---|---| +| GET | `/api/station/current` | Current station readings. | +| GET | `/api/station/rainfall` | Rainfall data. | +| GET | `/api/station/waterlevel` | Water level data. | +| GET | `/api/station/notification` | Latest notifications. | +| GET | `/api/station/history` | Historical data. | +| GET | `/api/station/siren` | Current siren status. | +| GET | `/api/station/siren/history` | Siren history. | -- List all stations (paginated, 5 per page) -- Add new station -- Edit station details (name, district, coordinates, type flags, CCTV link) -- Delete station -- Shows station type counts (rainfall, water level, siren) +### Actions -### 14. Admin: User Management - -**Route**: `/usermgmt` -**Access**: Admin only -**Controller**: `AdminController` - -- List all users (paginated, 5 per page) -- Add new user -- Edit user details (name, email, access level, block/unblock) -- Reset user password -- Delete user -- Shows user counts by role - -### 15. API Endpoints - -**Prefix**: `/api/` -**Auth**: Custom login (no token-based auth) - -| Endpoint | Method | Description | -|----------|--------|-------------| -| `/api/station/current` | GET | All stations with current data | -| `/api/station/rainfall` | GET | Current rainfall data per station | -| `/api/station/waterlevel` | GET | Current water level data per station | -| `/api/station/notification` | GET | Today's notifications per station | -| `/api/station/history` | GET | 3-day notification history | -| `/api/station/siren` | GET | Current siren status | -| `/api/station/siren/history` | GET | 3-day siren history | -| `/api/login` | POST | Login (username + password) | -| `/api/alert` | POST | Send FCM push notification | - -### 16. Profile Management - -**Route**: `/profile` -**Access**: Authenticated users -**Controller**: `ProfileController` (Breeze scaffold) - -- Update name and email -- Update password -- Delete account - -### 17. Localization - -**Route**: `/locale/{locale}` -**Access**: Public -**Controller**: `LocaleController` - -- Switch between English (`en`) and Bahasa Malaysia (`bm`) -- Persisted in session -- Applied via `LocalizationMiddleware` on every request - -### 18. Authentication - -**Routes**: `/login`, `/register`, `/forgot-password`, `/reset-password` -**Access**: Guest only (for login/register) -**Scaffold**: Laravel Breeze - -- Email/password authentication -- Password reset via email (Gmail SMTP) -- Email verification -- Session-based auth +| Method | Endpoint | Description | +|---|---|---| +| POST | `/api/login` | Validates user credentials. | +| POST | `/api/alert` | Submits an alert. | diff --git a/docs/06-DATABASE.md b/docs/06-DATABASE.md index 3fc3f611..8eb788b1 100644 --- a/docs/06-DATABASE.md +++ b/docs/06-DATABASE.md @@ -1,131 +1,260 @@ -# Database Schema +# Database -## Tables +SIDES uses **PostgreSQL 16** running in a Docker container. Laravel connects +via `pgsql` driver using the Docker service name as the host. -### `station` +## Connection Details -Primary table storing telemetry station metadata. +| Setting | Value | Source | +|-----------------|------------------------------|---------------------------| +| DB_CONNECTION | `pgsql` | `src/.env` | +| DB_HOST | `postgres` (Docker service) | `src/.env` | +| DB_PORT | `5432` | `src/.env` | +| DB_DATABASE | `${POSTGRES_DB}` | `src/.env` | +| DB_USERNAME | `${POSTGRES_USER}` | `src/.env` | +| DB_PASSWORD | `${POSTGRES_PASSWORD}` | `src/.env` | -| Column | Type | Description | -|--------|------|-------------| -| `stationid` | `varchar` (PK) | Unique station identifier (e.g., "KBLG0026") | -| `name` | `varchar(255)` | Station display name | -| `district` | `varchar(255)` | District location | -| `lng` | `float` | Longitude coordinate | -| `lat` | `float` | Latitude coordinate | -| `mainriverbasin` | `varchar(255)` | Main river basin name | -| `subriverbasin` | `varchar(255)` | Sub river basin name | -| `rainfall` | `integer` | Has rainfall sensor (1=yes, 0=no) | -| `waterlevel` | `integer` | Has water level sensor (1=yes, 0=no) | -| `siren` | `integer` (nullable) | Has siren (1=yes, 0=no) | -| `cctv_link` | `varchar(500)` (nullable) | URL to CCTV feed | +Default database name: **sides_db** (set by `POSTGRES_DB` in the root `.env`). -### `rainfall` +The container (`tckdev-db`) uses a named Docker volume **pgdata** for +persistent storage. Data survives container restarts and rebuilds. -Stores rainfall readings from telemetry stations. - -| Column | Type | Description | -|--------|------|-------------| -| `id` | `bigint` (PK, auto) | Auto-increment ID | -| `stationid` | `varchar` | Station identifier (FK to station) | -| `timestamp` | `timestamp` | Reading timestamp | -| `anncum` | `double` | Annual cumulative rainfall | -| `daily` | `double` | Daily cumulative rainfall | -| `hourly` | `double` | Hourly rainfall | -| `currentrf` | `double` | Current rainfall | -| `battery` | `double` | Battery voltage | - -### `waterlevel` - -Stores water level readings with threshold values. - -| Column | Type | Description | -|--------|------|-------------| -| `id` | `bigint` (PK, auto) | Auto-increment ID | -| `stationid` | `varchar` | Station identifier (FK to station) | -| `datetime` | `timestamp` | Reading timestamp | -| `waterlevel` | `double` | Current water level (meters) | -| `alert` | `double` | Alert threshold level | -| `warning` | `double` | Warning threshold level | -| `danger` | `double` | Danger threshold level | - -### `siren` - -Stores siren activation records. - -| Column | Type | Description | -|--------|------|-------------| -| `id` | `bigint` (PK, auto) | Auto-increment ID | -| `stationid` | `varchar` | Station identifier (FK to station) | -| `stationtype` | `varchar` | Station type identifier | -| `active_time` | `timestamp` | Siren activation time | -| `level` | `varchar` | Siren level (`H`=Danger, `L`=Warning, `N`=Normal) | - -### `notification` - -Stores threshold-exceeded alert records. - -| Column | Type | Description | -|--------|------|-------------| -| `id` | `bigint` (PK, auto) | Auto-increment ID | -| `stationid` | `varchar` | Station identifier (FK to station) | -| `timestamp` | `timestamp` | Alert timestamp | -| `stationtype` | `integer` | Type: 1=rainfall, 2=waterlevel, 3=siren | -| `level` | `varchar` | Alert level (Alert, Warning, Danger) | -| `active_time` | `timestamp` (nullable) | Activation time | - -### `users` - -Stores application users. - -| Column | Type | Description | -|--------|------|-------------| -| `id` | `bigint` (PK, auto) | Auto-increment ID | -| `name` | `varchar(255)` | Username | -| `email` | `varchar(255)` (unique) | Email address | -| `email_verified_at` | `timestamp` (nullable) | Email verification timestamp | -| `password` | `varchar(255)` | Bcrypt-hashed password | -| `access_level` | `integer` | 1=Admin, 2=User | -| `is_blocked` | `boolean` | Account blocked status | -| `login_attempts` | `integer` | Failed login attempt count | -| `remember_token` | `varchar` | Remember me token | -| `created_at`, `updated_at` | `timestamp` | Laravel timestamps | - -### Laravel Standard Tables - -- `password_reset_tokens` — Password reset tokens -- `sessions` — Database-backed sessions -- `cache` / `cache_locks` — Cache store -- `jobs` / `job_batches` / `failed_jobs` — Queue system -- `migrations` — Migration tracking - -## Relationships +## Entity-Relationship Diagram ``` -station (1) ──< (many) rainfall via stationid -station (1) ──< (many) waterlevel via stationid -station (1) ──< (many) siren via stationid -station (1) ──< (many) notification via stationid + station (PK: stationid) + |--------------| + | stationid |---+----------------+----------------+ + | name | | | | + | district | | | | + | lng, lat | | | | + | mainriverbasin| | | | + | subriverbasin| | | | + | rainfall | | | | + | waterlevel | | | | + | siren | | | | + | cctv_link | | | | + |--------------| | | | + v v v + rainfall waterlevel siren + (PK: id) (PK: id) (PK: id) + |--------------| |--------------| |--------------| + | id | | id | | id | + | stationid | | stationid | | stationid | + | timestamp | | datetime | | stationtype | + | anncum | | waterlevel | | active_time | + | daily | | alert | | level | + | hourly | | warning | |--------------| + | currentrf | | danger | + | battery | |--------------| + |--------------| + | | + v v + notification (PK: id) + |--------------| + | id | + | stationid | + | timestamp | + | stationtype | + | level | + | active_time | + |--------------| ``` -**Note**: No database-level foreign keys or constraints exist. All relationships are maintained at the application level. +**Note:** There are no database-level foreign keys between `station` and the +data tables. Relationships are enforced at the application level by matching +`stationid` values. -## Indexes +## Application Tables -- `users.email` — unique index -- `sessions.last_activity` — index -- `sessions.user_id` — index -- No additional indexes on data tables (potential performance concern) +### station -## Default Data +Primary reference table for all monitoring stations. -### Default Admin User +| Column | Type | Nullable | Notes | +|-----------------|---------------|----------|----------------------| +| stationid | string (PK) | no | External station ID | +| name | string | no | Station display name | +| district | string | no | | +| lng | float | no | Longitude | +| lat | float | no | Latitude | +| mainriverbasin | string | no | | +| subriverbasin | string | no | | +| rainfall | integer | no | Flag/counter | +| waterlevel | integer | no | Flag/counter | +| siren | integer | yes | Added by migration | +| cctv_link | string | yes | Added by migration | -Created via `DatabaseSeeder` and migration `2025_12_11_124201_add_default_user_to_users_table.php`: +Migration: `2025_11_06_071853_create_station_table.php` +Alter migrations: `2025_11_08_004548`, `2025_11_25_113158` -- **Username**: `admin` (seeder) / `admin` (migration) -- **Email**: `admin@example.com` -- **Password**: `password123` -- **Access Level**: 1 (Admin) +### rainfall -**Note**: The admin user is created in both the seeder AND a migration, which would cause a duplicate key error if both run. +Rainfall sensor readings per station. + +| Column | Type | Nullable | +|------------|-------------|----------| +| id | bigint (PK) | no | +| stationid | string | no | +| timestamp | timestamp | no | +| anncum | double | no | +| daily | double | no | +| hourly | double | no | +| currentrf | double | no | +| battery | double | no | + +Migration: `2025_11_06_072709_create_rainfall__table.php` + +### waterlevel + +Water level sensor readings per station. + +| Column | Type | Nullable | +|-------------|-------------|----------| +| id | bigint (PK) | no | +| stationid | string | no | +| datetime | timestamp | no | +| waterlevel | double | no | +| alert | double | no | +| warning | double | no | +| danger | double | no | + +Migration: `2025_11_06_072738_create_waterlevel_table.php` + +### siren + +Siren activation records. + +| Column | Type | Nullable | +|--------------|-------------|----------| +| id | bigint (PK) | no | +| stationid | string | no | +| stationtype | string | no | +| active_time | timestamp | no | +| level | string | no | + +Migration: `2025_11_07_031601_create_siren__table.php` + +The `stationtype` column is always `3` for siren inserts (see stationtype +values below). + +### notification + +Notification log for alerts across all station types. + +| Column | Type | Nullable | +|--------------|-------------|----------| +| id | bigint (PK) | no | +| stationid | string | no | +| timestamp | timestamp | no | +| stationtype | integer | no | +| level | string | no | +| active_time | timestamp | yes | + +Migration: `2025_11_07_024940_create_notification_table.php` + +### users + +Application users with access control and blocking support. + +| Column | Type | Nullable | Default | Notes | +|--------------------|-------------|----------|---------|-------------------| +| id | bigint (PK) | no | auto | | +| name | string | no | | | +| email | string | no | | unique | +| email_verified_at | timestamp | yes | | | +| password | string | no | | bcrypt hashed | +| access_level | integer | no | 2 | 1=admin, 2=user | +| is_blocked | boolean | no | false | | +| login_attempts | integer | no | 0 | | +| remember_token | string | yes | | | +| created_at | timestamp | yes | | | +| updated_at | timestamp | yes | | | + +Base: `0001_01_01_000000_create_users_table.php` +Alter: `2025_11_07_063825`, `2025_11_07_065418` + +## stationtype Values + +The `stationtype` field is used in the `notification` and `siren` tables to +identify the source station type. + +| Value | Type | +|-------|-------------| +| 1 | Rainfall | +| 2 | Water level | +| 3 | Siren | + +## Laravel Standard Tables + +These tables are created by Laravel's built-in migrations and support +framework features (sessions, caching, queue jobs). + +| Table | Purpose | +|-----------------------|-------------------------------| +| password_reset_tokens | Password reset tokens | +| sessions | Database-backed sessions | +| cache | Application cache store | +| cache_locks | Atomic cache locks | +| jobs | Queue job queue | +| job_batches | Batched job tracking | +| failed_jobs | Failed job records | +| migrations | Migration tracking | + +## Default Admin User + +The `DatabaseSeeder` creates a default admin user on first run using +`firstOrCreate`, making it idempotent -- safe to re-run without duplicates. + +```php +User::firstOrCreate( + ['email' => env('ADMIN_EMAIL', 'admin@example.com')], + [ + 'name' => 'Admin', + 'password' => Hash::make(env('ADMIN_PASSWORD', str()->random(32))), + 'access_level' => 1, + ] +); +``` + +- **Email**: set by `ADMIN_EMAIL` env var (default: `admin@example.com`) +- **Password**: set by `ADMIN_PASSWORD` env var (default: random 32-char string) +- **access_level**: `1` (admin) + +The seeder is triggered automatically when `RUN_SEEDER=true` in the Docker +environment (see `docker/entrypoint.sh`). + +The migration `2025_12_11_124201_add_default_user_to_users_table.php` exists +but is a no-op (empty `up()` and `down()`) -- admin creation is handled +exclusively by the seeder. + +## Seeding and Migrations + +Migrations and seeding run automatically at container startup via +`docker/entrypoint.sh`: + +```bash +# Runs when RUN_MIGRATIONS=true +php artisan migrate --force + +# Runs when RUN_SEEDER=true +php artisan db:seed --force +``` + +Both `RUN_MIGRATIONS` and `RUN_SEEDER` default to `true` in +`docker-compose.yml`. + +## Design Notes + +**No foreign keys.** The `stationid` column in `rainfall`, `waterlevel`, +`siren`, and `notification` references `station.stationid` at the application +level only. There are no database-level `REFERENCES` constraints. This means +orphaned data rows can exist if a station is deleted without cleaning +associated records. + +**No additional indexes.** Beyond primary keys and Laravel's default indexes +(on `sessions.user_id`, `sessions.last_activity`, `jobs.queue`), no custom +indexes exist on the data tables. This is a potential performance concern as +the `rainfall`, `waterlevel`, and `notification` tables grow -- queries +filtering by `stationid` or ordering by timestamp will perform full table +scans. diff --git a/docs/07-API.md b/docs/07-API.md index dc69a284..0c716f0e 100644 --- a/docs/07-API.md +++ b/docs/07-API.md @@ -1,20 +1,26 @@ # API Reference +All API endpoints are defined in `src/routes/api.php` and served under the `/api` prefix. There is no authentication middleware on any API route -- all endpoints are publicly accessible. + +--- + ## Authentication ### POST `/api/login` -Login via API (returns user info, no token). +Validates user credentials against the `users` table. No token or session is created; this endpoint only confirms the username and password are correct. **Request Body:** + ```json { "username": "admin", - "password": "password123" + "password": "" } ``` **Success Response (200):** + ```json { "error": false, @@ -26,6 +32,7 @@ Login via API (returns user info, no token). ``` **Error Response (200):** + ```json { "error": true, @@ -33,10 +40,11 @@ Login via API (returns user info, no token). } ``` -**Notes:** -- No authentication token is generated — this endpoint only validates credentials -- No session is created — API endpoints are stateless -- All other API endpoints have **no authentication** — they are publicly accessible +**Implementation details:** + +- Uses a parameterized query (`DB::select` with `?` placeholder) to look up the user by `name` +- Password verification uses `Hash::check()` against the bcrypt hash stored in the `password` column +- Returns generic "Wrong Password/Username" for both unknown user and wrong password (no user enumeration) --- @@ -44,53 +52,48 @@ Login via API (returns user info, no token). ### GET `/api/station/current` -Returns all stations with their latest rainfall, water level, and siren data. +Returns all stations that have valid coordinates (`lat`/`lng` not null) joined with their latest rainfall reading, water level reading, and siren status. Each row is the most recent record from the `rainfall`, `waterlevel`, and `siren` tables per station. -**Response:** -```json -[ - { - "stationid": "KBLG0026", - "name": "Stesen Kupang", - "district": "Baling", - "lng": 100.7521, - "lat": 5.7879, - "rainfall": 1, - "waterlevel": 1, - "siren": 1, - "rainfall_value": 12.5, - "rainfall_time": "2025-11-06T14:00:00", - "waterlevel_value": 3.2, - "waterlevel_time": "2025-11-06T14:00:00", - "siren_level": "N", - "siren_time": "2025-11-06T14:00:00" - } -] -``` +**Response fields per station:** + +| Field | Source | Description | +|-------|--------|-------------| +| `stationid` | `station` | Station identifier (e.g. `KBLG0026`) | +| `name` | `station` | Station name | +| `district` | `station` | District name | +| `lng`, `lat` | `station` | GPS coordinates | +| `rainfall_value` | `rainfall.hourly` | Latest hourly rainfall | +| `rainfall_time` | `rainfall.timestamp` | Timestamp of latest rainfall reading | +| `waterlevel_value` | `waterlevel.waterlevel` | Latest water level | +| `waterlevel_time` | `waterlevel.datetime` | Timestamp of latest water level reading | +| `siren_level` | `siren.level` | Latest siren level (`"N"` = Normal) | +| `siren_time` | `siren.active_time` | Timestamp of latest siren event | + +Stations with no data across all three types (rainfall, waterlevel, siren) are excluded. ### GET `/api/station/rainfall` -Returns latest rainfall data for rainfall-enabled stations. +Returns the latest rainfall record per station. Only includes stations where the `station.rainfall` flag is set and a matching `rainfall` record exists. Returns all columns from both `station` and `rainfall` tables. ### GET `/api/station/waterlevel` -Returns latest water level data for waterlevel-enabled stations. +Returns the latest water level record per station. Only includes stations where the `station.waterlevel` flag is set and a matching `waterlevel` record exists. Returns all columns from both `station` and `waterlevel` tables. ### GET `/api/station/notification` -Returns today's latest notification per station. +Returns today's latest notification per station. Joins `station` with `notification` where `notification.timestamp::date = CURRENT_DATE`, returning only the most recent notification per station for the current day. Ordered by timestamp descending. ### GET `/api/station/history` -Returns notification history for the last 3 days (latest per station per day). +Returns notification history for the last 3 days. For each station, returns the single latest notification per day (one row per station per day). Ordered by timestamp descending. ### GET `/api/station/siren` -Returns current siren status for siren-equipped stations (last 3 days). +Returns current siren status for siren-equipped stations. Joins `station` with `siren` where `station.siren = 1`, returning only the most recent siren event per station within the last 3 days. Ordered by `active_time` descending. ### GET `/api/station/siren/history` -Returns siren history for the last 3 days (excluding Normal level). +Returns siren history for the last 3 days, excluding Normal-level entries (`siren.level != "N"`). Only includes stations with `station.siren = 1`. Ordered by `active_time` descending. --- @@ -98,9 +101,10 @@ Returns siren history for the last 3 days (excluding Normal level). ### POST `/api/alert` -Sends an FCM push notification. Called by the Python autoscript. +Sends an FCM push notification to a topic. Called by `sidesdecode.py` when sensor thresholds are triggered. **Request Body:** + ```json { "stationid": "KBLG0026", @@ -114,26 +118,55 @@ Sends an FCM push notification. Called by the Python autoscript. | Field | Type | Description | |-------|------|-------------| | `stationid` | string | Station identifier | -| `level` | string | Alert level: "Alert", "Warning", "Danger" | -| `stationtype` | integer | 1=Rainfall, 2=Water Level, 3=Siren | +| `level` | string | Alert level (e.g. `"Alert"`, `"Warning"`, `"Danger"`) | +| `stationtype` | integer | Sensor type: `1` = Rainfall, `2` = Water Level, `3` = Siren | + +**FCM topic routing:** + +The controller maps `stationtype` and `level` to an FCM topic: + +| stationtype | level | FCM Topic | +|-------------|-------|-----------| +| 1 (Rainfall) | any non-"Danger" | `FCM_TOPIC_RAINFALL_WARNING` | +| 1 (Rainfall) | "Danger" | `FCM_TOPIC_RAINFALL_DANGER` | +| 2 (Water Level) | any non-"Danger" | `FCM_TOPIC_WATERLEVEL_ALERT` | +| 2 (Water Level) | "Danger" | `FCM_TOPIC_WATERLEVEL_DANGER` | +| 3 (Siren) | any | `FCM_TOPIC_RAINFALL_WARNING` | + +The notification title is formatted as `"{Type} {Level} Alert"` and the body as `"{stationid} : {Type} Have Triggered {Level}"`. **Response:** + ```json { "status": 200 } ``` -The `status` is the HTTP status code returned by the FCM API (200 = success). +The `status` value is the HTTP status code returned by the FCM v1 API (`200` = delivered successfully). + +**Implementation:** + +- `FcmService` loads Firebase credentials from the path in `FIREBASE_CREDENTIALS` and uses `Google\Auth\Credentials\ServiceAccountCredentials` to obtain an OAuth2 access token +- Sends to `https://fcm.googleapis.com/v1/projects/{FIREBASE_PROJECT_ID}/messages:send` +- Android priority is set to `"high"` + +**Security note:** This endpoint has no authentication. Anyone with network access can trigger push notifications. --- -## Public Web API +## Public Web Routes + +These are defined in `src/routes/web.php` (not under `/api`) but return JSON or perform redirects. ### GET `/stations` -Returns all stations with coordinates as JSON (used by the public map). +Returns all stations with coordinates as JSON. Used by the public-facing map. No authentication required. + +### GET `/dashboard` + +Returns the same combined station data as `/api/station/current`. Served at the root URL `/` as well. No authentication required. ### GET `/locale/{locale}` -Switches language. Valid values: `en`, `bm`. Redirects back to previous page. +Switches the application language. Accepts `en` or `bm`. Redirects back to the previous page. diff --git a/docs/08-CONFIGURATION.md b/docs/08-CONFIGURATION.md index f3bfbcb0..3223283e 100644 --- a/docs/08-CONFIGURATION.md +++ b/docs/08-CONFIGURATION.md @@ -1,119 +1,179 @@ -# Configuration Reference + -## Environment Variables +# Configuration -### Application Settings +SIDES uses a layered configuration approach: a root `.env` for Docker Compose infrastructure, and a separate `src/.env` for the Laravel application. This document covers both. -| Variable | Default | Description | -|----------|---------|-------------| -| `APP_NAME` | `SIDES` | Application name | -| `APP_ENV` | `local` | Environment (local/production) | -| `APP_KEY` | (generated) | Encryption key | -| `APP_DEBUG` | `true` | Show debug errors | -| `APP_URL` | `https://sides.tck.com.my` | Application URL | -| `APP_LOCALE` | `en` | Default language | -| `APP_TIMEZONE` | `Asia/Kuala_Lumpur` | Timezone (set in config/app.php) | +--- -### Database +## Environment Variables Overview -| Variable | Value | Description | -|----------|-------|-------------| -| `DB_CONNECTION` | `pgsql` | Database driver | -| `DB_HOST` | `192.168.0.211` | Database host (external IP) | -| `DB_PORT` | `5432` | PostgreSQL port | -| `DB_DATABASE` | `sides_db` | Database name (via `POSTGRES_DB`) | -| `DB_USERNAME` | `tck` | Database user (via `POSTGRES_USER`) | -| `DB_PASSWORD` | `projectdev##1` | Database password (via `POSTGRES_PASSWORD`) | +### Root `.env` (Docker Compose) -### Docker-level Database +Located at the project root. Controls container-level configuration for PostgreSQL, pgAdmin, Firebase, admin seeding, and FTP access. See `.env.example` for the full list. -| Variable | Value | Description | -|----------|-------|-------------| -| `POSTGRES_DB` | `tckdev` | Docker postgres database name | -| `POSTGRES_USER` | `tck` | Docker postgres user | -| `POSTGRES_PASSWORD` | `projectdev##1` | Docker postgres password | +| Variable | Required | Default | Description | +|---|---|---|---| +| `POSTGRES_DB` | Yes | — | PostgreSQL database name | +| `POSTGRES_USER` | Yes | — | PostgreSQL user | +| `POSTGRES_PASSWORD` | Yes | — | PostgreSQL password | +| `PGADMIN_EMAIL` | Yes | — | pgAdmin login email | +| `PGADMIN_PASSWORD` | Yes | — | pgAdmin login password | +| `FIREBASE_PROJECT_ID` | Yes | — | Firebase project identifier | +| `FIREBASE_CREDENTIALS` | Yes | — | Path to Firebase credentials JSON file | +| `FCM_TOPIC_RAINFALL_WARNING` | No | `rainfall_warning` | FCM topic for rainfall warnings | +| `FCM_TOPIC_RAINFALL_DANGER` | No | `rainfall_danger` | FCM topic for rainfall danger alerts | +| `FCM_TOPIC_WATERLEVEL_ALERT` | No | `waterlevel_alert` | FCM topic for water level alerts | +| `FCM_TOPIC_WATERLEVEL_DANGER` | No | `waterlevel_danger` | FCM topic for water level danger alerts | +| `ADMIN_EMAIL` | No | `admin@example.com` | Admin email used by `DatabaseSeeder` | +| `ADMIN_PASSWORD` | No | `admin123` | Admin password used by `DatabaseSeeder` | +| `FTP_SERVER` | No | — | FTP server hostname for `sidesdecode.py` | +| `FTP_USERNAME` | No | — | FTP username | +| `FTP_PASSWORD` | No | — | FTP password | -**Note**: There is a discrepancy — Docker `.env` creates database `tckdev` while Laravel `.env` connects to `sides_db` on an external host. +### App `.env` (Laravel) -### Firebase / FCM +Located at `src/.env`. Controls Laravel framework and application settings. See `src/.env.example` for the full list. -| Variable | Description | -|----------|-------------| -| `FIREBASE_PROJECT_ID` | `sides-b4abb` | -| `FIREBASE_CREDENTIALS` | `storage/app/firebase/sides-b4abb-3604a7cf7584.json` | -| `FCM_TOPIC_RAINFALL_WARNING` | `rainfall_warning` | -| `FCM_TOPIC_RAINFALL_DANGER` | `rainfall_danger` | -| `FCM_TOPIC_WATERLEVEL_ALERT` | `waterlevel_alert` | -| `FCM_TOPIC_WATERLEVEL_DANGER` | `waterlevel_danger` | +| Variable | Required | Default | Description | +|---|---|---|---| +| `APP_NAME` | Yes | `SIDES` | Application name | +| `APP_ENV` | Yes | `local` | Environment: `local`, `production` | +| `APP_KEY` | Yes | — | Encryption key (generated via `php artisan key:generate`) | +| `APP_DEBUG` | Yes | `true` | Enable debug mode (`false` in production) | +| `APP_URL` | Yes | `http://localhost` | Application URL | +| `DB_CONNECTION` | Yes | `pgsql` | Database driver | +| `DB_HOST` | Yes | `postgres` | Database host (Docker service name) | +| `DB_PORT` | Yes | `5432` | Database port | +| `DB_DATABASE` | Yes | `${POSTGRES_DB}` | Database name (references root `.env`) | +| `DB_USERNAME` | Yes | `${POSTGRES_USER}` | Database user (references root `.env`) | +| `DB_PASSWORD` | Yes | `${POSTGRES_PASSWORD}` | Database password (references root `.env`) | +| `SESSION_DRIVER` | No | `database` | Session storage driver | +| `CACHE_STORE` | No | `database` | Cache storage driver | +| `QUEUE_CONNECTION` | No | `database` | Queue driver | +| `MAIL_MAILER` | No | `log` | Mail driver (`smtp` for production) | +| `MAIL_HOST` | No | `127.0.0.1` | SMTP server hostname | +| `MAIL_PORT` | No | `2525` | SMTP port | +| `MAIL_USERNAME` | No | `null` | SMTP username | +| `MAIL_PASSWORD` | No | `null` | SMTP password | +| `MAIL_FROM_ADDRESS` | No | `hello@example.com` | Sender email address | +| `MAIL_FROM_NAME` | No | `${APP_NAME}` | Sender display name | +| `FIREBASE_PROJECT_ID` | Yes | — | Firebase project identifier | +| `FIREBASE_CREDENTIALS` | Yes | — | Path to Firebase credentials JSON | +| `FCM_TOPIC_*` | No | See defaults | FCM notification topic names | -### Mail (SMTP) - -| Variable | Value | Description | -|----------|-------|-------------| -| `MAIL_MAILER` | `smtp` | Mail driver | -| `MAIL_HOST` | `smtp.gmail.com` | Gmail SMTP | -| `MAIL_PORT` | `587` | SMTP port (TLS) | -| `MAIL_USERNAME` | `sideskupang@gmail.com` | Gmail account | -| `MAIL_PASSWORD` | `ipmu zifw bpmf fsyp` | Gmail App Password | -| `MAIL_FROM_ADDRESS` | `sideskupang@gmail.com` | From address | -| `MAIL_FROM_NAME` | `${APP_NAME}` | From name | - -### Session & Cache - -| Variable | Value | Description | -|----------|-------|-------------| -| `SESSION_DRIVER` | `database` | Session storage | -| `SESSION_LIFETIME` | `120` | Session timeout (minutes) | -| `CACHE_STORE` | `database` | Cache storage | -| `QUEUE_CONNECTION` | `database` | Queue driver | +--- ## Docker Compose Configuration -### Networks +The stack is defined in `docker-compose.yml` and consists of four services on a shared `tckdev_net` network: -- `tckdev_net` — Custom bridge network for all containers +### Services + +| Service | Image | Container | Port | Description | +|---|---|---|---|---| +| `app` | Custom (`Dockerfile`) | `tckdev-app` | 9000 (internal) | PHP-FPM 8.2 application | +| `postgres` | `postgres:16` | `tckdev-db` | `5432:5432` | PostgreSQL database | +| `web` | `nginx:stable-alpine` | `tckdev-web` | `80:80` | Nginx reverse proxy | +| `pgadmin` | `dpage/pgadmin4` | `tckdev-pgAdmin` | `5050:80` | pgAdmin database manager | ### Volumes -| Volume | Container | Purpose | -|--------|-----------|---------| -| `./src:/var/www/html` | app, web | Laravel application code | -| `./docker/postgres/data:/var/lib/postgres/data` | postgres | PostgreSQL data | -| `./docker/nginx/default.conf:/etc/nginx/conf.d/default.conf` | web | Nginx config | -| `./backup:/var/lib/pgadmin/storage/...` | pgadmin | pgAdmin backup storage | +**Named volumes:** +- `pgdata` -- PostgreSQL data persistence (`/var/lib/postgresql/data`) +- `pgadmin_data` -- pgAdmin session and settings -### Ports +**Bind mounts:** +- `./src:/var/www/html` -- Application source (mounted on both `app` and `web`) +- `./docker/nginx/default.conf:/etc/nginx/conf.d/default.conf` -- Nginx config (on `web`) +- `./backup:/backups` -- Backup directory (on `pgadmin`) -| Host Port | Container Port | Service | -|-----------|---------------|---------| -| 80 | 80 | Nginx (web) | -| 5432 | 5432 | PostgreSQL | -| 5050 | 80 | pgAdmin | -| 6060 | 8080 | Adminer | +### App Container Environment + +The `app` service passes these environment variables to control startup behavior: + +| Variable | Default | Description | +|---|---|---| +| `RUN_MIGRATIONS` | `true` | Run `php artisan migrate --force` on boot | +| `RUN_SEEDER` | `true` | Run `php artisan db:seed --force` on boot | +| `ADMIN_EMAIL` | `${ADMIN_EMAIL:-admin@example.com}` | Admin email for `DatabaseSeeder` | +| `ADMIN_PASSWORD` | `${ADMIN_PASSWORD:-admin123}` | Admin password for `DatabaseSeeder` | + +### Entrypoint + +The custom entrypoint (`docker/entrypoint.sh`) runs on every container start and performs: + +1. Copies `.env.example` to `.env` if no `.env` exists +2. Runs `composer install` if `vendor/` is missing +3. Runs `npm install` if `node_modules/` is missing +4. Runs `npm run build` if `public/build/` is missing +5. Runs database migrations if `RUN_MIGRATIONS=true` +6. Runs database seeder if `RUN_SEEDER=true` +7. Caches config, routes, and views + +### Health Check + +PostgreSQL has a health check (`pg_isready`) with 5-second intervals. Both `app` and `pgadmin` wait for PostgreSQL to be healthy before starting (`depends_on` with `condition: service_healthy`). + +--- ## Nginx Configuration -Located at `docker/nginx/default.conf`: +Located at `docker/nginx/default.conf`. Key settings: -- Root: `/var/www/html/public` -- PHP processing via FastCGI to `app:9000` -- Security headers: X-Frame-Options, X-XSS-Protection, X-Content-Type-Options -- Laravel URL rewriting: `try_files $uri $uri/ /index.php?$query_string` -- Hidden file access denied (except `.well-known`) +- **Document root:** `/var/www/html/public` +- **FastCGI pass:** `app:9000` (PHP-FPM on the `app` service) +- **Security headers:** `X-Frame-Options: SAMEORIGIN`, `X-XSS-Protection: 1; mode=block`, `X-Content-Type-Options: nosniff` +- **Laravel rewriting:** `try_files $uri $uri/ /index.php?$query_string` (standard Laravel URL rewriting) +- **Hidden files:** Access to dotfiles (except `.well-known`) is denied + +--- ## Laravel Configuration -### Middleware Registration +### Application Settings -Registered in `bootstrap/app.php`: +- **HTTPS enforcement:** `URL::forceScheme('https')` is called in `AppServiceProvider::boot()` (`src/app/Providers/AppServiceProvider.php`) +- **Timezone:** `Asia/Kuala_Lumpur` (set in `src/config/app.php`) +- **Locale:** `en` (default), with fallback locale also `en` -- `admin` → `AdminMiddleware::class` (alias) -- `LocalizationMiddleware` appended to web middleware group +### Middleware -### HTTPS Forcing +Registered in `src/bootstrap/app.php`: -`AppServiceProvider::boot()` calls `URL::forceScheme('https')` globally, making all generated URLs use HTTPS. +| Alias | Class | Group | Description | +|---|---|---|---| +| `admin` | `AdminMiddleware` | Alias | Protects admin-only routes | +| — | `LocalizationMiddleware` | `web` (appended) | Handles language/locale switching | -### Timezone +### Session, Cache, and Queue -Set to `Asia/Kuala_Lumpur` in `config/app.php` (UTC+8, Malaysia Time). +All three subsystems use the `database` driver by default, storing data in the PostgreSQL database. No external cache or queue server is required. + +### Mail + +In production, mail is sent via SMTP through Gmail (`smtp.gmail.com:587`). In local development, the `log` mailer is used (emails are written to the log file instead of being sent). + +--- + +## External Services + +### Firebase Cloud Messaging (FCM) + +Used for push notifications (rainfall warnings, rainfall danger, water level alerts, water level danger). Configuration requires: + +- `FIREBASE_PROJECT_ID` -- The Firebase project ID +- `FIREBASE_CREDENTIALS` -- Path to the service account JSON key file (stored in `storage/app/firebase/`) +- `FCM_TOPIC_*` variables -- Topic names for each notification type + + + +### FTP (sidesdecode.py) + +The `autoscript/sidesdecode.py` script connects to an external FTP server to download station data files. It reads `FTP_SERVER`, `FTP_USERNAME`, and `FTP_PASSWORD` from environment variables, falling back to defaults if not set. + + + +### PostgreSQL + +Runs as a Docker container using the official `postgres:16` image. Connection credentials are shared between the root `.env` and `src/.env` via variable interpolation (`${POSTGRES_DB}`, `${POSTGRES_USER}`, `${POSTGRES_PASSWORD}`).