commit 3b0a0a0d65feabf25cf61f621605d30398f7bd3b Author: root Date: Wed May 20 16:11:56 2026 +0800 Add codebase map: 7 structured analysis documents diff --git a/.planning/codebase/ARCHITECTURE.md b/.planning/codebase/ARCHITECTURE.md new file mode 100644 index 00000000..13c04962 --- /dev/null +++ b/.planning/codebase/ARCHITECTURE.md @@ -0,0 +1,199 @@ + +# Architecture + +**Analysis Date:** 2026-05-20 + +## System Overview + +```text +┌─────────────────────────────────────────────────────────────────┐ +│ External Data Feed │ +│ FTP Server (myvscada.com) → autoscript/sidesdecode.py (Python) │ +└──────────────────────────┬──────────────────────────────────────┘ + │ psycopg2 direct INSERT + HTTP POST /api/alert + ▼ +┌─────────────────────────────────────────────────────────────────┐ +│ Laravel 12 Web Application │ +│ `src/` — PHP 8.2, served via nginx + php-fpm in Docker │ +├──────────────┬──────────────────┬───────────────────────────────┤ +│ Web Routes │ API Routes │ Admin Routes │ +│ `routes/ │ `routes/ │ `routes/web.php` │ +│ web.php` │ api.php` │ (admin middleware) │ +├──────────────┴──────────────────┴───────────────────────────────┤ +│ Controller Layer │ +│ MapController RainfallController WaterLevelController │ +│ SirenController NotificationController AdminController │ +│ cctvController LocaleController ProfileController │ +│ Api\StationController Api\AuthController Api\AlertController │ +├─────────────────────────────────────────────────────────────────┤ +│ Service Layer │ +│ FcmService (`app/Services/FcmService.php`) — Firebase FCM push │ +│ Export classes (`app/Exports/`) — Excel & PDF generation │ +├─────────────────────────────────────────────────────────────────┤ +│ Middleware Stack │ +│ AdminMiddleware · LocalizationMiddleware │ +├─────────────────────────────────────────────────────────────────┤ +│ Database — PostgreSQL │ +│ station · rainfall · waterlevel · siren · notification · users │ +└─────────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────┐ +│ Firebase Cloud Messaging (FCM) │ +│ Push notifications via topic-based messaging │ +└─────────────────────────────────────────────────────────────────┘ +``` + +## Component Responsibilities + +| Component | Responsibility | File | +|-----------|----------------|------| +| MapController | Dashboard / home page with station map & current readings | `src/app/Http/Controllers/MapController.php` | +| RainfallController | Rainfall data display, graphs, historical, threshold (IDF), Excel export | `src/app/Http/Controllers/RainfallController.php` | +| WaterLevelController | Water level data display, graphs, historical, Excel export | `src/app/Http/Controllers/WaterLevelController.php` | +| SirenController | Siren status display, siren history, PDF export | `src/app/Http/Controllers/SirenController.php` | +| NotificationController | Alarm notifications (rainfall/waterlevel/siren), history, PDF export | `src/app/Http/Controllers/NotificationController.php` | +| AdminController | Station CRUD, user CRUD, password management | `src/app/Http/Controllers/AdminController.php` | +| cctvController | CCTV link display for stations | `src/app/Http/Controllers/cctvController.php` | +| LocaleController | Language switching (English/Bahasa Melayu) | `src/app/Http/Controllers/LocaleController.php` | +| ProfileController | User profile management (Breeze-generated) | `src/app/Http/Controllers/ProfileController.php` | +| Api\StationController | REST API for mobile/external: station data, rainfall, waterlevel, notifications, siren | `src/app/Http/Controllers/Api/StationController.php` | +| Api\AuthController | API login (stateless, username+password) | `src/app/Http/Controllers/Api/AuthController.php` | +| Api\AlertController | Receives alert from Python script, sends FCM push notification | `src/app/Http/Controllers/Api/AlertController.php` | +| FcmService | Firebase FCM topic-based push notification sender | `src/app/Services/FcmService.php` | +| sidesdecode.py | External Python script: reads CSV from FTP, inserts into PostgreSQL, triggers alerts | `autoscript/sidesdecode.py` | + +## Pattern Overview + +**Overall:** Server-Side Rendered MVC (Laravel monolith) + +**Key Characteristics:** +- No Eloquent ORM for domain tables — all data access uses raw SQL via `DB::select()` and `DB::table()` +- Only `User` model exists in `app/Models/`; station, rainfall, waterlevel, siren, notification tables have no Eloquent models +- Blade templates for all server-rendered views under `resources/views/layout/` +- Laravel Breeze provides authentication scaffolding (login, register, password reset) +- API routes are unauthenticated (no token/API key middleware) except the `login` endpoint +- Data ingestion is handled externally by a Python script that connects directly to PostgreSQL (bypasses Laravel entirely) +- i18n via session-based locale with translation files in `lang/en/` and `lang/bm/` + +## Layers + +**Route Layer:** +- Purpose: HTTP request routing +- Location: `src/routes/` +- Contains: `web.php` (web + admin), `api.php` (REST API), `auth.php` (Breeze auth), `console.php` (artisan commands) +- Depends on: Controllers +- Used by: HTTP clients (browsers, mobile app, Python script) + +**Controller Layer:** +- Purpose: Request handling, query execution, view rendering +- Location: `src/app/Http/Controllers/` +- Contains: Domain controllers + API controllers +- Depends on: `DB` facade, `Excel` facade, `Pdf` facade, `FcmService` +- Used by: Route layer + +**Service Layer (minimal):** +- Purpose: Reusable business logic (only FCM at present) +- Location: `src/app/Services/FcmService.php` +- Contains: Firebase Cloud Messaging integration +- Depends on: `google/auth` package, `Http` facade + +**View Layer:** +- Purpose: Server-rendered HTML via Blade templates +- Location: `src/resources/views/` +- Contains: Layout templates, admin views, notification views, PDF templates, auth views, Blade components +- Depends on: Controller data via `compact()` + +## Data Flow + +### Primary Request Path (Web Dashboard) + +1. Browser requests `/` or `/dashboard` → Route (`src/routes/web.php:73`) +2. `MapController::getCurrentData()` executes a 4-table LEFT JOIN query (station + rainfall + waterlevel + siren, each with MAX subquery) (`src/app/Http/Controllers/MapController.php:29`) +3. Returns `view('layout.dashboard', compact('data'))` — Blade renders with station pins on map + +### Data Ingestion Path (Python → PostgreSQL → FCM Alert) + +1. `sidesdecode.py` runs (likely via cron) → connects to FTP server, downloads CSV files (`autoscript/sidesdecode.py:42-49`) +2. Parses each CSV line, extracts station readings (`process_line()` at line 143) +3. Inserts into PostgreSQL tables directly via `psycopg2` — `rainfall`, `waterlevel`, `siren` tables +4. When threshold exceeded (rainfall ≥30mm/hr, water level ≥ alert, siren ≠ 'N'): inserts into `notification` table and calls `send_alert_to_laravel()` (line 124) +5. `POST /api/alert` → `Api\AlertController::send()` (`src/app/Http/Controllers/Api/AlertController.php:18`) +6. `FcmService::sendToTopic()` sends Firebase push notification to topic + +### API Path (Mobile App) + +1. Mobile app calls `POST /api/login` → `Api\AuthController::login()` validates credentials, returns user JSON (no token) +2. Mobile app calls `GET /api/station/current` → `Api\StationController::getCurrentData()` returns all station current data as JSON +3. Other endpoints: `/api/station/rainfall`, `/api/station/waterlevel`, `/api/station/notification`, `/api/station/siren`, `/api/station/history` + +**State Management:** +- Web: Session-based authentication via Laravel Breeze (default `web` guard) +- API: Stateless — no token-based auth; login returns plain user data without JWT/Sanctum token + +## Key Abstractions + +**Export Classes:** +- Purpose: Excel file generation using `maatwebsite/excel` +- Examples: `src/app/Exports/HourlyRainfallExport.php`, `src/app/Exports/WaterLevelExport.php` +- Pattern: Implement `FromCollection`, `WithHeadings`, `ShouldAutoSize`; execute raw SQL in `collection()` method + +**Notification (Laravel):** +- Purpose: Password reset email +- Example: `src/app/Notifications/ResetPasswordNotification.php` +- Pattern: Extends Laravel's `Notification` class, delivers via mail channel + +## Entry Points + +**Web Entry Point:** +- Location: `src/public/index.php` (standard Laravel) +- Triggers: All web browser requests +- Responsibilities: Bootstraps Laravel application, handles HTTP kernel pipeline + +**API Entry Point:** +- Location: `src/routes/api.php` +- Triggers: Mobile app HTTP requests (prefixed with `/api/`) +- Responsibilities: Station data retrieval, authentication, alert triggering + +**CLI Entry Point:** +- Location: `src/artisan` +- Triggers: Command line, cron +- Responsibilities: Migrations, cache clear, queue listen, etc. + +**External Script:** +- Location: `autoscript/sidesdecode.py` +- Triggers: Likely cron or manual execution +- Responsibilities: FTP file polling, CSV parsing, direct PostgreSQL insertion, alert triggering via HTTP API + +## Architectural Constraints + +- **No ORM for domain data:** All domain tables (station, rainfall, waterlevel, siren, notification) are queried via raw SQL (`DB::select()`, `DB::table()`). Only the `users` table has an Eloquent model (`src/app/Models/User.php`). +- **Dual database connections:** The Python script connects directly to PostgreSQL via `psycopg2` with hardcoded credentials, completely bypassing Laravel. The Laravel app can connect to either SQLite (local dev) or PostgreSQL (production) based on `.env`. +- **No API authentication tokens:** The API login endpoint returns user data without issuing a token (no Sanctum/Passport). Subsequent API calls have no authentication middleware. +- **Raw SQL with interpolation risk:** Some controllers build SQL strings with direct variable interpolation (e.g., `$stationCondition` in `RainfallController.php:46`) rather than parameterized queries. Other places correctly use parameter binding. +- **Single-process Python script:** The data ingestion script runs sequentially, processing one FTP file at a time with no parallelism. +- **HTTPS forced globally:** `AppServiceProvider::boot()` calls `URL::forceScheme('https')` (`src/app/Providers/AppServiceProvider.php:23`) + +## Error Handling + +**Strategy:** Basic try-catch with redirect back + flash messages + +**Patterns:** +- Controllers use `redirect()->back()->with('success', ...)` or `->with('error', ...)` for user feedback +- `AdminController` wraps operations in try-catch, returning `ValidationException` first error as flash message +- Python script uses `conn.rollback()` on exception and logs to console +- No centralized exception handling customization in `bootstrap/app.php` (empty `withExceptions` callback) + +## Cross-Cutting Concerns + +**Logging:** Laravel's default file logging to `src/storage/logs/laravel.log`. Python script logs to `autoscript/sidesdecode.log` and `autoscript/sidesdecode_error.log`. + +**Validation:** Controller-level validation using `$request->validate()`. No form request classes for domain operations. Only `ProfileUpdateRequest` exists (`src/app/Http/Requests/ProfileUpdateRequest.php`). + +**Authentication:** Laravel Breeze (session-based) for web. Custom username-based login for API. Admin access controlled via `AdminMiddleware` checking `access_level === 1`. + +**Internationalization:** Session-based locale stored in `locale` key. Supports `en` (English) and `bm` (Bahasa Melayu). Translation files in `src/lang/en/` and `src/lang/bm/`. `LocalizationMiddleware` reads locale from session on every request. + +--- + +*Architecture analysis: 2026-05-20* diff --git a/.planning/codebase/CONCERNS.md b/.planning/codebase/CONCERNS.md new file mode 100644 index 00000000..1e208c4b --- /dev/null +++ b/.planning/codebase/CONCERNS.md @@ -0,0 +1,211 @@ +# Codebase Concerns + +**Analysis Date:** 2026-05-20 + +## Security Concerns + +### CRITICAL: Firebase Service Account Key Committed to Storage +- **Risk:** The file `src/storage/app/firebase/sides-b4abb-3604a7cf7584.json` is a Firebase service account credential file present in the repository. It contains `project_id: "sides-b4abb"` and full private key data. `src/.gitignore` does not exclude `storage/app/firebase/`. +- **Files:** `src/storage/app/firebase/sides-b4abb-3604a7cf7584.json`, `src/app/Services/FcmService.php` (lines 17-19: reads this file at runtime via `env('FIREBASE_CREDENTIALS')`) +- **Impact:** Anyone with repo access has full Firebase Admin SDK privileges (send notifications, access Firestore, etc.) +- **Fix:** Move credentials to environment-only storage, add `storage/app/firebase/*.json` to `.gitignore`, rotate the exposed key. + +### CRITICAL: SQL Injection via String Interpolation in Raw Queries +- **Risk:** Multiple controllers build raw SQL queries by directly interpolating user input into SQL strings without parameterized bindings. +- **Files:** + - `src/app/Http/Controllers/RainfallController.php` line 46: `$stationCondition = " AND s.stationid = '{$stationFilter}'"` — `$stationFilter` comes directly from `$request->get('station')` and is concatenated into `DB::select()` on line 55. + - `src/app/Http/Controllers/RainfallController.php` lines 85-103: `$dateFilter` and `$displayDate` interpolated into CAST expressions. + - `src/app/Http/Controllers/WaterLevelController.php` line 30: `$stationCondition = " WHERE s.stationid = '{$stationFilter}' "` — same pattern. + - `src/app/Http/Controllers/WaterLevelController.php` line 37: `$dateCondition = " AND w.datetime = '{$sqlDate}' "`. +- **Impact:** An attacker can inject arbitrary SQL through the `station` or `date` query parameters, potentially extracting or modifying all data. +- **Fix:** Replace all `DB::select()` with parameterized queries using `?` placeholders or named bindings, or use Eloquent/Query Builder with proper `->where()` calls. + +### CRITICAL: Hardcoded Credentials in Autoscript +- **Risk:** `autoscript/sidesdecode.py` contains hardcoded FTP and PostgreSQL credentials in plain text. +- **Files:** `autoscript/sidesdecode.py` lines 22-31: + - FTP: `ftp_username = "tck"`, `ftp_password = "tck6789"`, server `myvscada.com` + - PostgreSQL: `pg_host = "192.168.0.211"`, `pg_user = "tck"`, `pg_password = "projectdev##1"` +- **Impact:** Credentials are exposed in source control. Internal IP address leaks network topology. +- **Fix:** Move all credentials to environment variables or a secrets manager. Add `autoscript/` to `.gitignore` or restructure to load config from `.env`. + +### HIGH: API Routes Have No Authentication or Rate Limiting +- **Risk:** All API routes in `src/routes/api.php` are completely unauthenticated. The API endpoints expose real-time station data, historical readings, notification history, and siren status without any auth check. +- **Files:** `src/routes/api.php` lines 10-20 — no `middleware('auth')` or `middleware('throttle')` on any route. `src/bootstrap/app.php` line 11 — API routes registered without any middleware group. +- **Impact:** Anyone can query all station data, and the `/api/alert` endpoint can be abused to send fraudulent push notifications to all users. +- **Fix:** Add `Route::middleware(['auth:sanctum'])` to the API route group. At minimum, protect `/api/alert` with authentication. + +### HIGH: API Login Endpoint Returns User Details Without Token +- **Risk:** `Api/AuthController.php` `login()` validates credentials then returns user `id`, `name`, `email`, and `access_level` in a plain JSON response without issuing any authentication token or session. +- **Files:** `src/app/Http/Controllers/Api/AuthController.php` lines 49-55 +- **Impact:** The login endpoint leaks user info but provides no mechanism for subsequent authenticated requests. No token-based auth system exists for the API. +- **Fix:** Implement Laravel Sanctum or Passport to issue API tokens on login. Remove password from the SELECT query (line 24). + +### HIGH: Hardcoded Default Admin Password in Migration +- **Risk:** Migration `2025_12_11_124201_add_default_user_to_users_table.php` seeds an admin user with email `admin@example.com` and password `password123`. +- **Files:** `src/database/migrations/2025_12_11_124201_add_default_user_to_users_table.php` line 12-13 +- **Impact:** If this migration runs in production, a trivially guessable admin account is created. There is no mechanism to force a password change. +- **Fix:** Use database seeders instead of migrations for seed data. Generate a random password from env vars. Add forced password change on first login. + +### MEDIUM: Admin Middleware Uses Loosely-Typed Access Level Check +- **Risk:** `AdminMiddleware` checks `Auth::user()->access_level !== 1` using strict comparison but the database column is `integer` and the middleware does not verify the user model loaded correctly. +- **Files:** `src/app/Http/Middleware/AdminMiddleware.php` line 26 +- **Impact:** The access_level field is not constrained by any foreign key or enum — any integer value is accepted in `AdminController::storeUser()`. +- **Fix:** Use a proper role/permission system (e.g., `spatie/laravel-permission`). At minimum, validate `access_level` is in `[1, 2]` range. + +### MEDIUM: No Foreign Key Constraints on Data Tables +- **Risk:** The `rainfall`, `waterlevel`, `notification`, and `siren` tables all have `stationid` columns but no foreign key constraints to `station.stationid`. Data integrity cannot be enforced at the database level. +- **Files:** All migrations in `src/database/migrations/2025_11_*` +- **Impact:** Orphaned records possible if stations are deleted. `AdminController::deleteStation()` (line 405) deletes a station without cleaning related data. +- **Fix:** Add foreign key constraints with cascade delete to all child tables referencing `station.stationid`. + +## Technical Debt + +### No Eloquent Models for Domain Entities +- **Issue:** The entire application uses `DB::table()` and raw `DB::select()` instead of Eloquent ORM. The only Eloquent model is `User`. Station, Rainfall, WaterLevel, Notification, and Siren have no models. +- **Files:** All controllers in `src/app/Http/Controllers/` +- **Impact:** No model-level validation, no relationships, no accessors/mutators, no model events. Code is harder to test and maintain. +- **Fix:** Create Eloquent models for `Station`, `Rainfall`, `WaterLevel`, `Notification`, `Siren`. Define relationships and refactor controllers. + +### Dashboard Query Duplicated Across Three Controllers +- **Issue:** The same complex 4-table LEFT JOIN query (station + rainfall + waterlevel + siren) is copy-pasted in `MapController::getCurrentData()`, `AuthenticatedSessionController::create()`, and `Api\StationController::getCurrentData()`. +- **Files:** + - `src/app/Http/Controllers/MapController.php` lines 35-63 + - `src/app/Http/Controllers/Auth/AuthenticatedSessionController.php` lines 28-56 + - `src/app/Http/Controllers/Api/StationController.php` lines 13-43 +- **Impact:** Any query change must be made in three places. Risk of divergence. +- **Fix:** Extract to a shared repository/service class or a dedicated model scope. + +### `AdminController` is a 427-line God Controller +- **Issue:** `AdminController` handles both station management and user management — two distinct domains. Contains 8 methods for CRUD on 2 different entities. +- **Files:** `src/app/Http/Controllers/AdminController.php` (427 lines) +- **Impact:** Hard to maintain. Violates Single Responsibility Principle. +- **Fix:** Split into `Admin\StationController` and `Admin\UserController`. + +### `.env.example` Mismatch with Actual Configuration +- **Issue:** `src/.env.example` is the default Laravel template with `DB_CONNECTION=sqlite`, but the application uses PostgreSQL via Docker. The Firebase env vars (`FIREBASE_PROJECT_ID`, `FIREBASE_CREDENTIALS`, `FCM_TOPIC_RAINFALL_WARNING`) are not documented in any `.env.example`. +- **Files:** `src/.env.example` (65 lines, default Laravel template) +- **Impact:** New developers cannot set up the project without reverse-engineering the code. +- **Fix:** Update `.env.example` with actual PostgreSQL connection details and all custom env vars. + +### Commented-Out Code Blocks +- **Issue:** Several files contain large blocks of commented-out code suggesting incomplete or abandoned features. +- **Files:** + - `src/app/Http/Controllers/RainfallController.php` lines 250-263: commented-out alternative graph query + - `autoscript/sidesdecode.py` lines 153-158, 489, 522-533: commented-out error handling and test functions + - `src/routes/web.php` line 30: commented-out threshold graph route + - `src/app/Services/FcmService.php` lines 27-30: commented-out alternative auth method +- **Fix:** Remove dead code. Use version control for history. + +## Performance Concerns + +### N+1 Subqueries in Station Data Queries +- **Issue:** The dashboard query uses correlated subqueries `(SELECT MAX(timestamp) FROM rainfall WHERE stationid = s.stationid)` for each of the 3 LEFT JOINs. For N stations, this results in ~3N additional subquery evaluations. +- **Files:** + - `src/app/Http/Controllers/MapController.php` lines 37-46 + - `src/app/Http/Controllers/Api/StationController.php` lines 14-24 + - `src/app/Http/Controllers/Auth/AuthenticatedSessionController.php` lines 30-39 +- **Impact:** Query performance degrades linearly with station count. +- **Fix:** Use window functions (`ROW_NUMBER() OVER PARTITION BY`) or pre-computed materialized views for latest readings. + +### No Database Indexes on Query-Critical Columns +- **Issue:** None of the migrations create indexes on columns used in WHERE, JOIN, or ORDER BY clauses. +- **Missing indexes:** + - `rainfall.stationid` and `rainfall.timestamp` — used in every rainfall query + - `waterlevel.stationid` and `waterlevel.datetime` — used in every waterlevel query + - `notification.stationid`, `notification.timestamp`, `notification.stationtype` — used in all notification queries + - `siren.stationid` and `siren.active_time` — used in all siren queries +- **Files:** `src/database/migrations/2025_11_06_*` and `2025_11_07_*` +- **Impact:** Full table scans on every query. Performance will degrade significantly as data accumulates. +- **Fix:** Add index migrations for all foreign key columns and frequently queried timestamp columns. + +### No Caching for Dashboard/Map Data +- **Issue:** The dashboard query (the heaviest query in the app) runs on every page load with no caching. The same data powers the home page, login page, and dashboard. +- **Files:** `src/app/Http/Controllers/MapController.php`, `src/app/Http/Controllers/Auth/AuthenticatedSessionController.php` +- **Impact:** Database hit on every request for data that changes at most every few minutes. +- **Fix:** Cache dashboard data for 1-5 minutes using `Cache::remember()`. + +### Heavy PDF Export Queries Without Pagination +- **Issue:** PDF export endpoints (`exportHistoryRfPDF`, `exportHistoryWlPDF`, `exportHistorySirenPDF`) load entire unbounded result sets with `->get()`. +- **Files:** + - `src/app/Http/Controllers/NotificationController.php` lines 122-126, 143-147 + - `src/app/Http/Controllers/SirenController.php` lines 67-74 +- **Impact:** Memory exhaustion with large datasets. No date filtering or row limits. +- **Fix:** Add date range filters and chunk processing for PDF generation. + +## Maintainability Concerns + +### Inconsistent Naming Conventions +- **Issue:** Controller names mix PascalCase and camelCase. The `cctvController` uses lowercase class name violating PSR-1/PSR-12. +- **Files:** + - `src/app/Http/Controllers/cctvController.php` — class `cctvController` (should be `CctvController`) + - `src/app/Http/Controllers/SirenController.php` method `SirenHistory()` — PascalCase method (should be `sirenHistory`) + - `src/app/Http/Controllers/NotificationController.php` method `SirenNotification()` — same issue +- **Fix:** Rename to follow PSR standards: `CctvController`, camelCase methods. + +### Typo: "potrait" Instead of "portrait" in PDF Generation +- **Issue:** Three PDF export methods pass `'potrait'` as the paper orientation. While DomPDF may not error, this is an invalid value that likely falls back to default. +- **Files:** + - `src/app/Http/Controllers/SirenController.php` line 77 + - `src/app/Http/Controllers/NotificationController.php` lines 129, 150 +- **Fix:** Change `'potrait'` to `'portrait'`. + +### No Test Coverage for Custom Application Logic +- **Issue:** Test files only contain Breeze defaults (auth tests). Zero tests cover custom controllers, API endpoints, or the data processing script. +- **Files:** `src/tests/` — only default Laravel/Breeze tests exist +- **Impact:** Any refactoring or fix risks introducing regressions with no safety net. +- **Fix:** Add feature tests for all API endpoints, admin CRUD operations, and PDF export endpoints. + +## Dependency Concerns + +### `google/auth` Used Directly Instead of Official Firebase SDK +- **Issue:** The project uses `google/auth` package directly with manual HTTP calls to FCM instead of the official `kreait/laravel-firebase` SDK. +- **Files:** `src/app/Services/FcmService.php`, `src/composer.json` line 12: `"google/auth": "^1.49"` +- **Impact:** More boilerplate, manual token management, no automatic retry or error handling. +- **Fix:** Replace with `kreait/laravel-firebase` for cleaner integration. + +### Docker Image Not Pinned, `composer:2.3` Outdated +- **Issue:** `Dockerfile` copies Composer from `composer:2.3` image (line 62) which is significantly behind current stable. `postgres` image in `docker-compose.yml` has no version tag (line 26: `image: postgres`). +- **Files:** `Dockerfile` line 62, `docker-compose.yml` line 26 +- **Impact:** Unpredictable builds. Breaking changes when pulling latest postgres image. +- **Fix:** Pin Composer to `composer:2` or latest. Pin Postgres to a specific major version (e.g., `postgres:16-alpine`). + +## Infrastructure Concerns + +### PostgreSQL Data Exposed on Port 5432 +- **Issue:** `docker-compose.yml` maps Postgres port 5432 directly to the host. Both pgAdmin (5050) and Adminer (6060) are also exposed in what appears to be a production configuration. +- **Files:** `docker-compose.yml` lines 34-35, 61, 74 +- **Impact:** Database and management tools accessible from the network without additional protection. +- **Fix:** Remove port mappings for production. Use Docker networks only. Adminer should not be in production compose. + +### No Health Checks in Docker Services +- **Issue:** None of the Docker services define `healthcheck` blocks. The `app` service uses `depends_on: postgres` without a health condition, so it may start before Postgres is ready. +- **Files:** `docker-compose.yml` +- **Fix:** Add health checks for `postgres` and configure `app` with `depends_on.postgres.condition: service_healthy`. + +### `src/` Directory Volume-Mounted but Also COPY'd in Dockerfile +- **Issue:** `Dockerfile` COPYs `src/` into the image (lines 72-75), but `docker-compose.yml` also mounts `./src:/var/www/html` as a volume (line 16). The COPY is wasted and creates confusion. +- **Files:** `Dockerfile` lines 72-75, `docker-compose.yml` line 16 +- **Fix:** Remove COPY from Dockerfile for development (volume mount handles it). Create a separate production Dockerfile that COPYs code. + +## Observability + +### No Structured Logging +- **Issue:** The application uses Laravel's default file logging (`LOG_CHANNEL=stack`, `LOG_STACK=single`). No centralized logging, no log levels differentiated by environment. +- **Files:** `src/.env.example` lines 18-21 +- **Impact:** Logs are local to the container and lost on restart. No way to search or alert on errors. +- **Fix:** Configure `LOG_CHANNEL=stderr` for Docker, or integrate with a log aggregation service. + +### No Error Tracking Service +- **Issue:** No error tracking integration (Sentry, Bugsnag, etc.) detected. Exceptions are caught locally and returned as flash messages in some controllers but silently swallowed in others. +- **Files:** `src/app/Http/Controllers/AdminController.php` lines 216-218: `$e->getMessage()` returned to user. +- **Impact:** Production errors invisible to developers. Raw exception messages may leak internal details to users. +- **Fix:** Integrate an error tracking service. Never return raw `$e->getMessage()` to end users. + +### No Monitoring or Alerting +- **Issue:** No monitoring tools, uptime checks, or alerting configuration detected. The application has a `/up` health endpoint (from Laravel 12) but nothing monitors it. +- **Impact:** Downtime or degraded performance goes undetected until users report it. +- **Fix:** Set up uptime monitoring for the `/up` endpoint. Add APM for performance tracking. + +--- + +*Concerns audit: 2026-05-20* diff --git a/.planning/codebase/CONVENTIONS.md b/.planning/codebase/CONVENTIONS.md new file mode 100644 index 00000000..746edc57 --- /dev/null +++ b/.planning/codebase/CONVENTIONS.md @@ -0,0 +1,188 @@ +# Coding Conventions + +**Analysis Date:** 2026-05-20 + +## Naming Conventions + +**Files:** +- Controllers: PascalCase (e.g., `SirenController.php`, `AdminController.php`, `MapController.php`) + - **Exception:** `cctvController.php` uses lowercase — inconsistent with project convention +- Models: PascalCase singular (e.g., `User.php`) +- Exports: PascalCase (e.g., `WaterLevelExport.php`, `HourlyRainfallExport.php`) +- Notifications: PascalCase (e.g., `ResetPasswordNotification.php`) +- Services: PascalCase (e.g., `FcmService.php`) +- Middleware: PascalCase (e.g., `AdminMiddleware.php`, `LocalizationMiddleware.php`) +- Form Requests: PascalCase (e.g., `ProfileUpdateRequest.php`, `LoginRequest.php`) +- Migrations: `YYYY_MM_DD_HHMMSS_description_snake_case.php` +- Blade views: `snake_case` directories + `kebab-case` or `snake_case` files (e.g., `layout/notification/rainfall.blade.php`) +- JavaScript: `camelCase.js` (e.g., `homemap.js`, `graph.js`, `rfhistory.js`) + +**Classes:** +- PascalCase for all classes, following PSR-4 autoloading +- Controllers suffixed with `Controller` (e.g., `RainfallController`) +- Exports suffixed with `Export` (e.g., `HourlyRainfallExport`) + +**Methods:** +- camelCase (e.g., `stationDisplay()`, `rainfallSum()`, `exportHistoricalWl()`) + - **Exception:** `SirenHistory()` uses PascalCase — inconsistent +- Public methods in controllers follow action naming: `index()`, `store()`, `update()`, `destroy()`, `edit()` + +**Variables:** +- camelCase for local variables (e.g., `$stationFilter`, `$rainfallData`, `$displayDate`) +- camelCase for class properties (e.g., `$stationid`, `$startDate` in Export classes) +- `$stationid` used as one word (not `$stationId`) — project-specific convention + +**Database Tables:** +- snake_case lowercase singular (e.g., `station`, `rainfall`, `waterlevel`, `siren`, `notification`, `users`) +- `users` is the only plural table name + +**Columns:** +- snake_case lowercase (e.g., `stationid`, `active_time`, `access_level`, `login_attempts`, `is_blocked`, `cctv_link`) +- No consistent convention: some use compound words without underscore (e.g., `stationid`, `waterlevel`) + +## Code Style + +**Formatting:** +- `.editorconfig` present at both project root and `src/` +- Indent: 4 spaces (no tabs, except Makefiles) +- Charset: UTF-8 +- End of line: LF +- Trailing whitespace trimmed (except `.md` files) +- YAML files: 2-space indent +- No ESLint or Prettier configured for JavaScript + +**Linting:** +- `laravel/pint` installed as dev dependency (PHP code style fixer) +- No evidence of a `pint.json` configuration file — using defaults +- No frontend linting tools configured + +**Bracket Style:** +- Opening braces on same line for classes and methods +- Control structures: opening brace on same line (K&R style) +- Multi-line function calls and arrays: trailing comma not consistently used + +**Import/Use Statement Organization:** +- Laravel framework imports first +- Then third-party packages (e.g., `Carbon`, `Maatwebsite\Excel`, `DomPDF`) +- Then application imports (e.g., `App\Models\User`, `App\Services\FcmService`) +- Blank line between groups is not consistently applied +- Example from `WaterLevelController.php`: + ```php + use Illuminate\Http\Request; + use Illuminate\Support\Facades\DB; + use Carbon\Carbon; + // Add this for Export data to excel + use App\Exports\WaterLevelExport; + use Maatwebsite\Excel\Facades\Excel; + ``` + +**Docblock Usage:** +- PHPDoc blocks present on model properties (`$fillable`, `$hidden`, `casts()`) +- Docblocks on factory methods and notification methods (Breeze scaffolded) +- Custom controllers use inline `//` comments extensively rather than docblocks +- Comment style in controllers follows a pattern: + ```php + // Function Retrieve Current Water Level Data + public function index(Request $request) + ``` +- Each SQL query is annotated with block comments: + ```php + // TABLE : STATION JOIN TABLE WATERLEVEL + // COLUMN : name,datetime,waterlevel + // INPUT : $stationCondition from resources/views/... + // OUTPUT : resources/views/layout/waterlevel.blade.php + ``` + +## File Organization + +**How files are grouped:** +- Controllers grouped by domain in flat structure under `app/Http/Controllers/` +- API controllers in `app/Http/Controllers/Api/` subdirectory +- Auth controllers in `app/Http/Controllers/Auth/` subdirectory (Breeze scaffolded) +- Exports in `app/Exports/` +- Services in `app/Services/` +- Notifications in `app/Notifications/` +- Views follow a `layout.{domain}` naming pattern (e.g., `layout.rainfall`, `layout.admin.stationmgmt`) + +**One class per file:** +- Adhered to throughout — one class per file + +**Directory naming:** +- All lowercase directories (e.g., `Controllers/`, `Models/`, `Services/`, `Exports/`) +- Views use nested lowercase directories (e.g., `layout/notification/history/`) + +## Git Conventions + +**Branch naming:** +- Only 2 commits in the repository — no discernible branch naming convention + +**Commit message patterns:** +- Observed messages: + - `first commit` + - `fix: configuration docker-compose.yml` +- The second commit suggests a possible `type: description` convention (fix:, feat:, etc.) + +## Laravel-Specific Patterns + +**Eloquent Model Patterns:** +- Only `User.php` exists as an Eloquent model (`app/Models/User.php`) +- `HasFactory` and `Notifiable` traits used on User +- `$fillable` array for mass assignment (not `$guarded`) +- `casts()` method (Laravel 12 style) instead of `$casts` property +- Custom notification override: `sendPasswordResetNotification()` + +**Controller Patterns:** +- **Not resource controllers** — methods are custom-named (e.g., `stationDisplay()`, `userDisplay()`, `storeStation()`) +- Heavy use of raw SQL via `DB::select()` and `DB::table()` instead of Eloquent ORM +- `collect(DB::select(...))` pattern to wrap raw SQL results into collections +- Controller validation done inline with `$request->validate()` — not consistently using Form Requests +- Redirect pattern: `redirect()->back()->with('success', __('toast.key'))` +- View compact pattern: `return view('blade.path', compact('var1', 'var2'))` + +**Form Request Usage:** +- `ProfileUpdateRequest` for profile updates (`app/Http/Requests/ProfileUpdateRequest.php`) +- `LoginRequest` for authentication (`app/Http/Requests/Auth/LoginRequest.php`) +- Admin controller does inline `$request->validate()` instead of Form Requests +- **Inconsistent:** Form Requests exist but most validation is inline + +**Resource/API Resource Usage:** +- Not used — API responses use `response()->json($data)` directly +- No API Resource classes defined + +**Policy Usage:** +- Not used — authorization handled via custom `AdminMiddleware` checking `access_level` +- `app/Http/Middleware/AdminMiddleware.php` checks `Auth::user()->access_level !== 1` + +**Event/Listener Patterns:** +- Not used — no events or listeners defined +- Notification push (FCM) is called directly from API controller + +**Route Patterns:** +- Named routes used consistently: `->name('rainfall')`, `->name('stationmanagement.store')` +- Route grouping by middleware: `Route::middleware('auth')`, `Route::middleware(['admin'])` +- Admin routes use POST for updates (not PUT/PATCH) — non-RESTful +- API routes in `routes/api.php` with no auth middleware + +**Localization:** +- Bilingual: English (`en`) and Bahasa Malaysia (`bm`) in `resources/lang/` +- `LocalizationMiddleware` sets locale via session +- `__('key')` helper used for translations in views and controllers + +**Middleware:** +- Custom middleware registered in `bootstrap/app.php` +- `AdminMiddleware` for admin-only routes +- `LocalizationMiddleware` for language switching + +**Export Pattern (Maatwebsite Excel):** +- Export classes implement `FromCollection`, `WithHeadings`, `ShouldAutoSize` +- Constructor injection for filter parameters +- Raw SQL queries in `collection()` method +- `headings()` method returns array of column names + +**PDF Export Pattern (DomPDF):** +- `Pdf::loadView('blade.path', compact('data'))->setPaper('a4','potrait')` +- Note: `'potrait'` is a typo — should be `'portrait'` (present in multiple files) + +--- + +*Convention analysis: 2026-05-20* diff --git a/.planning/codebase/INTEGRATIONS.md b/.planning/codebase/INTEGRATIONS.md new file mode 100644 index 00000000..7b0ddce2 --- /dev/null +++ b/.planning/codebase/INTEGRATIONS.md @@ -0,0 +1,155 @@ +# External Integrations + +**Analysis Date:** 2026-05-20 + +## External Services + +### Firebase Cloud Messaging (FCM) +- **Purpose:** Push notifications to mobile/web clients when sensor thresholds are triggered +- **SDK/Client:** `google/auth` ^1.49 (Google Auth for OAuth2 token generation) +- **Implementation:** `src/app/Services/FcmService.php` + - Authenticates via Service Account Credentials (`FIREBASE_CREDENTIALS` env → JSON file) + - Sends topic-based notifications to `FCM_TOPIC_RAINFALL_WARNING` + - Calls FCM HTTP v1 API: `https://fcm.googleapis.com/v1/projects/{projectId}/messages:send` +- **Triggered by:** `src/app/Http/Controllers/Api/AlertController.php` (API endpoint `POST /api/alert`) +- **Auth env vars:** `FIREBASE_PROJECT_ID`, `FIREBASE_CREDENTIALS`, `FCM_TOPIC_RAINFALL_WARNING` + +### FTP Server (myvscada.com) +- **Purpose:** Retrieving raw sensor data files (CSV) from remote SCADA FTP server +- **Implementation:** `autoscript/sidesdecode.py` + - FTP server: `myvscada.com`, credentials hardcoded in script + - Folder structure: `files/SIDES/SUCCESS/{year}/{month}/{day}/` + - Error folder: `files/SIDES/ERROR/` + - Processes CSV files containing rainfall, water level, siren, and battery data + +### SIDES API (Internal Callback) +- **Purpose:** Alert notification trigger from ETL script to Laravel API +- **Implementation:** `autoscript/sidesdecode.py` → `send_alert_to_laravel()` + - Posts to `https://sides.tck.com.my/api/alert` + - Payload: `{ stationid, level, stationtype }` + - Received by: `src/app/Http/Controllers/Api/AlertController.php` + +## Third-Party Libraries + +### Google Auth (`google/auth` ^1.49) +- **Connects to:** Google OAuth2 for Firebase service account authentication +- **Used in:** `src/app/Services/FcmService.php` +- **Purpose:** Generates access tokens for FCM HTTP v1 API calls + +### Laravel DomPDF (`barryvdh/laravel-dompdf` ^3.1) +- **Connects to:** N/A (local PDF generation) +- **Used in:** Siren and notification history PDF exports +- **Purpose:** Generates downloadable PDF reports from Blade templates + +### Maatwebsite Excel (`maatwebsite/excel` ^3.1) +- **Connects to:** N/A (local Excel generation) +- **Used in:** `src/app/Exports/HourlyRainfallExport.php`, `src/app/Exports/WaterLevelExport.php` +- **Purpose:** Export historical rainfall and water level data to Excel (.xlsx) + +### psycopg2 (Python) +- **Connects to:** PostgreSQL database directly (bypasses Laravel) +- **Used in:** `autoscript/sidesdecode.py` +- **Purpose:** Direct database insertion of sensor data from decoded CSV files + +## Internal Services + +### API Routes (REST) +- **Base path:** `/api/` +- **Defined in:** `src/routes/api.php` +- **Endpoints:** + - `GET /api/station/current` — Current station data (rainfall, water level, siren) + - `GET /api/station/rainfall` — Rainfall-specific data + - `GET /api/station/waterlevel` — Water level-specific data + - `GET /api/station/notification` — Current day notifications + - `GET /api/station/history` — 3-day notification history + - `GET /api/station/siren` — Latest siren status + - `GET /api/station/siren/history` — 3-day siren history + - `POST /api/login` — Mobile/API authentication + - `POST /api/alert` — Trigger FCM push notification + +### Python ETL Script +- **Location:** `autoscript/sidesdecode.py` +- **Flow:** FTP download → CSV decode → PostgreSQL insert → API alert trigger +- **Data types:** Rainfall, water level, siren status, battery levels +- **Threshold logic:** Rainfall hourly ≥30mm (Warning), ≥60mm (Danger); Water level alert/warning/danger from CSV config +- **Logs:** `autoscript/sidesdecode.log`, `autoscript/sidesdecode_error.log` + +## Authentication Providers + +### Web Authentication (Session-based) +- **Provider:** Laravel Breeze (`laravel/breeze ^2.3`) +- **Guard:** Session-based (`web` guard, `session` driver) +- **User model:** `App\Models\User` (`src/app/Models/User.php`) +- **Features:** Login, registration, email verification, password reset, password confirmation +- **Role system:** `access_level` field on users table (1 = admin, 0 = regular) +- **Account security:** `is_blocked` and `login_attempts` fields on users table +- **Implementation:** `src/routes/auth.php`, `src/app/Http/Controllers/Auth/` + +### API Authentication +- **Provider:** Custom token-based (via `POST /api/login`) +- **Implementation:** `src/app/Http/Controllers/Api/AuthController.php` +- **Used by:** Mobile client or external data consumers + +### Password Reset +- **Custom notification:** `src/app/Notifications/ResetPasswordNotification.php` +- **Channel:** Email (SMTP) +- **Branded:** "SIDES - Password Reset" subject line + +## Payment / Billing + +Not applicable — This is an environmental monitoring dashboard, no payment integration. + +## Email / Notifications + +### Email +- **Default mailer:** `log` (writes to Laravel log, not actually sent in current config) — `src/config/mail.php` +- **Configured transports:** SMTP, SES, Postmark, Resend, Sendmail, Log, Array +- **Env vars:** `MAIL_MAILER`, `MAIL_HOST`, `MAIL_PORT`, `MAIL_USERNAME`, `MAIL_PASSWORD` +- **From address:** Configurable via `MAIL_FROM_ADDRESS` and `MAIL_FROM_NAME` + +### Push Notifications +- **Firebase Cloud Messaging (FCM)** — Topic-based push notifications +- **Implementation:** `src/app/Services/FcmService.php` +- **Trigger:** Sensor threshold breaches detected by `autoscript/sidesdecode.py` + +### In-App Notifications +- **Notification views:** `src/resources/views/` — rainfall, water level, siren notification pages +- **Data source:** `notification` table in PostgreSQL + +## File Storage + +### Primary Storage +- **Default disk:** `local` — `src/storage/app/private/` (`src/config/filesystems.php`) +- **Public disk:** `src/storage/app/public/` — symlinked to `public/storage` +- **No cloud storage configured** — S3 config present but not active + +### FTP File Management +- **FTP server:** `myvscada.com` — remote sensor data storage +- **File lifecycle:** SUCCESS folder → processed → optional move to SUCCESS/ERROR subfolders + +## Queue / Background Jobs + +### Queue Configuration +- **Default driver:** `database` (`src/config/queue.php`) +- **Table:** `jobs` (database-backed queue) +- **Failed jobs:** `failed_jobs` table, driver `database-uuids` +- **Job batching:** `job_batches` table +- **Run command:** `php artisan queue:listen --tries=1` (via `composer dev`) +- **Available drivers:** sync, database, beanstalkd, SQS, redis (not actively used beyond database) + +## Caching + +### Cache Configuration +- **Default store:** `database` (`src/config/cache.php`) +- **Table:** `cache` (database-backed cache) +- **Available stores:** array, database, file, memcached, redis, dynamodb, octane, failover +- **Redis:** Configured but not actively used (`src/config/database.php` has Redis connection block) +- **Session driver:** `database` (sessions table) + +## Search + +No dedicated search engine integration (Elasticsearch, Algolia, etc.). All queries use direct PostgreSQL with raw SQL joins and subqueries. + +--- + +*Integration audit: 2026-05-20* diff --git a/.planning/codebase/STACK.md b/.planning/codebase/STACK.md new file mode 100644 index 00000000..f020dc61 --- /dev/null +++ b/.planning/codebase/STACK.md @@ -0,0 +1,144 @@ +# Technology Stack + +**Analysis Date:** 2026-05-20 + +## Runtime & Language Versions + +**Primary:** +- PHP 8.2 — Backend application logic, served via PHP-FPM (`src/`) +- Composer 2.3 — PHP dependency management (installed in Dockerfile) + +**Secondary:** +- Python 3 — Data ingestion/ETL script (`autoscript/sidesdecode.py`) +- JavaScript (ES Modules) — Frontend interactivity (`src/resources/js/`) +- Node.js LTS — Frontend build toolchain (installed in Dockerfile) +- SQL (PostgreSQL dialect) — Raw queries in controllers and exports + +## Backend Framework + +**Core:** +- Laravel 12.0 (`laravel/framework ^12.0`) — Full-stack MVC framework +- Laravel Breeze 2.3 (`laravel/breeze ^2.3`) — Auth scaffolding (login, register, password reset) +- PHP 8.2 constraint (`"php": "^8.2"` in `src/composer.json`) + +**Key Backend Dependencies:** + +| Package | Version | Purpose | +|---------|---------|---------| +| `laravel/framework` | ^12.0 | Core framework (routing, Eloquent, middleware, etc.) | +| `laravel/tinker` | ^2.10.1 | Interactive REPL for artisan | +| `barryvdh/laravel-dompdf` | ^3.1 | PDF generation (siren/water level history reports) | +| `maatwebsite/excel` | ^3.1 | Excel export (rainfall/water level historical data) | +| `google/auth` | ^1.49 | Google OAuth2 for Firebase Cloud Messaging auth | +| `laravel/pail` | ^1.2.2 | Real-time log tailing via CLI | +| `laravel/sail` | ^1.41 | Docker dev environment (installed but using custom Docker) | + +**Dev Dependencies:** + +| Package | Version | Purpose | +|---------|---------|---------| +| `phpunit/phpunit` | ^11.5.3 | Testing framework | +| `fakerphp/faker` | ^1.23 | Test data generation | +| `mockery/mockery` | ^1.6 | Mocking framework | +| `laravel/pint` | ^1.24 | Code style fixer | +| `nunomaduro/collision` | ^8.6 | Error page rendering | + +**Middleware Stack** (registered in `src/bootstrap/app.php`): +- Standard Laravel web middleware group +- `App\Http\Middleware\LocalizationMiddleware` — Session-based locale (en/bm) +- `App\Http\Middleware\AdminMiddleware` — Role-based access (access_level === 1) +- `admin` alias registered for route middleware + +## Frontend Framework + +**Core:** +- Alpine.js 3.4 — Lightweight reactive UI framework (`src/resources/js/app.js`) +- Blade — Server-side templating engine (all views in `src/resources/views/`) +- Axios 1.11 — HTTP client for AJAX (`src/resources/js/bootstrap.js`) + +**Build Tools:** +- Vite 7.0.7 — Build tool and dev server (`src/vite.config.js`) +- `laravel-vite-plugin` ^2.0.0 — Laravel integration for Vite +- `concurrently` ^9.0.1 — Parallel dev server orchestration + +**CSS Framework:** +- Tailwind CSS 3.1 — Utility-first CSS (`src/tailwind.config.js`) +- `@tailwindcss/forms` ^0.5.2 — Form element styling plugin +- `@tailwindcss/vite` ^4.0.0 — Vite plugin for Tailwind v4 compat +- PostCSS with Autoprefixer (`src/postcss.config.js`) + +**Frontend Dependencies:** + +| Package | Version | Purpose | +|---------|---------|---------| +| `alpinejs` | ^3.4.2 | Reactive UI bindings | +| `axios` | ^1.11.0 | HTTP requests | +| `flatpickr` | ^4.6.13 | Date/time picker widget | + +**Vite Entry Points** (`src/vite.config.js`): +- `resources/css/app.css` — Tailwind base/styles +- `resources/js/app.js` — Alpine.js bootstrap +- `resources/css/style.css` — Custom styles +- `resources/js/script.js` — Custom JavaScript + +## Database + +**Primary:** +- PostgreSQL — Production database (via Docker container `tckdev-db`) + - Host: `postgres` container (internal), port 5432 + - Connection configured via `DB_*` env vars in `src/.env` + - Driver: `pgsql` with `pdo_pgsql` PHP extension + +**Development/Testing:** +- SQLite — Default fallback and test database (`src/config/database.php` default: `sqlite`) +- In-memory SQLite for PHPUnit tests (`src/phpunit.xml`: `DB_DATABASE=:memory:`) + +**ORM / Query Builder:** +- Eloquent ORM — Used for User model (`src/app/Models/User.php`) +- Raw SQL queries — Extensively used via `DB::select()` and `DB::table()` joins in controllers + - `src/app/Http/Controllers/Api/StationController.php` — Complex joins with subqueries + - `src/app/Exports/HourlyRainfallExport.php` — CTEs with dynamic column generation + - `src/app/Exports/WaterLevelExport.php` — Parameterized raw SQL + +**Migration System:** +- Laravel Migrations (`src/database/migrations/`) +- Key tables: `users`, `station`, `rainfall`, `waterlevel`, `siren`, `notification`, `cache`, `jobs` + +## DevOps & Infrastructure + +**Containerization:** +- Docker Compose 3.9 (`docker-compose.yml`) — 5 services: + - `app` — PHP 8.2-FPM (custom Dockerfile, `php:8.2-fpm` base) + - `web` — Nginx stable-alpine (reverse proxy to PHP-FPM) + - `postgres` — PostgreSQL database + - `pgadmin` — pgAdmin4 database management UI (port 5050) + - `adminer` — Adminer lightweight DB management (port 6060) + +**Web Server:** +- Nginx stable-alpine (`docker/nginx/default.conf`) + - FastCGI pass to `app:9000` (PHP-FPM) + - Security headers: X-Frame-Options, X-XSS-Protection, X-Content-Type-Options + +**Process Manager:** +- PHP-FPM 8.2 — Application server inside Docker + +**Build/Task Runner:** +- Makefile — Docker orchestration shortcuts (up, build, init, migrate, test, etc.) +- Composer scripts — `dev` (concurrent: serve, queue, pail, vite), `setup`, `test` + +**No CI/CD detected** — No CI pipeline configuration files found. + +## Key Libraries + +| Library | Purpose | File References | +|---------|---------|-----------------| +| `google/auth` | Firebase OAuth2 token generation for FCM push notifications | `src/app/Services/FcmService.php` | +| `barryvdh/laravel-dompdf` | PDF export for siren/water level/rainfall history reports | `src/app/Http/Controllers/SirenController.php`, `src/app/Http/Controllers/NotificationController.php` | +| `maatwebsite/excel` | Excel export for hourly rainfall and water level data | `src/app/Exports/HourlyRainfallExport.php`, `src/app/Exports/WaterLevelExport.php` | +| `flatpickr` | Date/time picker in historical data views | `src/package.json` | +| `psycopg2` (Python) | PostgreSQL driver for ETL data ingestion script | `autoscript/sidesdecode.py` | +| `ftplib` (Python) | FTP client for retrieving sensor data files | `autoscript/sidesdecode.py` | + +--- + +*Stack analysis: 2026-05-20* diff --git a/.planning/codebase/STRUCTURE.md b/.planning/codebase/STRUCTURE.md new file mode 100644 index 00000000..c0654a4d --- /dev/null +++ b/.planning/codebase/STRUCTURE.md @@ -0,0 +1,280 @@ +# Codebase Structure + +**Analysis Date:** 2026-05-20 + +## Directory Layout + +``` +tckdev/ # Project root +├── autoscript/ # External Python data ingestion script +│ ├── sidesdecode.py # FTP → PostgreSQL data pipeline +│ ├── sidesdecode.log # Script log output +│ └── sidesdecode_error.log # Script error log +├── backup/ # Backup storage +├── docker/ # Docker configuration files +│ ├── nginx/ +│ │ └── default.conf # Nginx reverse proxy config +│ ├── postgres/ +│ │ └── data/ # PostgreSQL data volume (gitignored) +│ └── image/ # Documentation images +├── src/ # Laravel application root +│ ├── app/ # Application code +│ │ ├── Exports/ # Excel export classes +│ │ ├── Http/ +│ │ │ ├── Controllers/ # Web controllers +│ │ │ │ ├── Api/ # API controllers +│ │ │ │ └── Auth/ # Breeze auth controllers +│ │ │ ├── Middleware/ # Custom middleware +│ │ │ └── Requests/ # Form request validation +│ │ ├── Models/ # Eloquent models (only User.php) +│ │ ├── Notifications/ # Laravel notifications +│ │ ├── Providers/ # Service providers +│ │ ├── Services/ # Service classes +│ │ └── View/ # View components (Blade components) +│ ├── bootstrap/ # Framework bootstrap +│ ├── config/ # Configuration files +│ ├── database/ +│ │ ├── migrations/ # Database migrations +│ │ ├── seeders/ # Database seeders +│ │ └── factories/ # Model factories +│ ├── lang/ # Translation files (en, bm) +│ ├── public/ # Web root (document root) +│ ├── resources/ +│ │ ├── css/ # Stylesheets +│ │ ├── js/ # Client-side JavaScript +│ │ └── views/ # Blade templates +│ ├── routes/ # Route definitions +│ ├── storage/ # Logs, cache, compiled views +│ ├── tests/ # Test files +│ └── vendor/ # Composer dependencies +├── tmp/ # Temporary files +├── .env # Environment variables (gitignored) +├── .env.example # Environment template +├── docker-compose.yml # Docker compose configuration +├── Dockerfile # PHP-FPM container definition +├── Makefile # Build/dev automation +├── database.db # SQLite database (local dev) +└── README.md # Project documentation +``` + +## Directory Purposes + +**`autoscript/`:** +- Purpose: External data pipeline that runs outside Laravel +- Contains: Python script for FTP-to-PostgreSQL data ingestion +- Key files: `sidesdecode.py` (main pipeline), log files +- Note: Connects directly to PostgreSQL, not through Laravel + +**`src/app/Http/Controllers/`:** +- Purpose: All web-facing controller logic +- Contains: Domain controllers for rainfall, water level, siren, notifications, admin, CCTV, map, locale +- Key files: `MapController.php`, `RainfallController.php`, `WaterLevelController.php`, `AdminController.php`, `NotificationController.php`, `SirenController.php`, `cctvController.php` + +**`src/app/Http/Controllers/Api/`:** +- Purpose: REST API endpoints for mobile/external consumption +- Contains: Station data, authentication, and alert controllers +- Key files: `StationController.php`, `AuthController.php`, `AlertController.php` + +**`src/app/Http/Controllers/Auth/`:** +- Purpose: Laravel Breeze authentication controllers +- Contains: Login, register, password reset, email verification controllers +- Note: Auto-generated by Breeze, generally should not be modified + +**`src/app/Exports/`:** +- Purpose: Excel export definitions using maatwebsite/excel +- Key files: `HourlyRainfallExport.php`, `WaterLevelExport.php` + +**`src/app/Services/`:** +- Purpose: Reusable service classes +- Key files: `FcmService.php` (Firebase Cloud Messaging) + +**`src/app/Models/`:** +- Purpose: Eloquent ORM models +- Contains: Only `User.php` — all other domain tables (station, rainfall, waterlevel, siren, notification) have no models +- Note: Domain data access uses `DB::select()` and `DB::table()` directly in controllers + +**`src/resources/views/layout/`:** +- Purpose: Main application Blade templates (non-auth pages) +- Contains: Dashboard, rainfall, water level, siren, threshold, historical views, admin views, notification views, CCTV +- Key files: `dashboard.blade.php`, `rainfall.blade.php`, `waterlevel.blade.php`, `siren/home.blade.php`, `threshold.blade.php` + +**`src/resources/views/layout/admin/`:** +- Purpose: Admin-specific views +- Key files: `stationmgmt.blade.php`, `usermgmt.blade.php` + +**`src/resources/views/layout/notification/`:** +- Purpose: Notification display views +- Contains: Current alarm views and history views for rainfall, water level, siren + +**`src/resources/views/pdf/`:** +- Purpose: PDF generation Blade templates +- Key files: `rfhistory.blade.php`, `wlhistory.blade.php`, `sirenhistory.blade.php` + +**`src/resources/views/auth/`:** +- Purpose: Breeze authentication views +- Contains: Login, register, password reset, email verification + +**`src/resources/js/`:** +- Purpose: Client-side JavaScript +- Key files: `app.js`, `bootstrap.js`, `script.js` (map interactions), `graph.js` (chart rendering), `rfhistory.js` (rainfall history), `homemap.js` (home page map), `homemap.js` + +**`src/resources/css/`:** +- Purpose: Stylesheets +- Key files: `app.css`, `style.css` + +**`src/lang/`:** +- Purpose: Translation files for i18n +- Contains: `en/` (English) and `bm/` (Bahasa Melayu) subdirectories + +**`src/database/migrations/`:** +- Purpose: Database schema definitions +- Contains: 13 migration files defining all tables + +**`docker/`:** +- Purpose: Docker infrastructure configuration +- Contains: Nginx config (`nginx/default.conf`), PostgreSQL data volume, documentation images + +## Key File Locations + +**Entry Points:** +- `src/public/index.php`: Laravel web entry point +- `src/artisan`: CLI command entry point +- `autoscript/sidesdecode.py`: External data pipeline entry point + +**Routing:** +- `src/routes/web.php`: Web routes (authenticated + public + admin) +- `src/routes/api.php`: API routes (no auth middleware) +- `src/routes/auth.php`: Breeze authentication routes +- `src/routes/console.php`: Artisan command definitions + +**Middleware:** +- `src/app/Http/Middleware/AdminMiddleware.php`: Admin access control (access_level === 1) +- `src/app/Http/Middleware/LocalizationMiddleware.php`: Session-based locale setting + +**Bootstrap / App Config:** +- `src/bootstrap/app.php`: Application bootstrap, middleware registration, routing +- `src/app/Providers/AppServiceProvider.php`: Forces HTTPS scheme globally + +**Configuration:** +- `src/config/database.php`: Database connection config (supports SQLite, MySQL, PostgreSQL) +- `src/config/app.php`: Application config +- `src/config/services.php`: Third-party service config (FCM, etc.) + +**Core Logic:** +- `src/app/Http/Controllers/MapController.php`: Dashboard / home page data aggregation +- `src/app/Http/Controllers/RainfallController.php`: Rainfall display + IDF threshold + export (423 lines, largest controller) +- `src/app/Http/Controllers/AdminController.php`: Station & user CRUD (427 lines) +- `src/app/Http/Controllers/Api/StationController.php`: API data endpoints + +**Testing:** +- `src/tests/`: Test directory (PHPUnit) +- `src/phpunit.xml`: PHPUnit configuration + +## Entry Points + +**Web Entry Points:** +- `/` or `/dashboard` → Map dashboard (public) +- `/home` → Home page (auth required) +- `/stations` → Station list JSON (public) +- `/login`, `/register` → Breeze auth pages + +**API Entry Points:** +- `GET /api/station/current` → Current station data +- `GET /api/station/rainfall` → Current rainfall data +- `GET /api/station/waterlevel` → Current water level data +- `GET /api/station/notification` → Current notifications +- `GET /api/station/history` → Notification history (3 days) +- `GET /api/station/siren` → Current siren data +- `GET /api/station/siren/history` → Siren history +- `POST /api/login` → API authentication +- `POST /api/alert` → Trigger FCM push notification + +**Admin Entry Points (admin middleware):** +- `/stationmanagement` → Station CRUD +- `/usermgmt` → User CRUD + +**External / Scheduled:** +- `autoscript/sidesdecode.py` → FTP polling script (likely cron-triggered) + +## Module / Domain Organization + +The application is organized around a **sensor station monitoring domain** with these domain areas: + +- **Station management**: Core entity representing physical monitoring stations. Controllers: `MapController`, `AdminController`. Table: `station`. +- **Rainfall monitoring**: Rainfall data display, historical queries, IDF threshold analysis, Excel export. Controller: `RainfallController`. Table: `rainfall`. +- **Water level monitoring**: Water level data display, historical queries, Excel export. Controller: `WaterLevelController`. Table: `waterlevel`. +- **Siren monitoring**: Siren status, history, PDF export. Controller: `SirenController`. Table: `siren`. +- **Notifications/Alarms**: Threshold-triggered alarm history for rainfall, water level, siren. PDF export. Controller: `NotificationController`. Table: `notification`. +- **Administration**: Station and user management (admin only). Controller: `AdminController`. +- **CCTV**: Station CCTV link display. Controller: `cctvController`. +- **Authentication**: User login, registration, password reset (Breeze). Controllers: `Auth/*`. + +**Namespace pattern:** `App\Http\Controllers\*` for web, `App\Http\Controllers\Api\*` for API. + +**Autoloading:** PSR-4 via Composer — `App\` → `src/app/`, `Database\` → `src/database/`. + +## Configuration Files + +| File | Purpose | +|------|---------| +| `src/.env` / `.env` | Environment variables (DB, FTP, Firebase, mail) | +| `src/.env.example` / `.env.example` | Environment template | +| `src/composer.json` | PHP dependencies and scripts | +| `src/package.json` | Node.js dependencies (Vite, Tailwind) | +| `src/vite.config.js` | Vite build configuration | +| `src/tailwind.config.js` | Tailwind CSS configuration | +| `src/postcss.config.js` | PostCSS configuration | +| `src/phpunit.xml` | PHPUnit test configuration | +| `src/config/*.php` | Laravel config files (app, auth, cache, database, filesystems, logging, mail, queue, services, session) | +| `docker-compose.yml` | Docker orchestration (nginx, php-fpm, postgres) | +| `Dockerfile` | PHP 8.2-FPM container with PostgreSQL extensions | +| `docker/nginx/default.conf` | Nginx reverse proxy configuration | +| `Makefile` | Build and development automation commands | + +## Static Assets + +**Location:** +- CSS: `src/resources/css/app.css`, `src/resources/css/style.css` +- JavaScript: `src/resources/js/` — `app.js`, `bootstrap.js`, `script.js`, `graph.js`, `rfhistory.js`, `homemap.js` +- Compiled output: `src/public/build/` (Vite output) + +**Build Toolchain:** +- Vite (`src/vite.config.js`) for asset bundling +- Tailwind CSS (`src/tailwind.config.js`) for utility classes +- PostCSS for CSS processing +- Build command: `npm run build` or `npm run dev` (watch mode) + +## Where to Add New Code + +**New Feature Page:** +- Route: Add to `src/routes/web.php` (authenticated group) or `src/routes/api.php` (API) +- Controller: Create in `src/app/Http/Controllers/` +- View: Create in `src/resources/views/layout/` +- JavaScript: Add to `src/resources/js/` + +**New API Endpoint:** +- Controller: Create in `src/app/Http/Controllers/Api/` +- Route: Add to `src/routes/api.php` + +**New Database Table:** +- Migration: `php artisan make:migration` in `src/database/migrations/` +- Note: No Eloquent model needed — use `DB::table()` or `DB::select()` following existing pattern + +**New Export (Excel/PDF):** +- Excel: Create class in `src/app/Exports/` implementing `FromCollection`, `WithHeadings` +- PDF: Create Blade template in `src/resources/views/pdf/` + +**New Middleware:** +- Create in `src/app/Http/Middleware/` +- Register alias in `src/bootstrap/app.php` → `$middleware->alias([...])` + +**New Notification Channel:** +- Create in `src/app/Notifications/` + +**New Language:** +- Add locale code to `LocaleController.php` allowed list +- Create translation directory in `src/lang/{locale}/` + +--- + +*Structure analysis: 2026-05-20* diff --git a/.planning/codebase/TESTING.md b/.planning/codebase/TESTING.md new file mode 100644 index 00000000..50b931ea --- /dev/null +++ b/.planning/codebase/TESTING.md @@ -0,0 +1,229 @@ +# Testing + +**Analysis Date:** 2026-05-20 + +## Test Framework + +**Runner:** +- PHPUnit 11.5.3 +- Config: `src/phpunit.xml` + +**Assertion Library:** +- PHPUnit built-in assertions +- Laravel testing assertions (`assertStatus`, `assertOk`, `assertSessionHasNoErrors`, `assertRedirect`, `assertGuest`, etc.) + +**Run Commands:** +```bash +# Via composer (recommended — clears config first) +composer test +# Which runs: php artisan config:clear && php artisan test + +# Direct artisan +php artisan test + +# Via PHPUnit directly +./vendor/bin/phpunit + +# Paratest (not configured) +``` + +**Test Environment:** +- `APP_ENV=testing` +- `DB_CONNECTION=sqlite` with `:memory:` database — tests run in-memory with no persistent state +- `CACHE_STORE=array` — no cache persistence +- `SESSION_DRIVER=array` — no session persistence +- `MAIL_MAILER=array` — emails captured, not sent +- `QUEUE_CONNECTION=sync` — jobs run synchronously +- `BCRYPT_ROUNDS=4` — faster hashing for tests + +## Test Organization + +**Location:** +- Tests co-located in `src/tests/` (standard Laravel structure) + +**Naming:** +- Test files: PascalCase + `Test.php` suffix (e.g., `ProfileTest.php`, `AuthenticationTest.php`) +- Test methods: `test_` prefix with snake_case (e.g., `test_profile_page_is_displayed()`) +- Namespacing follows PSR-4: `Tests\Feature`, `Tests\Unit`, `Tests\Feature\Auth` + +**Structure:** +``` +src/tests/ +├── Feature/ +│ ├── Auth/ +│ │ ├── AuthenticationTest.php +│ │ ├── EmailVerificationTest.php +│ │ ├── PasswordConfirmationTest.php +│ │ ├── PasswordResetTest.php +│ │ ├── PasswordUpdateTest.php +│ │ └── RegistrationTest.php +│ ├── ExampleTest.php +│ └── ProfileTest.php +├── Unit/ +│ └── ExampleTest.php +└── TestCase.php +``` + +## Test Types Present + +**Unit Tests:** +- Only the default scaffolded `ExampleTest.php` with `test_that_true_is_true()` +- Extends `PHPUnit\Framework\TestCase` (no Laravel bootstrapping) +- **No real unit tests written for application code** + +**Feature Tests:** +- Breeze-scaffolded auth tests (6 files in `Auth/` directory) +- Breeze-scaffolded `ProfileTest.php` +- Default `ExampleTest.php` (basic GET `/` returns 200) +- `TestCase.php` extends Laravel's base test case + +**Integration Tests:** +- Not present + +**Browser Tests (Dusk, Cypress, etc.):** +- Not present + +**API Tests:** +- Not present — no tests for the API endpoints in `routes/api.php` + +## Test Coverage + +**Estimated coverage level:** Very Low (~5%) + +Only Breeze-scaffolded tests exist. No custom application code has tests. + +**Key areas covered:** +- Authentication flow (login, registration, password reset, email verification) — via Breeze scaffolding +- Profile management (view, update, delete account) — via Breeze scaffolding +- Basic application health check (`GET /` returns 200) + +**Key areas NOT covered (high priority gaps):** +- All custom controllers: `RainfallController`, `WaterLevelController`, `SirenController`, `AdminController`, `MapController`, `NotificationController`, `cctvController`, `LocaleController` +- API endpoints: `Api/StationController`, `Api/AuthController`, `Api/AlertController` +- Service layer: `FcmService` +- Export classes: `WaterLevelExport`, `HourlyRainfallExport` +- Middleware: `AdminMiddleware`, `LocalizationMiddleware` +- Database queries and raw SQL logic throughout all controllers +- Authorization logic (admin access level checks) +- Login blocking logic (3 failed attempts → blocked account) +- PDF generation (`NotificationController`, `SirenController`) +- Excel export functionality + +## Test Data + +**Factories:** +- `UserFactory` at `src/database/factories/UserFactory.php` +- Generates: name, email, email_verified_at, password (`'password'`), remember_token +- Has `unverified()` state modifier +- Uses `Hash::make('password')` with static caching +- **No factories for other models** — no Eloquent models exist for station, rainfall, waterlevel, siren, or notification tables + +**Seeders:** +- `DatabaseSeeder` at `src/database/seeders/DatabaseSeeder.php` +- Seeds a single admin user: name=`Admin`, email=`admin@example.com`, password=`password123`, access_level=1 +- **No test-specific seeders** + +**Fixtures:** +- Not used + +**Mocking Strategy:** +- `mockery/mockery` installed as dev dependency +- **No actual mocking used in any test file** +- All feature tests use `RefreshDatabase` trait and real database interactions (SQLite in-memory) + +## CI Integration + +**How tests run in CI:** +- No CI configuration detected (no `.github/workflows/`, no `.gitlab-ci.yml`, no `Jenkinsfile`) +- Docker setup exists (`Dockerfile`, `docker-compose.yml`) but no CI integration + +**Test commands:** +```bash +# Full test suite +composer test +# Equivalent to: php artisan config:clear --ansi && php artisan test + +# Specific test file +php artisan test --filter=ProfileTest + +# Specific test method +php artisan test --filter=test_profile_page_is_displayed + +# Parallel tests (not configured) +``` + +## Frontend Testing + +**Test Framework:** +- None configured — no Jest, Vitest, Mocha, or any frontend test runner +- No test-related scripts in `package.json` +- `package.json` only has `build` and `dev` scripts + +**Component Tests:** +- Not present + +**E2E Tests:** +- Not present + +## Writing New Tests — Recommended Patterns + +**For feature tests (controllers), follow the existing Breeze pattern:** +```php +create(); + + $response = $this + ->actingAs($user) + ->get('/rainfall'); + + $response->assertOk(); + $response->assertViewIs('layout.rainfall'); + } +} +``` + +**For API tests:** +```php +getJson('/api/station/current'); + + $response->assertOk(); + $response->assertJsonStructure([]); + } +} +``` + +**Key considerations for testing this codebase:** +1. Most controllers use raw SQL with `DB::select()` — tests need actual database tables via migrations (handled by `RefreshDatabase`) +2. The `station`, `rainfall`, `waterlevel`, `siren`, `notification` tables need factories or manual inserts for meaningful tests +3. SQLite in-memory may not support all PostgreSQL-specific features used in raw SQL (e.g., `DISTINCT ON`, `EXTRACT`, `::date` casting, `INTERVAL`) +4. Tests for `AdminController` require users with `access_level = 1` (admin) and the `'admin'` middleware alias + +--- + +*Testing analysis: 2026-05-20*