admin/sides
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix: seeder idempotent with firstOrCreate

Browse Source 
Use firstOrCreate instead of create so db:seed can run safely
on container restart without duplicate key violation.
This commit is contained in:
root
2026-05-21 02:31:47 +08:00
parent bb8d951287
commit 9122deaacd
11 changed files with 1935 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

312
AUDIT_REPORT.md Normal file
View File

@@ -0,0 +1,312 @@
# AUDIT REPORT — SIDES (tckdev)
**Date**: 2026-05-20
**Auditor**: Automated Code Review
**Scope**: Full brownfield codebase audit
---
## Executive Summary
SIDES is a flood early warning system built on Laravel 12 + PostgreSQL + Docker. The application is **functional for its purpose** but has significant security vulnerabilities, code quality issues, and architectural concerns that should be addressed before any further development or production deployment.
**Risk Level: HIGH**
---
## 1. CRITICAL — Security Vulnerabilities
### 1.1 SQL Injection (HIGH)
**Multiple controllers** build raw SQL queries with string interpolation instead of parameterized queries.
**`RainfallController::index()` (line 46):**
```php
$stationCondition = " AND s.stationid = '{$stationFilter}'";
```
**`WaterLevelController::index()` (lines 30, 37):**
```php
$stationCondition = " WHERE s.stationid = '{$stationFilter}' ";
$dateCondition = " AND w.datetime = '{$sqlDate}' ";
```
**`RainfallController::index()` (lines 55-106):**
```php
$rainfallData = collect(DB::select("
SELECT ... CAST('$displayDate' AS timestamp) ...
... CAST('$dateFilter' as date) - INTERVAL '6 days' ...
$stationCondition
"));
```
User input (`$stationFilter`, `$dateFilter`, `$displayDate`, `$sqlDate`) is interpolated directly into SQL strings. An attacker could inject SQL through these parameters.
**Impact**: Full database compromise (read, modify, delete data)
**Files affected**: `RainfallController.php`, `WaterLevelController.php`
**Fix**: Use parameterized queries with `?` placeholders or named bindings (`:param`) consistently.
### 1.2 Hardcoded Credentials in Source Code (HIGH)
**`autoscript/sidesdecode.py` (lines 22-31):**
```python
ftp_server = "myvscada.com"
ftp_username = "tck"
ftp_password = "tck6789"
pg_host = "192.168.0.211"
pg_database = "sides_db"
pg_user = "tck"
pg_password = "projectdev##1"
```
**`src/.env` (committed to git):**
```
MAIL_PASSWORD="ipmu zifw bpmf fsyp"
POSTGRES_PASSWORD="projectdev##1"
PGADMIN_PASSWORD="projectdev##1"
```
**`src/storage/app/firebase/sides-b4abb-3604a7cf7584.json`** — Firebase service account key is committed to the repository.
**Impact**: Anyone with repository access has all credentials
**Fix**: Move to environment variables, add `.env` to `.gitignore`, rotate all exposed credentials
### 1.3 Default Admin Credentials (HIGH)
**`DatabaseSeeder.php` and migration `2025_12_11_124201`:**
```php
'name' => 'admin',
'password' => Hash::make('password123'),
```
**Impact**: Anyone who discovers this gets admin access
**Fix**: Remove hardcoded credentials, require admin setup on first run, use strong passwords
### 1.4 API Endpoints Unauthenticated (MEDIUM)
All API endpoints except `/api/login` have **no authentication**:
```php
Route::get('/station/current', [StationController::class, 'getCurrentData']);
Route::get('/station/rainfall', [StationController::class, 'getRainfallData']);
// ... all publicly accessible
```
**`/api/alert`** is publicly accessible — anyone can trigger push notifications:
```php
Route::post('/alert',[AlertController::class,'send']);
```
**Impact**: Data exfiltration, unauthorized push notification spam
**Fix**: Add API authentication (sanctum/passport tokens) to all endpoints
### 1.5 API Login Returns Password Hash (LOW)
**`Api\AuthController::login()` (line 23):**
```php
$user = DB::select("SELECT id, name, email, access_level, password FROM users ...");
```
While the full `$user` object isn't returned, the query fetches the password hash unnecessarily.
**Fix**: Remove `password` from the SELECT query.
---
## 2. HIGH — Code Quality Issues
### 2.1 No Eloquent ORM Usage
The entire application uses `DB::table()` and `DB::select()` (query builder / raw SQL) instead of Eloquent models. Only the `User` model exists.
**Impact**:
- No type safety, no attribute casting, no relationship definitions
- Difficulty maintaining and extending the codebase
- No model-level validation or events
**Fix**: Create Eloquent models for `Station`, `Rainfall`, `WaterLevel`, `Siren`, `Notification` with proper relationships.
### 2.2 No Database Foreign Keys
All table relationships (`stationid` references) are maintained at the application level with no database constraints:
```php
// Migration: no foreign key
$table->string('stationid');
// vs what it should be:
$table->foreign('stationid')->references('stationid')->on('station');
```
**Impact**: Data integrity cannot be guaranteed at the database level. Orphaned records possible.
### 2.3 Missing Database Indexes
No indexes exist on frequently queried columns:
- `rainfall.stationid` — used in every rainfall query
- `rainfall.timestamp` — used for date filtering and max() queries
- `waterlevel.stationid` — used in every water level query
- `waterlevel.datetime` — used for date filtering
- `siren.stationid` — used in every siren query
- `siren.active_time` — used for max() and ordering
- `notification.stationid` — used in every notification query
- `notification.timestamp` — used for date filtering
**Impact**: Query performance degrades significantly as data grows
**Fix**: Add composite indexes on `(stationid, timestamp)` for data tables
### 2.4 Duplicate Admin User Creation
Admin user is created in **both** the DatabaseSeeder AND a migration:
- `DatabaseSeeder.php` line 22: `User::create(['name' => 'Admin', ...])`
- Migration `2025_12_11_124201`: `DB::table('users')->insert(['name' => 'admin', ...])`
**Impact**: Running `migrate:fresh --seed` will create two admin users with different names ("Admin" and "admin")
**Fix**: Remove one — keep only the seeder.
### 2.5 Inconsistent Naming Conventions
| Issue | Example |
|-------|---------|
| Case inconsistency | `cctvController` (lowercase), `SirenHistory` (PascalCase) |
| Mixed route naming | `sirenhistory` vs `historicalrf` vs `historicalwl` |
| Mixed table naming | `waterlevel.datetime` vs `rainfall.timestamp` (same concept, different column names) |
| Variable typos | `$dateTImeFilter` (capital I), `$dateFilter` |
### 2.6 No Input Validation on Some Endpoints
`MapController::getStations()` and `MapController::getCurrentData()` accept no parameters but other controllers accept GET/POST parameters with inconsistent validation.
### 2.7 Commented-Out Code Blocks
Multiple large blocks of commented-out code throughout:
- `RainfallController.php` lines 250-263 (alternative graph query)
- `WaterLevelController.php` date conditions
- `sidesdecode.py` file management functions
---
## 3. MEDIUM — Architectural Concerns
### 3.1 N+1 Query Pattern in MapController
`MapController::getCurrentData()` uses subqueries in JOINs for latest data:
```php
->whereRaw('r.timestamp = (SELECT MAX(timestamp) FROM rainfall WHERE stationid = s.stationid)')
```
This executes a subquery for every station for every table join (3 subqueries per station). Should use a CTE or window function.
### 3.2 No Caching
Real-time data queries (dashboard, map, station listings) run complex SQL on every page load with no caching layer despite data only updating when the Python script runs.
### 3.3 No Form Request Classes
Validation is done inline in controllers despite Laravel's Form Request feature. Only `LoginRequest` and `ProfileUpdateRequest` exist from Breeze scaffolding.
### 3.4 No API Token Authentication
The API login endpoint validates credentials but returns no token. This makes the API unusable for stateless/mobile clients that need persistent authentication.
### 3.5 FCM Topic Fixed
`AlertController::send()` always sends to `FCM_TOPIC_RAINFALL_WARNING` regardless of alert type:
```php
$topic = env('FCM_TOPIC_RAINFALL_WARNING');
```
Water level and siren alerts go to the rainfall topic, not their respective topics.
### 3.6 Water Level Alert Bug
**`sidesdecode.py` line 378:**
```python
send_alert_to_laravel(station_id, level, 1) # stationtype=1 (rainfall) instead of 2 (water level)
```
Water level threshold alerts are sent with `stationtype=1` instead of `stationtype=2`, causing them to be classified as rainfall alerts.
### 3.7 `is_blocked` Not Enforced
The `is_blocked` column exists on users but no middleware checks it. Blocked users can still log in and use the system.
### 3.8 No Rate Limiting
No rate limiting on login, API endpoints, or form submissions.
---
## 4. LOW — Minor Issues
### 4.1 `APP_DEBUG=true` in `.env`
Debug mode should be `false` in any non-local environment.
### 4.2 `database.db` and `database.sqlite` Committed
SQLite database files are in the repository root and `src/database/`.
### 4.3 Unused Packages
- `laravel/sail` — installed but Docker Compose is used directly
- `laravel/pail` — installed but no log streaming is used
- `nunomaduro/collision` — dev dependency, standard
### 4.4 `.env` Committed
Both `.env` files (root and `src/`) are committed to the repository. They should be in `.gitignore`.
### 4.5 `.editorconfig` Duplication
`.editorconfig` exists in both root and `src/` directories.
### 4.6 No `.gitignore` for Autoscript Logs
`autoscript/sidesdecode.log` and `sidesdecode_error.log` are tracked in git.
### 4.7 Mixed CSS Frameworks
Both Tailwind CSS and Bootstrap 5 are loaded on pages, creating potential style conflicts and unnecessary bundle size.
### 4.8 No Tests for Domain Logic
Only Breeze default tests exist (`AuthenticationTest`, `EmailVerificationTest`, etc.). No tests for domain controllers, data pipeline, or export logic.
---
## 5. Summary Table
| # | Issue | Severity | File(s) | Effort |
|---|-------|----------|---------|--------|
| 1.1 | SQL Injection | CRITICAL | RainfallController, WaterLevelController | Medium |
| 1.2 | Hardcoded credentials | CRITICAL | sidesdecode.py, .env | Low |
| 1.3 | Default admin password | CRITICAL | DatabaseSeeder, migration | Low |
| 1.4 | Unauthenticated API | HIGH | routes/api.php | Medium |
| 1.5 | Password hash in query | LOW | AuthController | Low |
| 2.1 | No Eloquent models | HIGH | All controllers | High |
| 2.2 | No foreign keys | HIGH | All migrations | Medium |
| 2.3 | Missing indexes | HIGH | All data tables | Medium |
| 2.4 | Duplicate admin creation | MEDIUM | Seeder + migration | Low |
| 2.5 | Inconsistent naming | LOW | Throughout | Medium |
| 3.1 | N+1 query pattern | MEDIUM | MapController | Medium |
| 3.2 | No caching | MEDIUM | Throughout | Medium |
| 3.5 | FCM topic hardcoded | MEDIUM | AlertController | Low |
| 3.6 | Wrong stationtype for WL | HIGH | sidesdecode.py:378 | Low |
| 3.7 | Blocked users not enforced | MEDIUM | Auth flow | Low |
| 3.8 | No rate limiting | MEDIUM | Throughout | Low |
| 4.1 | Debug mode on | LOW | .env | Low |
| 4.2 | DB files committed | LOW | Root | Low |
| 4.4 | .env committed | LOW | Root | Low |
| 4.7 | Mixed CSS frameworks | LOW | Blade templates | Medium |
| 4.8 | No domain tests | LOW | tests/ | High |
---
## 6. Recommended Priority
1. **Immediate** (before any further changes): Fix SQL injection, rotate credentials, remove `.env` from git
2. **Short-term**: Fix water level stationtype bug, add API auth, add database indexes, fix FCM topic routing
3. **Medium-term**: Create Eloquent models, add foreign keys, implement caching, add input validation
4. **Long-term**: Full test coverage, clean up naming conventions, choose single CSS framework

530
autoscript/sidesdecode.py Normal file
View File

@@ -0,0 +1,530 @@
from ftplib import FTP
from datetime import datetime
from datetime import date
from io import BytesIO
import os
import time
import psycopg2
import requests
today = date.today()
day = f"{today.day:02}"
month = f"{today.month:02}"
year = str(today.year)
today_str_file = datetime.today().strftime('%d%m%Y')
ftp_server = os.environ.get("FTP_SERVER", "myvscada.com")
ftp_username = os.environ.get("FTP_USERNAME", "tck")
ftp_password = os.environ.get("FTP_PASSWORD", "")
ftp_root_folder = f"files/SIDES/SUCCESS/{year}/{month}/{day}"
pg_host = os.environ.get("PG_HOST", "postgres")
pg_database = os.environ.get("PG_DATABASE", "sides_db")
pg_user = os.environ.get("PG_USER", "tck")
pg_password = os.environ.get("PG_PASSWORD", "")
# Connect to PostgreSQL
conn = psycopg2.connect(
host=pg_host,
database=pg_database,
user=pg_user,
password=pg_password
)
cursor = conn.cursor()
# Connect to FTP
ftp = FTP(ftp_server)
ftp.login(user=ftp_username, passwd=ftp_password)
ftp.cwd(ftp_root_folder)
# Get today's date in yymmdd format
today_str = datetime.today().strftime('%y%m%d')
# List all subfolders in /files/JKR
folders = ftp.nlst()
#Move File to Error Folder Function
def move_to_error_folder(filename,filecontent,error_folder="/"+ftp_root_folder+"/ERROR/"):
with FTP(ftp_server) as ftp:
ftp.login(ftp_username, ftp_password)
parts = error_folder.strip("/").split("/")
current_path = ''
for part in parts :
current_path += "/" + part
try :
ftp.mkd(current_path)
except Exception :
pass
ftp.cwd(error_folder)
if isinstance(filecontent,str) :
filecontent = filecontent.encode("utf-8")
bio = BytesIO(filecontent)
ftp.storbinary(f"STOR {filename}",bio)
bio.close()
print(f"⚠️ Moved {file_name} into {error_folder}")
try:
ftp.cwd('..')
ftp.delete(filename)
print(f"🗑️ Removed {file_name} from {ftp_root_folder}")
except Exception as e:
print(f"⚠️ Could not remove {file_name} from {ftp_root_folder}: {e}")
#Move File to Success Folder Function
def move_to_success_folder(filename,filecontent,success_folder="/"+ftp_root_folder+"/SUCCESS/"):
with FTP(ftp_server) as ftp :
ftp.login(ftp_username,ftp_password)
parts = success_folder.strip("/").split("/")
current_path =""
for part in parts :
current_path += "/" + part
try :
ftp.mkd(current_path)
print(f"Created Folder : {current_path}")
except Exception:
pass
ftp.cwd(success_folder)
if isinstance(filecontent,str):
filecontent = filecontent.encode("utf-8")
with open("temp_upload.tmp", "wb") as temp_file :
temp_file.write(filecontent)
with open("temp_upload.tmp", "rb") as temp_file:
ftp.storbinary(f"STOR {filename}",temp_file)
os.remove("temp_upload.tmp")
print(f"Uploaded {filename} to {success_folder}")
try :
ftp.cwd(f"/{ftp_root_folder}")
ftp.delete(filename)
print(f"🗑️ Removed {file_name} from {ftp_root_folder}")
except Exception as e:
print(f"⚠️ Could not remove {file_name} from {ftp_root_folder}: {e}")
#Send Alert Function
def send_alert_to_laravel(stationid,level,stationtype):
#Each parameter variable is from decoded process in Function process_line()
#Data will send to SIDES API in src/app/Http/Controllers/Api/AlertController.php
payload = {
"stationid" : stationid,
"level" : level,
"stationtype" :stationtype,
}
#Request and send the payload data to API URL
try:
response = requests.post("https://sides.tck.com.my/api/alert",json=payload,timeout=5)
print("Alert Sent:" ,response.status_code)
except Exception as e:
print("Failed to send Alert",e)
# Define processing function once
def process_line(line,filename=None):
# Strip each ',' seperator in file line
columns = line.strip().split(",")
#Check Columns Length each line
if len(columns) < 25:
print(f"Skipping malformed line: {line}")
#Uncomment If needed
# try :
# #Move decode folder into ERROR folder files/SIDES/ERROR
# #move_to_error_folder(filename,line,error_folder="/"+ftp_root_folder+"/ERROR")
# except Exception as e :
# print(f"⚠️ Failed to move malformed line to ERROR folder: {e}")
return
# Variable Declaration based on CSV Column Data
#Common Data Variable that needed
station_id = columns[1]
timestamp = columns[4]
#Siren Data Variable
sirenid = columns[18]
siren = columns[19]
#Rainfall Variable
anncumm = float(columns[21]) if columns[21] else None
dailycumm = float(columns[22]) if columns[22] else None
hourlycumm = float(columns[23]) if columns[23] else None
currrf = float(columns[24]) if columns[24] else None
#Water Level Variable
waterlevel = float(columns[36]) if columns[36] else None
#Water Level Threshold Variable
wldgr = float(columns[17]) if columns[17] else None
wlwarn = float(columns[16]) if columns[16] else None
wlalert = float(columns[15]) if columns[15] else None
#Battery Variable
battery = float(columns[6]) if columns[6] else None
try:
#Timestamp Format
datetime_object = datetime.strptime(timestamp, "%y%m%d%H%M%S")
#Uncomment If Using Separate Date And Time Column
# date_part = datetime_object.strftime("%Y-%m-%d")
# time_part = datetime_object.strftime("%H:%M:%S")
# date_part2 = datetime_object.date()
#Rainfall Data Insertion Into Database
#Check dailycumm variable Or hourtycumm variable is Not Null
#If either dailycumm or hourlycum, is Null. Skip this process
if dailycumm != None or hourlycumm != None :
#Execute Query for check data is already available or not in rainfall Table
#TABLE : RAINFALL
#COLUMN : stationid,timestamp
#INPUT : station_id,datetime_object
#OUTPUT : -
cursor.execute("""
SELECT COUNT(*)
FROM rainfall
WHERE stationid = %s AND timestamp = %s
""", (station_id, datetime_object))
#Delcarable Variable Record Exists Condition
record_exists = cursor.fetchone()[0] > 0
# Check If Record is exist
# If record not exist execute INSERT rainfall data into rainfall Table
if not record_exists:
#Execute Insert Query into rainfall table
#TABLE : RAINFALL
#COLUMN : stationid,timestamp,anncum,daily,hourly,currentrf,battery
#INPUT : station_id ,date_object,anncumm,dailycumm,hourlycumm,currrf,battery
#OUTPUT : -
cursor.execute("""
INSERT INTO rainfall (stationid,timestamp, anncum, daily, hourly, currentrf,battery)
VALUES (%s, %s, %s, %s, %s, %s, %s)
""", (station_id, datetime_object,anncumm, dailycumm, hourlycumm, currrf,battery))
conn.commit()
# If Rainfall Threshold Warning Threshold
if hourlycumm >= 30 :
#Threshold Level Declarion
if hourlycumm >= 30 and hourlycumm < 60 :
level = 'Warning'
elif hourlycumm >= 60 :
level = 'Danger'
else :
level = 'Normal'
#Execute Query for check data is already available or not in notification Table
#TABLE : NOTIFICATION
#COLUMN : stationid,timestamp,stationtype
#INPUT : station_id,datetime_object,1
#OUTPUT : -
cursor.execute("""
SELECT COUNT(*)
FROM notification
WHERE stationid = %s AND timestamp = %s AND stationtype = %s
""", (station_id, datetime_object,1))
record_exists = cursor.fetchone()[0] > 0
# Check If Record is exist
# If record not exist execute INSERT rainfall data that trigger threshold into notification Table
if not record_exists:
#Execute Insert Query into notification table
#stationtype = 1 because rainfall data
#TABLE : NOTIFICATION
#COLUMN : stationid,timestamp.stationtype,level,active_time
#INPUT : station_id,datetime_object,1,level,datetime_object
#OUTPUT : -
cursor.execute("""
INSERT INTO notification (stationid,timestamp, stationtype, level,active_time)
VALUES (%s, %s, %s, %s, %s)
""", (station_id, datetime_object,1, level,datetime_object))
conn.commit()
#Call Alert Function with station id , level , stationtype as parameter
#Send Status Station Detail to laravel API for Alert Notification
send_alert_to_laravel(station_id,level,1)
#Print Rainfall Details
print(f"Station ID {station_id}")
print(f"Timestamp {timestamp}")
print(f"Daily Cumm : {dailycumm}")
print(f"Hourly : {hourlycumm}\n")
#Water Level Data Insertion Into Database
#Check waterlevel is Not Null
#If water level is Null. Skip this process
if waterlevel != None :
#Execute Query for check data is already available or not in waterlevel Table
#TABLE : WATERLEVEL
#COLUMN : stationid,datetime
#INPUT : station_id,datetime_object
#OUTPUT : -
cursor.execute("""
SELECT COUNT(*)
FROM waterlevel
WHERE stationid = %s AND datetime = %s
""", (station_id, datetime_object))
record_exists = cursor.fetchone()[0] > 0
# Check If Record is exist
# If record not exist execute INSERT waterlevel data into waterlevel Table
if not record_exists:
#Execute Insert Query into waterlevel table
#TABLE : WATERLEVEL
#COLUMN : stationid,datetime,waterlevel,alert,warning,danger
#INPUT : station_id,datetime_object,waterlevel,wlalert,wlwarn,wldgr
#OUTPUT : -
cursor.execute("""
INSERT INTO waterlevel (stationid,datetime, waterlevel, alert, warning,danger)
VALUES (%s, %s, %s, %s, %s, %s)
""", (station_id, datetime_object,waterlevel, wlalert, wlwarn, wldgr))
conn.commit()
# If Water Level Threshold Trigger
if waterlevel >= wlalert :
#Water Level Level Declaration
if waterlevel >= wlalert and waterlevel < wlwarn :
level = 'Alert'
elif waterlevel >= wlwarn and waterlevel < wldgr :
level = 'Warning'
elif waterlevel >= wldgr:
level = 'Danger'
else :
level = 'Normal'
#Execute Query for check data is already available or not in notification Table
#TABLE : NOTIFICATION
#COLUMN : stationid,timestamp,stationtype
#INPUT : station_id,datetime_object,2
#OUTPUT : -
cursor.execute("""
SELECT COUNT(*)
FROM notification
WHERE stationid = %s AND timestamp = %s AND stationtype = %s
""", (station_id, datetime_object,2))
record_exists = cursor.fetchone()[0] > 0
# Check If Record is exist
# If record not exist execute INSERT waterlevel data that trigger threshold notification Table
if not record_exists:
#Execute Insert Query into notification table
#stationtype = 2 because waterlevel data
#TABLE : NOTIFICATION
#COLUMN : stationid,timestamp,stationtype,level,active_time
#INPUT : station_id,datetime_object,2,level,datetime_object
#OUTPUT : -
cursor.execute("""
INSERT INTO notification (stationid,timestamp, stationtype, level,active_time)
VALUES (%s, %s, %s, %s, %s)
""", (station_id, datetime_object,2, level,datetime_object))
conn.commit()
#Call Alert Function with station id , level , stationtype as parameter
#Send Status Station Detail to laravel API for Alert Notification
send_alert_to_laravel(station_id,level,2)
print(f"Station ID {station_id}")
print(f"Timestamp {timestamp}")
print(f"Water Level : {waterlevel}")
print(f"Danger : {wldgr}")
print(f"Warning : {wlwarn}\n")
#Siren Data Insertion Into Database
#Check sirenid variable is Not Null
#If sirenid is Null. Skip this process
if sirenid != None:
#Execute Query for check data is already available or not in siren Table
#TABLE : SIREN
#COLUMN : stationid,active_time
#INPUT : station_id,datetime_object
#OUTPUT : -
cursor.execute("""
SELECT COUNT(*)
FROM siren
WHERE stationid = %s AND active_time = %s
""", (station_id, datetime_object))
record_exists = cursor.fetchone()[0] > 0
#Siren level Declaration
if siren == 'H' :
level = 'Danger'
elif siren == 'L':
level = 'Warning'
else :
level = 'Normal'
# check siren condition
# If Siren is Normal
if siren == 'N':
# Check If Record is exist
# If record not exist execute INSERT siren data into siren Table
if not record_exists:
#TABLE : SIREN
#COLUMN : stationid,active_time,level
#INPUT : station_id,datetime_object,siren
#OUTPUT : -
cursor.execute("""
INSERT INTO siren (stationid, stationtype, active_time, level)
VALUES (%s, %s, %s, %s)
""", (station_id, 3, datetime_object, siren))
conn.commit()
#Print Siren Details
print(f"Station ID {station_id}")
print(f"SRID {sirenid}")
print(f"Timestamp {timestamp}")
print(f"Alarm : {siren}\n")
print(f"Level : {level}\n")
#If siren not Normal
elif siren != 'N' :
# Check If Record is exist
# If record not exist execute INSERT siren data into siren Table
if not record_exists:
#Execute Insert Query into siren table
#TABLE : SIREN
#COLUMN : stationid,active_time,level
#INPUT : station_id,datetime_object,siren
#OUTPUT : -
cursor.execute("""
INSERT INTO siren (stationid, stationtype, active_time, level)
VALUES (%s, %s, %s, %s)
""", (station_id, 3, datetime_object, siren))
conn.commit()
#Print Siren Table
print(f"Station ID {station_id}")
print(f"SRID {sirenid}")
print(f"Timestamp {timestamp}")
print(f"Alarm : {siren}\n")
#Send Station Alert To API When Siren Level is not normal
#Call Alert Function with station id , level , stationtype as parameter
#Send Status Station Detail to laravel API for Alert Notification
if level != 'Normal' :
send_alert_to_laravel(station_id,level,3)
except Exception as e:
conn.rollback()
#Move decode folder into ERROR folder files/SIDES/ERROR
#move_to_error_folder(filename,line,error_folder="/"+ftp_root_folder+"/ERROR")
print(f"Error processing line: {e}")
#Start Process FTP File
try:
#Folder File List Variable Declaration
files = ftp.nlst()
#Loop Through the Folder/Files
for file_name in files:
#Skip Tideda File
if "rf" in file_name.lower():
#print(f"Skipping file: {file_name} (contains 'rf')")
continue
#Skip Others Day File
if today_str not in file_name:
#print(f"Skipping file: {file_name} (not from today {today_str})")
continue
print(f"Processing file: {file_name}")
all_lines = []
#Retrieve the files as a text , line by line
ftp.retrlines(
f"RETR {file_name}",
lambda line: (all_lines.append(line), process_line(line, file_name))
)
file_content = "\n".join(all_lines)
#Uncomment if needed
#Move decode folder into success folder files/SIDES/SUCCESS/year/month/day
#move_to_success_folder(file_name,file_content,success_folder="/"+ftp_root_folder+"/SUCCESS/"+year+"/"+month+"/"+day)
except Exception as e:
print(f"Failed to process folder: {e}")
#Uncomment this for test Alert Notification
# send_alert_to_laravel('KBLG0026','Warning',1)
# send_alert_to_laravel('KBLG0026','Danger',2)
# send_alert_to_laravel('KBLG0031','Warning',3)
# Close connections
ftp.quit()
cursor.close()
conn.close()

BIN
backup/sides_20260120 Normal file
View File

Binary file not shown.

71
docs/01-OVERVIEW.md Normal file
View File

@@ -0,0 +1,71 @@
# SIDES - Project Overview
## What Is SIDES?
**Sabo Integrated Debris Flow Monitoring and Early Warning System (SIDES)** is a flood and debris flow early warning system for **Sungai Kupang**, Malaysia. It is operated under **Jabatan Pengairan dan Saliran (JPS)** — the Malaysian Department of Irrigation and Drainage.
The system monitors rainfall, water levels, and siren activations at remote telemetry stations and presents real-time data on a web dashboard with an interactive map.
## Purpose
SIDES provides:
- **Real-time monitoring** of hydrological and meteorological data from field stations
- **Early warning alerts** when rainfall or water levels exceed danger thresholds
- **Siren status tracking** for stations equipped with warning sirens
- **Historical data analysis** with exportable reports (Excel, PDF)
- **Push notifications** to mobile devices via Firebase Cloud Messaging (FCM)
- **Admin management** of stations, users, and access control
## Domain Context
- **Location**: Sungai Kupang, Malaysia (Kedah region based on station IDs)
- **Operator**: JPS (Department of Irrigation and Drainage), Malaysia
- **Data source**: Remote telemetry stations transmit CSV data via FTP to a central server
- **Data pipeline**: Python script (`autoscript/sidesdecode.py`) fetches FTP files, parses CSV, inserts into PostgreSQL, and triggers push notifications
## User Roles
| Role | Access Level | Description |
|------|-------------|-------------|
| **Public** | Unauthenticated | View dashboard map with station data |
| **User** | `access_level = 2` | View all monitoring pages, historical data, export reports |
| **Admin** | `access_level = 1` | Full access including station management and user management |
## High-Level Architecture
```
[Telemetry Stations] --CSV/FTP--> [FTP Server (myvscada.com)]
|
[Python Script (cron)]
|
v
[PostgreSQL Database] <--data-- [sidesdecode.py] --alert--> [Laravel API]
|
v
[Firebase FCM]
|
v
[Mobile Devices]
[Laravel Web App] <--reads-- [PostgreSQL Database]
|
v
[Web Browser] <--Leaflet Map, Charts, Tables-->
```
## Application Name
The application is registered as **SIDES** in Laravel config (`APP_NAME=SIDES`).
## Bilingual Support
The application supports:
- **English** (en) — default
- **Bahasa Malaysia** (bm)
Language files are stored in `src/lang/{en,bm}/` and toggled via `LocaleController` with session-based persistence.
## Project Name Origin
The repository is named `tckdev` (likely **TCK Development**), referencing the organization running the SIDES system.

165
docs/02-ARCHITECTURE.md Normal file
View File

@@ -0,0 +1,165 @@
# Architecture
## Technology Stack
| Layer | Technology | Version |
|-------|-----------|---------|
| Backend | Laravel (PHP) | 12.x |
| PHP | PHP-FPM | 8.2 |
| Frontend | Blade Templates + Alpine.js | 3.x |
| CSS | Tailwind CSS + Bootstrap 5.3 | 3.x / 5.3 |
| Charts | Custom JS (Chart.js via `graph.js`) | — |
| Maps | Leaflet.js | 1.9.4 |
| Build Tool | Vite | 7.x |
| Database | PostgreSQL | 15 |
| Cache/Session | Database (via Laravel) | — |
| Queue | Database (via Laravel) | — |
| Containerization | Docker + Docker Compose | 3.9 |
| Web Server | Nginx | stable-alpine |
| PDF Generation | barryvdh/laravel-dompdf | 3.1 |
| Excel Export | maatwebsite/excel | 3.1 |
| Push Notifications | Firebase Cloud Messaging (FCM) via Google Auth | 1.49 |
| Date Picker | Flatpickr | 4.6 |
| Auth Scaffold | Laravel Breeze | 2.3 |
| Data Pipeline | Python (psycopg2, ftplib) | 3.x |
## Container Architecture
The application runs in 5 Docker containers defined in `docker-compose.yml`:
```
┌──────────────────────────────────────────────────────────┐
│ Docker Network: tckdev_net │
│ │
│ ┌─────────┐ ┌─────────┐ ┌──────────┐ │
│ │ web │ │ app │ │ postgres │ │
│ │ (nginx) │──>│ (PHP-FPM│──>│ (DB) │ │
│ │ :80 │ │ :9000) │ │ :5432 │ │
│ └─────────┘ └─────────┘ └──────────┘ │
│ │
│ ┌─────────┐ ┌─────────┐ │
│ │ pgAdmin │ │ adminer │ │
│ │ :5050 │ │ :6060 │ (DB management tools) │
│ └─────────┘ └─────────┘ │
└──────────────────────────────────────────────────────────┘
```
### Container Details
| Container | Image | Port | Purpose |
|-----------|-------|------|---------|
| `tckdev-app` | Custom (PHP 8.2-FPM + Composer 2.3 + Node.js) | 9000 (internal) | Laravel application |
| `tckdev-web` | nginx:stable-alpine | 80 | Reverse proxy & static files |
| `tckdev-db` | postgres:15 | 5432 | PostgreSQL database |
| `tckdev-pgAdmin` | dpage/pgadmin4 | 5050 | Database management UI |
| `tckdev-adminer` | adminer | 6060 | Lightweight DB management UI |
## Request Flow
```
Browser --> Nginx (:80) --> PHP-FPM (:9000) --> Laravel Router
|
┌─────────────┼─────────────────┐
v v v
Web Routes API Routes Auth Routes
(web.php) (api.php) (auth.php)
| | |
v v v
Controllers Api/Controllers Breeze Controllers
| |
v v
DB::raw() SQL DB::raw() SQL
| |
v v
PostgreSQL PostgreSQL
|
v
Blade Views --> HTML Response
```
## Middleware Stack
| Middleware | Route Group | Purpose |
|-----------|------------|---------|
| `auth` | Web (protected) | Require authenticated session |
| `admin` | Admin routes | Require `access_level = 1` |
| `LocalizationMiddleware` | All web routes | Set locale from session (`en`/`bm`) |
| `guest` | Auth routes (login/register) | Redirect away if already authenticated |
## Database Design
### Core Tables
```
┌─────────────┐ ┌──────────────┐
│ station │ │ users │
│─────────────│ │──────────────│
│ stationid* │ │ id* │
│ name │ │ name │
│ district │ │ email │
│ lng, lat │ │ password │
│ mainriverbasin│ │ access_level │
│ subriverbasin│ │ is_blocked │
│ rainfall │ │ login_attempts│
│ waterlevel │ └──────────────┘
│ siren │
│ cctv_link │
└──────┬──────┘
│
│ (stationid FK via application logic, not DB constraint)
│
┌────┼────────────┬──────────────┐
v v v v
┌──────────┐ ┌────────────┐ ┌───────────┐
│ rainfall │ │ waterlevel │ │ siren │
│──────────│ │────────────│ │───────────│
│ id* │ │ id* │ │ id* │
│ stationid│ │ stationid │ │ stationid │
│ timestamp│ │ datetime │ │ stationtype│
│ anncum │ │ waterlevel │ │ active_time│
│ daily │ │ alert │ │ level │
│ hourly │ │ warning │ └───────────┘
│ currentrf│ │ danger │
│ battery │ └────────────┘
└──────────┘
│
v
┌──────────────┐
│ notification │
│──────────────│
│ id* │
│ stationid │
│ timestamp │
│ stationtype │ (1=rainfall, 2=waterlevel, 3=siren)
│ level │ (Normal, Alert, Warning, Danger)
│ active_time │
└──────────────┘
```
### Station Types
The `station` table uses boolean flags to indicate which monitoring types each station supports:
- `rainfall = 1` — Station has rainfall monitoring
- `waterlevel = 1` — Station has water level monitoring
- `siren = 1` — Station has a warning siren
### Foreign Key Note
There are **no database-level foreign keys** between `station` and the data tables. Relationships are maintained at the application level via `stationid` column matching.
### Laravel Standard Tables
The application also uses standard Laravel tables:
- `users` — Authentication and authorization
- `password_reset_tokens` — Password reset flow
- `sessions` — Database-backed sessions
- `cache`, `cache_locks` — Database cache store
- `jobs`, `job_batches`, `failed_jobs` — Database queue
## Authentication & Authorization
- **Web auth**: Laravel Breeze (session-based, Blade views)
- **API auth**: Custom token-less login via `Api\AuthController` (returns user info, no token generation)
- **Admin access**: Controlled by `AdminMiddleware` checking `access_level === 1`
- **User blocking**: Users can be blocked via `is_blocked` flag (though no middleware enforces it currently)

122
docs/03-DEPLOYMENT.md Normal file
View File

@@ -0,0 +1,122 @@
# Deployment & Infrastructure
## Prerequisites
- Docker ^19.*
- Docker Compose
- Make (optional, for Makefile commands)
## Environment Configuration
### Root `.env` (Docker-level)
Located at `/root/sides/tckdev/.env`:
```
POSTGRES_DB="tckdev"
POSTGRES_USER="tck"
POSTGRES_PASSWORD="projectdev##1"
PGADMIN_EMAIL="tck68000@gmail.com"
PGADMIN_PASSWORD="projectdev##1"
```
### Application `.env` (Laravel-level)
Located at `/root/sides/tckdev/src/.env`:
Key configuration:
- `APP_URL=https://sides.tck.com.my`
- `DB_CONNECTION=pgsql` → `DB_HOST=192.168.0.211` (direct IP, not Docker service name)
- `SESSION_DRIVER=database`
- `CACHE_STORE=database`
- `QUEUE_CONNECTION=database`
- `MAIL_MAILER=smtp` via Gmail SMTP (`sideskupang@gmail.com`)
- Firebase FCM credentials at `storage/app/firebase/sides-b4abb-3604a7cf7584.json`
## Initial Setup
### From Scratch (New Project)
```bash
make create-project
```
This runs:
1. Creates `src/` directory
2. Builds Docker images
3. Starts containers
4. Runs `composer create-project laravel/laravel .`
5. Generates application key
6. Creates storage link
7. Sets permissions
8. Installs npm dependencies
### From Existing Code (Clone)
```bash
make init
```
This runs:
1. Builds and starts containers
2. `composer install`
3. Copies `.env.example` to `.env`
4. Generates application key
5. Creates storage link
6. Sets permissions
7. `npm install`
8. `php artisan migrate:fresh --seed`
## Makefile Commands Reference
| Command | Description |
|---------|-------------|
| `make up` | Start containers (detached) |
| `make down` | Stop containers, remove orphans |
| `make down-v` | Stop containers, remove volumes |
| `make build` | Build Docker images |
| `make remake` | Full destroy + reinit |
| `make destroy` | Remove all containers, images, volumes |
| `make stop` | Stop containers |
| `make restart` | Down + up |
| `make ps` | Show container status |
| `make logs` | Show all container logs |
| `make logs-watch` | Follow all logs |
| `make log-app` | Show app container logs |
| `make log-db` | Show database logs |
| `make app` | Shell into app container |
| `make web` | Shell into web container |
| `make migrate` | Run migrations |
| `make fresh` | Fresh migration with seed |
| `make seed` | Run seeders |
| `make tinker` | Open Laravel tinker |
| `make test` | Run PHPUnit tests |
| `make cache` | Optimize autoload + cache |
| `make cache-clear` | Clear all caches |
## Access Points
| Service | URL |
|---------|-----|
| Application | http://localhost:80 |
| pgAdmin | http://localhost:5050 |
| Adminer | http://localhost:6060 |
| PostgreSQL | localhost:5432 |
## Database Connection Discrepancy
**Important**: The `docker-compose.yml` defines a `postgres` service, but `src/.env` connects to `192.168.0.211` — an external PostgreSQL host, not the Docker container. This means:
- In **development/local**: The app may connect to an external database instead of the Docker postgres container
- The Docker postgres service is still available at `postgres:5432` for inter-container communication
- The Python autoscript also connects to `192.168.0.211` directly
To use the Docker PostgreSQL, change `DB_HOST=postgres` in `src/.env`.
## Production Notes
- `APP_DEBUG=true` is set in `.env` — should be `false` in production
- `URL::forceScheme('https')` is enabled globally in `AppServiceProvider`
- The app URL is `https://sides.tck.com.my`
- Gmail SMTP is used for password reset emails
- Firebase credentials JSON file is stored in `storage/app/firebase/`

143
docs/04-DATA-PIPELINE.md Normal file
View File

@@ -0,0 +1,143 @@
# Data Pipeline: Python Autoscript
## Overview
The file `autoscript/sidesdecode.py` is the data ingestion pipeline that:
1. Connects to an FTP server where telemetry stations upload CSV data
2. Downloads and parses CSV files for the current day
3. Inserts rainfall, water level, and siren data into PostgreSQL
4. Triggers push notifications when thresholds are exceeded
## How It Runs
The script is designed to be run on a **schedule** (likely cron job), processing new data files uploaded by remote telemetry stations throughout the day.
## FTP Connection
```
Server: myvscada.com
Username: tck
Password: tck6789
Path: files/SIDES/SUCCESS/{year}/{month}/{day}/
```
The script navigates to today's date folder and lists all files.
### File Filtering
- Skips files containing "rf" in the filename (Tideda format files)
- Only processes files with today's date (`yymmdd` format) in the filename
## CSV Format
Each line in the CSV file contains 37+ comma-separated columns. Key columns extracted:
| Column Index | Field | Description |
|-------------|-------|-------------|
| 1 | `station_id` | Station identifier (e.g., KBLG0026) |
| 4 | `timestamp` | Timestamp in `yymmddHHMMSS` format |
| 6 | `battery` | Battery voltage |
| 15 | `wlalert` | Water level alert threshold |
| 16 | `wlwarn` | Water level warning threshold |
| 17 | `wldgr` | Water level danger threshold |
| 18 | `sirenid` | Siren identifier |
| 19 | `siren` | Siren status (`H`=Danger/High, `L`=Warning/Low, `N`=Normal) |
| 21 | `anncumm` | Annual cumulative rainfall |
| 22 | `dailycumm` | Daily cumulative rainfall |
| 23 | `hourlycumm` | Hourly rainfall |
| 24 | `currrf` | Current rainfall |
| 36 | `waterlevel` | Current water level reading |
## Data Processing Logic
### Rainfall Data
1. Check if `dailycumm` or `hourlycumm` is not null
2. Check if record already exists for this station+timestamp
3. If new, INSERT into `rainfall` table
4. **Threshold check**: If `hourlycumm >= 30`:
- `30 <= hourly < 60` → **Warning** level
- `hourly >= 60` → **Danger** level
- INSERT into `notification` table
- Send push notification via Laravel API
### Water Level Data
1. Check if `waterlevel` is not null
2. Check if record already exists for this station+datetime
3. If new, INSERT into `waterlevel` table (with alert/warning/danger thresholds)
4. **Threshold check**: If `waterlevel >= alert`:
- `alert <= wl < warning` → **Alert** level
- `warning <= wl < danger` → **Warning** level
- `wl >= danger` → **Danger** level
- INSERT into `notification` table
- Send push notification via Laravel API
### Siren Data
1. Check if `sirenid` is not null
2. Check if record already exists for this station+active_time
3. Determine level from siren status:
- `H` → **Danger**
- `L` → **Warning**
- `N` → **Normal**
4. INSERT into `siren` table
5. If level is not Normal, send push notification via Laravel API
## Alert Notification Flow
When a threshold is triggered, the script calls `send_alert_to_laravel()`:
```python
def send_alert_to_laravel(stationid, level, stationtype):
payload = {
"stationid": stationid,
"level": level,
"stationtype": stationtype, # 1=rainfall, 2=waterlevel, 3=siren
}
response = requests.post("https://sides.tck.com.my/api/alert", json=payload, timeout=5)
```
This hits the Laravel `AlertController` which:
1. Builds notification title/body based on station type and level
2. Calls `FcmService::sendToTopic()` which:
- Reads Firebase service account credentials
- Gets an OAuth2 access token from Google
- Sends FCM message to topic (e.g., `rainfall_warning`)
- Push notification arrives on subscribed mobile devices
## PostgreSQL Connection
The script connects directly to PostgreSQL:
```python
pg_host = "192.168.0.211"
pg_database = "sides_db"
pg_user = "tck"
pg_password = "projectdev##1"
```
**Note**: This is a hardcoded external IP, not using the Docker container. The database name is `sides_db` (different from the Docker `.env` which uses `tckdev`).
## File Management (Commented Out)
The script contains (commented out) functions for:
- `move_to_error_folder()` — Move malformed files to an FTP error folder
- `move_to_success_folder()` — Move processed files to a success archive folder
These are currently disabled — files remain in the source folder after processing.
## Log Files
- `autoscript/sidesdecode.log` — Processing output
- `autoscript/sidesdecode_error.log` — Error output
## Known Issues
1. **Hardcoded credentials** — FTP and PostgreSQL credentials are embedded in the script
2. **No deduplication beyond same-timestamp** — If the script runs twice, it skips exact duplicates but has no broader deduplication
3. **Commented out file management** — Processed files are not moved/archived
4. **Water level alert sends `stationtype=1`** instead of `2` (likely a bug at line 378)
5. **No error recovery** — If the script crashes mid-processing, some data may be partially inserted
6. **No connection pooling** — New FTP and database connections each run

203
docs/05-FEATURES.md Normal file
View File

@@ -0,0 +1,203 @@
# Features & User Guide
## Feature Overview
SIDES provides the following feature areas:
### 1. Public Dashboard (Map View)
**Route**: `/` or `/dashboard`
**Access**: Public (no auth required)
**Controller**: `MapController::getCurrentData()`
Displays an interactive Leaflet.js map showing all monitoring stations with their current readings:
- Station markers color-coded by status
- Popup showing latest rainfall, water level, and siren data
- Real-time station data from PostgreSQL
### 2. Rainfall Monitoring
**Route**: `/rainfall`
**Access**: Authenticated users
**Controller**: `RainfallController::index()`
- Table showing rainfall data for all rainfall-enabled stations
- Filter by station and date
- Displays: hourly, daily, and 7-day historical values
- Last data update timestamp
### 3. Rainfall Graph
**Route**: `/rainfall/graph/{stationid}`
**Access**: Authenticated users
**Controller**: `RainfallController::rainfallGraph()`
- Interactive chart showing hourly rainfall for the selected station today
- X-axis: time (HH:MM), Y-axis: rainfall value
### 4. Early Warning Threshold (IDF)
**Route**: `/threshold`
**Access**: Authenticated users
**Controller**: `RainfallController::rainfallSum()`
- 6-hour cumulative rainfall data for early warning
- Shows hourly values and timestamps for each of the last 6 hours
- Filter by station and date
- Threshold graph data available via API
### 5. Historical Rainfall
**Route**: `/rainfall/historical`
**Access**: Authenticated users
**Controller**: `RainfallController::historicalRainfall()`
- Detailed hourly rainfall breakdown (24 hours per row)
- Filter by station and date range
- Shows total 24h rainfall per day
- **Export to Excel**: `/rainfall/historical/export`
### 6. Water Level Monitoring
**Route**: `/waterlevel`
**Access**: Authenticated users
**Controller**: `WaterLevelController::index()`
- Table showing current water level readings
- Displays: water level value, alert/warning/danger thresholds
- Filter by station and date
- Interactive water level graph
### 7. Historical Water Level
**Route**: `/waterlevel/historical`
**Access**: Authenticated users
**Controller**: `WaterLevelController::wlHistory()`
- Hourly water level data for a selected station and date
- Shows threshold levels alongside actual readings
- **Export to Excel**: `/waterlevel/historical/export`
### 8. Siren Monitoring
**Route**: `/siren`
**Access**: Authenticated users
**Controller**: `SirenController::index()`
- Current siren status for all siren-equipped stations
- Shows last 3 days of siren activations
- Siren levels: Normal (N), Warning (L), Danger (H)
### 9. Siren History
**Route**: `/sirenhistory`
**Access**: Authenticated users
**Controller**: `SirenController::SirenHistory()`
- Paginated history of all siren activations (excluding Normal)
- **Export to PDF**: `/export/siren-history/pdf`
### 10. Notifications
**Rainfall Notifications**: `/notificationrf`
**Water Level Notifications**: `/notificationwl`
**Siren Notifications**: `/notificationsiren`
**Access**: Authenticated users
**Controller**: `NotificationController`
- Shows today's threshold-triggered notifications
- Grouped by type (rainfall, water level, siren)
### 11. Notification History
**Rainfall History**: `/historyrf`
**Water Level History**: `/historywl`
**Access**: Authenticated users
**Controller**: `NotificationController`
- Paginated history of all notifications
- **Export to PDF**:
- `/export/rainfall-history/pdf`
- `/export/waterlevel-history/pdf`
### 12. CCTV Links
**Route**: `/cctv`
**Access**: Authenticated users
**Controller**: `cctvController::index()`
- Lists stations with CCTV links (where `cctv_link` is not null and `waterlevel = 1`)
- Links open external CCTV feeds
### 13. Admin: Station Management
**Route**: `/stationmanagement`
**Access**: Admin only (`access_level = 1`)
**Controller**: `AdminController`
- List all stations (paginated, 5 per page)
- Add new station
- Edit station details (name, district, coordinates, type flags, CCTV link)
- Delete station
- Shows station type counts (rainfall, water level, siren)
### 14. Admin: User Management
**Route**: `/usermgmt`
**Access**: Admin only
**Controller**: `AdminController`
- List all users (paginated, 5 per page)
- Add new user
- Edit user details (name, email, access level, block/unblock)
- Reset user password
- Delete user
- Shows user counts by role
### 15. API Endpoints
**Prefix**: `/api/`
**Auth**: Custom login (no token-based auth)
| Endpoint | Method | Description |
|----------|--------|-------------|
| `/api/station/current` | GET | All stations with current data |
| `/api/station/rainfall` | GET | Current rainfall data per station |
| `/api/station/waterlevel` | GET | Current water level data per station |
| `/api/station/notification` | GET | Today's notifications per station |
| `/api/station/history` | GET | 3-day notification history |
| `/api/station/siren` | GET | Current siren status |
| `/api/station/siren/history` | GET | 3-day siren history |
| `/api/login` | POST | Login (username + password) |
| `/api/alert` | POST | Send FCM push notification |
### 16. Profile Management
**Route**: `/profile`
**Access**: Authenticated users
**Controller**: `ProfileController` (Breeze scaffold)
- Update name and email
- Update password
- Delete account
### 17. Localization
**Route**: `/locale/{locale}`
**Access**: Public
**Controller**: `LocaleController`
- Switch between English (`en`) and Bahasa Malaysia (`bm`)
- Persisted in session
- Applied via `LocalizationMiddleware` on every request
### 18. Authentication
**Routes**: `/login`, `/register`, `/forgot-password`, `/reset-password`
**Access**: Guest only (for login/register)
**Scaffold**: Laravel Breeze
- Email/password authentication
- Password reset via email (Gmail SMTP)
- Email verification
- Session-based auth

131
docs/06-DATABASE.md Normal file
View File

@@ -0,0 +1,131 @@
# Database Schema
## Tables
### `station`
Primary table storing telemetry station metadata.
| Column | Type | Description |
|--------|------|-------------|
| `stationid` | `varchar` (PK) | Unique station identifier (e.g., "KBLG0026") |
| `name` | `varchar(255)` | Station display name |
| `district` | `varchar(255)` | District location |
| `lng` | `float` | Longitude coordinate |
| `lat` | `float` | Latitude coordinate |
| `mainriverbasin` | `varchar(255)` | Main river basin name |
| `subriverbasin` | `varchar(255)` | Sub river basin name |
| `rainfall` | `integer` | Has rainfall sensor (1=yes, 0=no) |
| `waterlevel` | `integer` | Has water level sensor (1=yes, 0=no) |
| `siren` | `integer` (nullable) | Has siren (1=yes, 0=no) |
| `cctv_link` | `varchar(500)` (nullable) | URL to CCTV feed |
### `rainfall`
Stores rainfall readings from telemetry stations.
| Column | Type | Description |
|--------|------|-------------|
| `id` | `bigint` (PK, auto) | Auto-increment ID |
| `stationid` | `varchar` | Station identifier (FK to station) |
| `timestamp` | `timestamp` | Reading timestamp |
| `anncum` | `double` | Annual cumulative rainfall |
| `daily` | `double` | Daily cumulative rainfall |
| `hourly` | `double` | Hourly rainfall |
| `currentrf` | `double` | Current rainfall |
| `battery` | `double` | Battery voltage |
### `waterlevel`
Stores water level readings with threshold values.
| Column | Type | Description |
|--------|------|-------------|
| `id` | `bigint` (PK, auto) | Auto-increment ID |
| `stationid` | `varchar` | Station identifier (FK to station) |
| `datetime` | `timestamp` | Reading timestamp |
| `waterlevel` | `double` | Current water level (meters) |
| `alert` | `double` | Alert threshold level |
| `warning` | `double` | Warning threshold level |
| `danger` | `double` | Danger threshold level |
### `siren`
Stores siren activation records.
| Column | Type | Description |
|--------|------|-------------|
| `id` | `bigint` (PK, auto) | Auto-increment ID |
| `stationid` | `varchar` | Station identifier (FK to station) |
| `stationtype` | `varchar` | Station type identifier |
| `active_time` | `timestamp` | Siren activation time |
| `level` | `varchar` | Siren level (`H`=Danger, `L`=Warning, `N`=Normal) |
### `notification`
Stores threshold-exceeded alert records.
| Column | Type | Description |
|--------|------|-------------|
| `id` | `bigint` (PK, auto) | Auto-increment ID |
| `stationid` | `varchar` | Station identifier (FK to station) |
| `timestamp` | `timestamp` | Alert timestamp |
| `stationtype` | `integer` | Type: 1=rainfall, 2=waterlevel, 3=siren |
| `level` | `varchar` | Alert level (Alert, Warning, Danger) |
| `active_time` | `timestamp` (nullable) | Activation time |
### `users`
Stores application users.
| Column | Type | Description |
|--------|------|-------------|
| `id` | `bigint` (PK, auto) | Auto-increment ID |
| `name` | `varchar(255)` | Username |
| `email` | `varchar(255)` (unique) | Email address |
| `email_verified_at` | `timestamp` (nullable) | Email verification timestamp |
| `password` | `varchar(255)` | Bcrypt-hashed password |
| `access_level` | `integer` | 1=Admin, 2=User |
| `is_blocked` | `boolean` | Account blocked status |
| `login_attempts` | `integer` | Failed login attempt count |
| `remember_token` | `varchar` | Remember me token |
| `created_at`, `updated_at` | `timestamp` | Laravel timestamps |
### Laravel Standard Tables
- `password_reset_tokens` — Password reset tokens
- `sessions` — Database-backed sessions
- `cache` / `cache_locks` — Cache store
- `jobs` / `job_batches` / `failed_jobs` — Queue system
- `migrations` — Migration tracking
## Relationships
```
station (1) ──< (many) rainfall via stationid
station (1) ──< (many) waterlevel via stationid
station (1) ──< (many) siren via stationid
station (1) ──< (many) notification via stationid
```
**Note**: No database-level foreign keys or constraints exist. All relationships are maintained at the application level.
## Indexes
- `users.email` — unique index
- `sessions.last_activity` — index
- `sessions.user_id` — index
- No additional indexes on data tables (potential performance concern)
## Default Data
### Default Admin User
Created via `DatabaseSeeder` and migration `2025_12_11_124201_add_default_user_to_users_table.php`:
- **Username**: `admin` (seeder) / `admin` (migration)
- **Email**: `admin@example.com`
- **Password**: `password123`
- **Access Level**: 1 (Admin)
**Note**: The admin user is created in both the seeder AND a migration, which would cause a duplicate key error if both run.

139
docs/07-API.md Normal file
View File

@@ -0,0 +1,139 @@
# API Reference
## Authentication
### POST `/api/login`
Login via API (returns user info, no token).
**Request Body:**
```json
{
"username": "admin",
"password": "password123"
}
```
**Success Response (200):**
```json
{
"error": false,
"id": 1,
"username": "admin",
"email": "admin@example.com",
"acc_lvl": 1
}
```
**Error Response (200):**
```json
{
"error": true,
"message": "Wrong Password/Username"
}
```
**Notes:**
- No authentication token is generated — this endpoint only validates credentials
- No session is created — API endpoints are stateless
- All other API endpoints have **no authentication** — they are publicly accessible
---
## Station Data
### GET `/api/station/current`
Returns all stations with their latest rainfall, water level, and siren data.
**Response:**
```json
[
{
"stationid": "KBLG0026",
"name": "Stesen Kupang",
"district": "Baling",
"lng": 100.7521,
"lat": 5.7879,
"rainfall": 1,
"waterlevel": 1,
"siren": 1,
"rainfall_value": 12.5,
"rainfall_time": "2025-11-06T14:00:00",
"waterlevel_value": 3.2,
"waterlevel_time": "2025-11-06T14:00:00",
"siren_level": "N",
"siren_time": "2025-11-06T14:00:00"
}
]
```
### GET `/api/station/rainfall`
Returns latest rainfall data for rainfall-enabled stations.
### GET `/api/station/waterlevel`
Returns latest water level data for waterlevel-enabled stations.
### GET `/api/station/notification`
Returns today's latest notification per station.
### GET `/api/station/history`
Returns notification history for the last 3 days (latest per station per day).
### GET `/api/station/siren`
Returns current siren status for siren-equipped stations (last 3 days).
### GET `/api/station/siren/history`
Returns siren history for the last 3 days (excluding Normal level).
---
## Alert (Push Notification)
### POST `/api/alert`
Sends an FCM push notification. Called by the Python autoscript.
**Request Body:**
```json
{
"stationid": "KBLG0026",
"level": "Warning",
"stationtype": 1
}
```
**Parameters:**
| Field | Type | Description |
|-------|------|-------------|
| `stationid` | string | Station identifier |
| `level` | string | Alert level: "Alert", "Warning", "Danger" |
| `stationtype` | integer | 1=Rainfall, 2=Water Level, 3=Siren |
**Response:**
```json
{
"status": 200
}
```
The `status` is the HTTP status code returned by the FCM API (200 = success).
---
## Public Web API
### GET `/stations`
Returns all stations with coordinates as JSON (used by the public map).
### GET `/locale/{locale}`
Switches language. Valid values: `en`, `bm`. Redirects back to previous page.

119
docs/08-CONFIGURATION.md Normal file
View File

@@ -0,0 +1,119 @@
# Configuration Reference
## Environment Variables
### Application Settings
| Variable | Default | Description |
|----------|---------|-------------|
| `APP_NAME` | `SIDES` | Application name |
| `APP_ENV` | `local` | Environment (local/production) |
| `APP_KEY` | (generated) | Encryption key |
| `APP_DEBUG` | `true` | Show debug errors |
| `APP_URL` | `https://sides.tck.com.my` | Application URL |
| `APP_LOCALE` | `en` | Default language |
| `APP_TIMEZONE` | `Asia/Kuala_Lumpur` | Timezone (set in config/app.php) |
### Database
| Variable | Value | Description |
|----------|-------|-------------|
| `DB_CONNECTION` | `pgsql` | Database driver |
| `DB_HOST` | `192.168.0.211` | Database host (external IP) |
| `DB_PORT` | `5432` | PostgreSQL port |
| `DB_DATABASE` | `sides_db` | Database name (via `POSTGRES_DB`) |
| `DB_USERNAME` | `tck` | Database user (via `POSTGRES_USER`) |
| `DB_PASSWORD` | `projectdev##1` | Database password (via `POSTGRES_PASSWORD`) |
### Docker-level Database
| Variable | Value | Description |
|----------|-------|-------------|
| `POSTGRES_DB` | `tckdev` | Docker postgres database name |
| `POSTGRES_USER` | `tck` | Docker postgres user |
| `POSTGRES_PASSWORD` | `projectdev##1` | Docker postgres password |
**Note**: There is a discrepancy — Docker `.env` creates database `tckdev` while Laravel `.env` connects to `sides_db` on an external host.
### Firebase / FCM
| Variable | Description |
|----------|-------------|
| `FIREBASE_PROJECT_ID` | `sides-b4abb` |
| `FIREBASE_CREDENTIALS` | `storage/app/firebase/sides-b4abb-3604a7cf7584.json` |
| `FCM_TOPIC_RAINFALL_WARNING` | `rainfall_warning` |
| `FCM_TOPIC_RAINFALL_DANGER` | `rainfall_danger` |
| `FCM_TOPIC_WATERLEVEL_ALERT` | `waterlevel_alert` |
| `FCM_TOPIC_WATERLEVEL_DANGER` | `waterlevel_danger` |
### Mail (SMTP)
| Variable | Value | Description |
|----------|-------|-------------|
| `MAIL_MAILER` | `smtp` | Mail driver |
| `MAIL_HOST` | `smtp.gmail.com` | Gmail SMTP |
| `MAIL_PORT` | `587` | SMTP port (TLS) |
| `MAIL_USERNAME` | `sideskupang@gmail.com` | Gmail account |
| `MAIL_PASSWORD` | `ipmu zifw bpmf fsyp` | Gmail App Password |
| `MAIL_FROM_ADDRESS` | `sideskupang@gmail.com` | From address |
| `MAIL_FROM_NAME` | `${APP_NAME}` | From name |
### Session & Cache
| Variable | Value | Description |
|----------|-------|-------------|
| `SESSION_DRIVER` | `database` | Session storage |
| `SESSION_LIFETIME` | `120` | Session timeout (minutes) |
| `CACHE_STORE` | `database` | Cache storage |
| `QUEUE_CONNECTION` | `database` | Queue driver |
## Docker Compose Configuration
### Networks
- `tckdev_net` — Custom bridge network for all containers
### Volumes
| Volume | Container | Purpose |
|--------|-----------|---------|
| `./src:/var/www/html` | app, web | Laravel application code |
| `./docker/postgres/data:/var/lib/postgres/data` | postgres | PostgreSQL data |
| `./docker/nginx/default.conf:/etc/nginx/conf.d/default.conf` | web | Nginx config |
| `./backup:/var/lib/pgadmin/storage/...` | pgadmin | pgAdmin backup storage |
### Ports
| Host Port | Container Port | Service |
|-----------|---------------|---------|
| 80 | 80 | Nginx (web) |
| 5432 | 5432 | PostgreSQL |
| 5050 | 80 | pgAdmin |
| 6060 | 8080 | Adminer |
## Nginx Configuration
Located at `docker/nginx/default.conf`:
- Root: `/var/www/html/public`
- PHP processing via FastCGI to `app:9000`
- Security headers: X-Frame-Options, X-XSS-Protection, X-Content-Type-Options
- Laravel URL rewriting: `try_files $uri $uri/ /index.php?$query_string`
- Hidden file access denied (except `.well-known`)
## Laravel Configuration
### Middleware Registration
Registered in `bootstrap/app.php`:
- `admin` → `AdminMiddleware::class` (alias)
- `LocalizationMiddleware` appended to web middleware group
### HTTPS Forcing
`AppServiceProvider::boot()` calls `URL::forceScheme('https')` globally, making all generated URLs use HTTPS.
### Timezone
Set to `Asia/Kuala_Lumpur` in `config/app.php` (UTC+8, Malaysia Time).