docs(phase-02): create phase plans for file management
This commit is contained in:
@@ -0,0 +1,163 @@
|
||||
# Phase 2: Data Persistence & File Management - Research
|
||||
|
||||
**Researched:** 2026-03-12
|
||||
**Phase:** 02-data-persistence-file-management
|
||||
|
||||
## Domain Overview
|
||||
|
||||
Phase 2 implements CSV data logging to flash memory with file management UI. This builds on Phase 1's Flask backend by adding:
|
||||
- File system operations (list, read, delete)
|
||||
- CSV file generation/management
|
||||
- tidEDA formatted export
|
||||
- File management UI page
|
||||
|
||||
## Requirements Analysis
|
||||
|
||||
| Requirement | Description | Implementation Approach |
|
||||
|-------------|-------------|------------------------|
|
||||
| FILE-01 | List CSV files in flash memory | Flask route returning JSON array of file metadata |
|
||||
| FILE-02 | View CSV file contents | Flask route serving file content with pagination |
|
||||
| FILE-03 | Navigate file list (scroll) | Frontend JavaScript with pagination or virtual scroll |
|
||||
| FILE-04 | Delete CSV files | Flask DELETE endpoint with confirmation |
|
||||
| FILE-05 | Generate tidEDA formatted files | Backend transformation endpoint |
|
||||
| BACK-04 | File management API | All above endpoints combined |
|
||||
|
||||
## Technical Implementation
|
||||
|
||||
### Storage Location
|
||||
- **Path:** `/myvscada/logger` (per CONTEXT.md)
|
||||
- **Note:** On Pi, this may need to be relative to project or use a configurable path
|
||||
|
||||
### File Naming Convention
|
||||
- Format: `SP{station_id}_{timestamp}.csv`
|
||||
- Example: `SP001_20260312_143022.csv`
|
||||
|
||||
### API Endpoints Needed
|
||||
|
||||
```
|
||||
GET /api/files - List all CSV files with metadata
|
||||
GET /api/files/<filename> - Get file contents
|
||||
DELETE /api/files/<filename> - Delete a file
|
||||
POST /api/files/export - Generate tidEDA format from CSV
|
||||
```
|
||||
|
||||
### Response Formats
|
||||
|
||||
**GET /api/files**
|
||||
```json
|
||||
{
|
||||
"files": [
|
||||
{
|
||||
"name": "SP001_20260312_143022.csv",
|
||||
"size": 1234,
|
||||
"created": "2026-03-12T14:30:22Z"
|
||||
}
|
||||
],
|
||||
"total": 10
|
||||
}
|
||||
```
|
||||
|
||||
**GET /api/files/<filename>**
|
||||
```json
|
||||
{
|
||||
"name": "SP001_20260312_143022.csv",
|
||||
"content": "timestamp,rainfall,solar,battery\n...",
|
||||
"lines": 50
|
||||
}
|
||||
```
|
||||
|
||||
### tidEDA Format
|
||||
|
||||
Based on standard environmental data transmission formats. The tidEDA format should contain:
|
||||
- Station identification
|
||||
- Timestamp
|
||||
- Sensor readings (rainfall, voltage)
|
||||
- Data quality indicators
|
||||
|
||||
**Suggested structure:**
|
||||
```
|
||||
$STATION,SP001
|
||||
$DATETIME,2026-03-12T14:30:00Z
|
||||
$DATA
|
||||
timestamp,rainfall_mm,solar_v,battery_v
|
||||
2026-03-12T14:30:00,0.2,12.4,12.8
|
||||
...
|
||||
$END
|
||||
```
|
||||
|
||||
## Frontend Implementation
|
||||
|
||||
### New Page: Files (`/files`)
|
||||
- Bootstrap 5.3 template (reuse existing patterns)
|
||||
- File list with:
|
||||
- Filename, size, date
|
||||
- View button
|
||||
- Delete button with confirmation
|
||||
- Export to tidEDA button
|
||||
- Scroll navigation (per FILE-03)
|
||||
- Touch-optimized 48px+ targets
|
||||
|
||||
### UI Components
|
||||
- File list table/card layout
|
||||
- Modal for file content viewing
|
||||
- Confirmation dialogs for delete
|
||||
- Loading states during API calls
|
||||
|
||||
## Dependencies
|
||||
|
||||
No new Python packages required beyond existing Flask:
|
||||
- `os`, `datetime`, `json` (stdlib)
|
||||
- Flask already installed
|
||||
|
||||
## Integration Points
|
||||
|
||||
1. **Main Navigation:** Add "Files" menu item
|
||||
2. **Settings API reuse:** Get station ID from settings
|
||||
3. **Socket.IO:** Optional - file list refresh on changes
|
||||
4. **Template base:** Extend existing Bootstrap 5.3 layout
|
||||
|
||||
## Validation Architecture
|
||||
|
||||
**Success Criteria Validation:**
|
||||
|
||||
1. User can view list of CSV files stored in flash memory
|
||||
- `GET /api/files` returns file array
|
||||
- Files page renders list correctly
|
||||
|
||||
2. User can navigate file list (scroll up/down)
|
||||
- Frontend implements scroll or pagination
|
||||
- Touch targets work on 7" display
|
||||
|
||||
3. User can view contents of selected CSV files
|
||||
- `GET /api/files/<filename>` returns content
|
||||
- Modal/displays shows formatted content
|
||||
|
||||
4. User can delete unwanted CSV files from flash memory
|
||||
- `DELETE /api/files/<filename>` removes file
|
||||
- UI updates after deletion
|
||||
|
||||
5. User can generate tidEDA formatted files
|
||||
- `POST /api/files/export` generates format
|
||||
- Returns downloadable content
|
||||
|
||||
## Risks & Mitigations
|
||||
|
||||
| Risk | Mitigation |
|
||||
|------|------------|
|
||||
| File path not accessible | Use configurable path, default to project data dir |
|
||||
| Large files crash display | Paginate content, limit preview lines |
|
||||
| Concurrent file access | Handle file locked errors gracefully |
|
||||
| tidEDA format incompatibility | Define format clearly, add version field |
|
||||
|
||||
## Implementation Order
|
||||
|
||||
1. Backend API routes first (file operations)
|
||||
2. Frontend file list page
|
||||
3. File viewing modal
|
||||
4. Delete functionality with confirmation
|
||||
5. tidEDA export endpoint
|
||||
6. Integration testing
|
||||
|
||||
---
|
||||
|
||||
*Research complete - ready for planning*
|
||||
Reference in New Issue
Block a user