object-segmentation/ARCHITECTURE.md

# Microscopy Object Detection Application - Architecture Design

## Project Overview

A desktop application for detecting organelles and membrane branching structures in microscopy images using YOLOv8s, with comprehensive training, validation, and visualization capabilities.

## Technology Stack

- **ML Framework**: Ultralytics YOLOv8 (YOLOv8s.pt model)
- **GUI Framework**: PySide6 (Qt6 for Python)
- **Visualization**: pyqtgraph
- **Database**: SQLite3
- **Python Version**: 3.8+

## Project Structure

```
object_detection/
├── main.py                          # Application entry point
├── requirements.txt                 # Python dependencies
├── config/
│   └── app_config.yaml             # Application configuration
├── src/
│   ├── __init__.py
│   ├── database/
│   │   ├── __init__.py
│   │   ├── db_manager.py           # Database operations manager
│   │   ├── models.py               # SQLAlchemy ORM models
│   │   └── schema.sql              # Database schema definition
│   ├── model/
│   │   ├── __init__.py
│   │   ├── yolo_wrapper.py         # YOLO model wrapper
│   │   └── inference.py            # Detection inference logic
│   ├── gui/
│   │   ├── __init__.py
│   │   ├── main_window.py          # Main application window
│   │   ├── tabs/
│   │   │   ├── __init__.py
│   │   │   ├── training_tab.py     # Model training interface
│   │   │   ├── validation_tab.py   # Validation metrics viewer
│   │   │   ├── detection_tab.py    # Real-time/batch detection
│   │   │   ├── results_tab.py      # Results viewer and management
│   │   │   └── annotation_tab.py   # Annotation tool (future)
│   │   ├── dialogs/
│   │   │   ├── __init__.py
│   │   │   ├── config_dialog.py    # Settings and configuration
│   │   │   └── model_dialog.py     # Model management dialog
│   │   └── widgets/
│   │       ├── __init__.py
│   │       ├── image_viewer.py     # Custom image display widget
│   │       └── plot_widget.py      # Custom plotting widgets
│   └── utils/
│       ├── __init__.py
│       ├── file_utils.py           # File operations
│       ├── logger.py               # Logging configuration
│       └── config_manager.py       # Configuration management
├── tests/
│   ├── __init__.py
│   ├── test_database.py
│   ├── test_model.py
│   └── test_gui.py
├── data/                           # Default data directory
│   ├── models/                     # Saved models
│   ├── datasets/                   # Training datasets
│   └── results/                    # Detection results
└── docs/
    ├── USER_GUIDE.md
    └── API_REFERENCE.md
```

## Database Schema

### Tables Overview

```mermaid
erDiagram
    MODELS ||--o{ DETECTIONS : produces
    IMAGES ||--o{ DETECTIONS : contains
    IMAGES ||--o{ ANNOTATIONS : has
    
    MODELS {
        integer id PK
        string model_name
        string model_version
        string model_path
        string base_model
        datetime created_at
        json training_params
        json metrics
    }
    
    IMAGES {
        integer id PK
        string relative_path
        string filename
        integer width
        integer height
        datetime captured_at
        datetime added_at
        string checksum
    }
    
    DETECTIONS {
        integer id PK
        integer image_id FK
        integer model_id FK
        string class_name
        float x_min
        float y_min
        float x_max
        float y_max
        float confidence
        datetime detected_at
        json metadata
    }
    
    ANNOTATIONS {
        integer id PK
        integer image_id FK
        string class_name
        float x_min
        float y_min
        float x_max
        float y_max
        string annotator
        datetime created_at
        boolean verified
    }
```

### Detailed Schema

#### **models** table
Stores information about trained models and their versions.

| Column | Type | Constraints | Description |
|--------|------|-------------|-------------|
| id | INTEGER | PRIMARY KEY | Unique model identifier |
| model_name | TEXT | NOT NULL | User-friendly model name |
| model_version | TEXT | NOT NULL | Version string (e.g., "v1.0") |
| model_path | TEXT | NOT NULL | Path to model weights file |
| base_model | TEXT | NOT NULL | Base model used (e.g., "yolov8s.pt") |
| created_at | TIMESTAMP | DEFAULT CURRENT_TIMESTAMP | Model creation timestamp |
| training_params | JSON | | Training hyperparameters |
| metrics | JSON | | Validation metrics (mAP, precision, recall) |

#### **images** table
Stores metadata about microscopy images.

| Column | Type | Constraints | Description |
|--------|------|-------------|-------------|
| id | INTEGER | PRIMARY KEY | Unique image identifier |
| relative_path | TEXT | NOT NULL, UNIQUE | Relative path from repository root |
| filename | TEXT | NOT NULL | Image filename |
| width | INTEGER | | Image width in pixels |
| height | INTEGER | | Image height in pixels |
| captured_at | TIMESTAMP | | Original capture timestamp (if available) |
| added_at | TIMESTAMP | DEFAULT CURRENT_TIMESTAMP | When added to database |
| checksum | TEXT | | MD5 hash for integrity verification |

#### **detections** table
Stores object detection results.

| Column | Type | Constraints | Description |
|--------|------|-------------|-------------|
| id | INTEGER | PRIMARY KEY | Unique detection identifier |
| image_id | INTEGER | FOREIGN KEY | Reference to images table |
| model_id | INTEGER | FOREIGN KEY | Reference to models table |
| class_name | TEXT | NOT NULL | Object class (e.g., "organelle", "membrane_branch") |
| x_min | REAL | NOT NULL | Bounding box left coordinate (normalized 0-1) |
| y_min | REAL | NOT NULL | Bounding box top coordinate (normalized 0-1) |
| x_max | REAL | NOT NULL | Bounding box right coordinate (normalized 0-1) |
| y_max | REAL | NOT NULL | Bounding box bottom coordinate (normalized 0-1) |
| confidence | REAL | NOT NULL | Detection confidence score (0-1) |
| detected_at | TIMESTAMP | DEFAULT CURRENT_TIMESTAMP | When detection was performed |
| metadata | JSON | | Additional metadata (processing time, etc.) |

#### **annotations** table
Stores manual annotations for training data (future feature).

| Column | Type | Constraints | Description |
|--------|------|-------------|-------------|
| id | INTEGER | PRIMARY KEY | Unique annotation identifier |
| image_id | INTEGER | FOREIGN KEY | Reference to images table |
| class_name | TEXT | NOT NULL | Annotated object class |
| x_min | REAL | NOT NULL | Bounding box left coordinate (normalized) |
| y_min | REAL | NOT NULL | Bounding box top coordinate (normalized) |
| x_max | REAL | NOT NULL | Bounding box right coordinate (normalized) |
| y_max | REAL | NOT NULL | Bounding box bottom coordinate (normalized) |
| annotator | TEXT | | Name of person who created annotation |
| created_at | TIMESTAMP | DEFAULT CURRENT_TIMESTAMP | Annotation timestamp |
| verified | BOOLEAN | DEFAULT 0 | Whether annotation is verified |

## Application Architecture

### Component Diagram

```mermaid
graph TB
    subgraph GUI Layer
        MW[Main Window]
        TT[Training Tab]
        VT[Validation Tab]
        DT[Detection Tab]
        RT[Results Tab]
        AT[Annotation Tab]
    end
    
    subgraph Business Logic
        YW[YOLO Wrapper]
        INF[Inference Engine]
        DBM[Database Manager]
        CM[Config Manager]
    end
    
    subgraph Data Layer
        DB[(SQLite Database)]
        FS[File System]
        YOLO[YOLOv8 Model]
    end
    
    MW --> TT
    MW --> VT
    MW --> DT
    MW --> RT
    MW --> AT
    
    TT --> YW
    VT --> YW
    DT --> INF
    RT --> DBM
    AT --> DBM
    
    YW --> YOLO
    YW --> DBM
    INF --> YOLO
    INF --> DBM
    DBM --> DB
    CM --> FS
    
    YW -.reads.-> FS
    INF -.reads.-> FS
```

### Key Components

#### 1. **YOLO Wrapper** ([`src/model/yolo_wrapper.py`](src/model/yolo_wrapper.py))
Encapsulates YOLOv8 operations:
- Load pre-trained YOLOv8s model
- Fine-tune on custom microscopy dataset
- Export trained models
- Provide training progress callbacks
- Calculate validation metrics

**Key Methods:**
```python
class YOLOWrapper:
    def __init__(self, model_path: str = "yolov8s.pt")
    def train(self, data_yaml: str, epochs: int, callbacks: dict)
    def validate(self, data_yaml: str) -> dict
    def predict(self, image_path: str, conf: float) -> list
    def export_model(self, format: str, output_path: str)
```

#### 2. **Database Manager** ([`src/database/db_manager.py`](src/database/db_manager.py))
Handles all database operations:
- Connection management
- CRUD operations for all tables
- Query builders for filtering and searching
- Transaction management
- Database migrations

**Key Methods:**
```python
class DatabaseManager:
    def __init__(self, db_path: str)
    def add_detection(self, detection_data: dict) -> int
    def add_image(self, image_data: dict) -> int
    def add_model(self, model_data: dict) -> int
    def get_detections(self, filters: dict) -> list
    def export_detections(self, format: str, output_path: str)
```

#### 3. **Inference Engine** ([`src/model/inference.py`](src/model/inference.py))
Manages detection operations:
- Real-time single image detection
- Batch processing with progress tracking
- Result formatting and storage
- Confidence thresholding

**Key Methods:**
```python
class InferenceEngine:
    def __init__(self, model_path: str, db_manager: DatabaseManager)
    def detect_single(self, image_path: str, conf: float) -> dict
    def detect_batch(self, image_paths: list, conf: float, callback: callable)
    def save_results(self, results: list, model_id: int)
```

#### 4. **GUI Components**

##### **Main Window** ([`src/gui/main_window.py`](src/gui/main_window.py))
- Tab container for different functionalities
- Menu bar with File, Tools, Help
- Status bar with progress indicators
- Global settings and repository configuration

##### **Training Tab** ([`src/gui/tabs/training_tab.py`](src/gui/tabs/training_tab.py))
Features:
- Dataset path selection (YOLO format)
- Training parameters configuration (epochs, batch size, learning rate)
- Start/stop training controls
- Real-time training progress with pyqtgraph plots:
  - Loss curves (training and validation)
  - mAP progression
- Training logs display

##### **Validation Tab** ([`src/gui/tabs/validation_tab.py`](src/gui/tabs/validation_tab.py))
Features:
- Model selection for validation
- Validation dataset selection
- Metrics visualization:
  - Confusion matrix
  - Precision-Recall curves
  - mAP by class
- Results table with per-class metrics

##### **Detection Tab** ([`src/gui/tabs/detection_tab.py`](src/gui/tabs/detection_tab.py))
Features:
- Model selection dropdown
- Detection modes:
  - Single image detection
  - Batch folder processing
  - Real-time camera feed (future)
- Confidence threshold slider
- Image viewer with bounding box overlay
- Detection results panel
- Save to database button

##### **Results Tab** ([`src/gui/tabs/results_tab.py`](src/gui/tabs/results_tab.py))
Features:
- Searchable detection history table
- Filters by date, model, class, confidence
- Image preview with detections overlay
- Statistics dashboard with pyqtgraph:
  - Detections per class histogram
  - Confidence distribution
  - Detection timeline
- Export options (CSV, JSON, Excel)

##### **Annotation Tab** ([`src/gui/tabs/annotation_tab.py`](src/gui/tabs/annotation_tab.py))
Future feature for manual annotation:
- Image browser and viewer
- Drawing tools for bounding boxes
- Class label assignment
- Annotation export to YOLO format
- Annotation statistics

## Workflow Diagrams

### Training Workflow

```mermaid
sequenceDiagram
    participant User
    participant TrainingTab
    participant YOLOWrapper
    participant DBManager
    participant Database
    
    User->>TrainingTab: Configure training parameters
    User->>TrainingTab: Select dataset (YOLO format)
    User->>TrainingTab: Click "Start Training"
    TrainingTab->>YOLOWrapper: train(data_yaml, epochs, callbacks)
    
    loop Every epoch
        YOLOWrapper-->>TrainingTab: Progress callback (loss, metrics)
        TrainingTab->>TrainingTab: Update plots
    end
    
    YOLOWrapper-->>TrainingTab: Training complete (model_path)
    TrainingTab->>DBManager: add_model(model_info)
    DBManager->>Database: INSERT INTO models
    Database-->>DBManager: model_id
    TrainingTab->>User: Show completion message
```

### Detection Workflow

```mermaid
sequenceDiagram
    participant User
    participant DetectionTab
    participant InferenceEngine
    participant YOLOWrapper
    participant DBManager
    participant Database
    
    User->>DetectionTab: Select model
    User->>DetectionTab: Choose image/folder
    User->>DetectionTab: Set confidence threshold
    User->>DetectionTab: Click "Detect"
    
    DetectionTab->>InferenceEngine: detect_batch(images, conf)
    
    loop For each image
        InferenceEngine->>YOLOWrapper: predict(image_path, conf)
        YOLOWrapper-->>InferenceEngine: detections list
        InferenceEngine->>DBManager: add_image(image_info)
        DBManager->>Database: INSERT INTO images
        Database-->>DBManager: image_id
        
        loop For each detection
            InferenceEngine->>DBManager: add_detection(detection_info)
            DBManager->>Database: INSERT INTO detections
        end
        
        InferenceEngine-->>DetectionTab: Progress update
    end
    
    InferenceEngine-->>DetectionTab: All detections complete
    DetectionTab->>User: Display results
```

## Configuration Management

### Application Configuration ([`config/app_config.yaml`](config/app_config.yaml))

```yaml
database:
  path: "data/detections.db"

image_repository:
  base_path: ""  # Set by user
  allowed_extensions: [".jpg", ".jpeg", ".png", ".tif", ".tiff"]

models:
  default_base_model: "yolov8s.pt"
  models_directory: "data/models"

training:
  default_epochs: 100
  default_batch_size: 16
  default_imgsz: 640
  default_patience: 50

detection:
  default_confidence: 0.25
  max_batch_size: 100

visualization:
  colors:
    organelle: "#FF6B6B"
    membrane_branch: "#4ECDC4"
  bbox_thickness: 2

export:
  formats: ["csv", "json", "excel"]
  default_format: "csv"

logging:
  level: "INFO"
  file: "logs/app.log"
```

## Data Flow

### Image Repository Structure
```
image_repository/
├── train/
│   ├── images/
│   │   ├── img001.png
│   │   └── ...
│   └── labels/
│       ├── img001.txt  (YOLO format)
│       └── ...
├── val/
│   ├── images/
│   └── labels/
└── test/
    └── images/
```

### YOLO Format Labels
Each label file contains lines with format:
```
<class_id> <x_center> <y_center> <width> <height>
```
All coordinates normalized to 0-1 range.

## Error Handling Strategy

1. **Database Errors**: Graceful degradation, show error dialog, log to file
2. **Model Loading Errors**: Validate model before loading, clear error messages
3. **File Access Errors**: Check permissions, validate paths, inform user
4. **Training Errors**: Catch ultralytics exceptions, save partial progress
5. **Memory Errors**: Implement batch processing limits, monitor memory usage

## Performance Considerations

1. **Database**:
   - Indexed columns: image_id, model_id, class_name, detected_at
   - Batch inserts for multiple detections
   - Connection pooling

2. **Image Processing**:
   - Lazy loading for large datasets
   - Thumbnail generation for previews
   - Async operations for batch processing

3. **GUI**:
   - Worker threads for long operations (training, batch detection)
   - Progress signals for UI updates
   - Image caching for faster display

## Testing Strategy

1. **Unit Tests**:
   - Database operations
   - YOLO wrapper methods
   - Utility functions

2. **Integration Tests**:
   - End-to-end training workflow
   - Detection and storage pipeline
   - Export functionality

3. **GUI Tests**:
   - User interaction flows
   - Signal/slot connections
   - Widget state management

## Future Enhancements

1. **Phase 1** (Core Features):
   - Basic training, validation, detection
   - Results storage and visualization
   - Model management

2. **Phase 2** (Enhanced Features):
   - Annotation tool implementation
   - Advanced filtering and search
   - Model comparison tools
   - Batch exports

3. **Phase 3** (Advanced Features):
   - Real-time camera detection
   - Multi-model ensemble
   - Cloud storage integration
   - Collaborative annotation

## Dependencies

### Python Packages
```
ultralytics>=8.0.0
PySide6>=6.5.0
pyqtgraph>=0.13.0
numpy>=1.24.0
opencv-python>=4.8.0
Pillow>=10.0.0
pyyaml>=6.0
sqlalchemy>=2.0.0
pandas>=2.0.0
openpyxl>=3.1.0
```

### System Requirements
- Python 3.8+
- CUDA-capable GPU (recommended for training)
- 8GB RAM minimum
- 2GB disk space for models and data