object-segmentation/ARCHITECTURE.md

Microscopy Object Detection Application - Architecture Design

Project Overview

A desktop application for detecting organelles and membrane branching structures in microscopy images using YOLOv8s, with comprehensive training, validation, and visualization capabilities.

Technology Stack

• ML Framework: Ultralytics YOLOv8 (YOLOv8s.pt model)
• GUI Framework: PySide6 (Qt6 for Python)
• Visualization: pyqtgraph
• Database: SQLite3
• Python Version: 3.8+

Project Structure

object_detection/
├── main.py                          # Application entry point
├── requirements.txt                 # Python dependencies
├── config/
│   └── app_config.yaml             # Application configuration
├── src/
│   ├── __init__.py
│   ├── database/
│   │   ├── __init__.py
│   │   ├── db_manager.py           # Database operations manager
│   │   ├── models.py               # SQLAlchemy ORM models
│   │   └── schema.sql              # Database schema definition
│   ├── model/
│   │   ├── __init__.py
│   │   ├── yolo_wrapper.py         # YOLO model wrapper
│   │   └── inference.py            # Detection inference logic
│   ├── gui/
│   │   ├── __init__.py
│   │   ├── main_window.py          # Main application window
│   │   ├── tabs/
│   │   │   ├── __init__.py
│   │   │   ├── training_tab.py     # Model training interface
│   │   │   ├── validation_tab.py   # Validation metrics viewer
│   │   │   ├── detection_tab.py    # Real-time/batch detection
│   │   │   ├── results_tab.py      # Results viewer and management
│   │   │   └── annotation_tab.py   # Annotation tool (future)
│   │   ├── dialogs/
│   │   │   ├── __init__.py
│   │   │   ├── config_dialog.py    # Settings and configuration
│   │   │   └── model_dialog.py     # Model management dialog
│   │   └── widgets/
│   │       ├── __init__.py
│   │       ├── image_viewer.py     # Custom image display widget
│   │       └── plot_widget.py      # Custom plotting widgets
│   └── utils/
│       ├── __init__.py
│       ├── file_utils.py           # File operations
│       ├── logger.py               # Logging configuration
│       └── config_manager.py       # Configuration management
├── tests/
│   ├── __init__.py
│   ├── test_database.py
│   ├── test_model.py
│   └── test_gui.py
├── data/                           # Default data directory
│   ├── models/                     # Saved models
│   ├── datasets/                   # Training datasets
│   └── results/                    # Detection results
└── docs/
    ├── USER_GUIDE.md
    └── API_REFERENCE.md

Database Schema

Tables Overview

erDiagram
    MODELS ||--o{ DETECTIONS : produces
    IMAGES ||--o{ DETECTIONS : contains
    IMAGES ||--o{ ANNOTATIONS : has
    
    MODELS {
        integer id PK
        string model_name
        string model_version
        string model_path
        string base_model
        datetime created_at
        json training_params
        json metrics
    }
    
    IMAGES {
        integer id PK
        string relative_path
        string filename
        integer width
        integer height
        datetime captured_at
        datetime added_at
        string checksum
    }
    
    DETECTIONS {
        integer id PK
        integer image_id FK
        integer model_id FK
        string class_name
        float x_min
        float y_min
        float x_max
        float y_max
        float confidence
        datetime detected_at
        json metadata
    }
    
    ANNOTATIONS {
        integer id PK
        integer image_id FK
        string class_name
        float x_min
        float y_min
        float x_max
        float y_max
        string annotator
        datetime created_at
        boolean verified
    }

Detailed Schema

models table

Stores information about trained models and their versions.

Column	Type	Constraints	Description
id	INTEGER	PRIMARY KEY	Unique model identifier
model_name	TEXT	NOT NULL	User-friendly model name
model_version	TEXT	NOT NULL	Version string (e.g., "v1.0")
model_path	TEXT	NOT NULL	Path to model weights file
base_model	TEXT	NOT NULL	Base model used (e.g., "yolov8s.pt")
created_at	TIMESTAMP	DEFAULT CURRENT_TIMESTAMP	Model creation timestamp
training_params	JSON		Training hyperparameters
metrics	JSON		Validation metrics (mAP, precision, recall)

images table

Stores metadata about microscopy images.

Column	Type	Constraints	Description
id	INTEGER	PRIMARY KEY	Unique image identifier
relative_path	TEXT	NOT NULL, UNIQUE	Relative path from repository root
filename	TEXT	NOT NULL	Image filename
width	INTEGER		Image width in pixels
height	INTEGER		Image height in pixels
captured_at	TIMESTAMP		Original capture timestamp (if available)
added_at	TIMESTAMP	DEFAULT CURRENT_TIMESTAMP	When added to database
checksum	TEXT		MD5 hash for integrity verification

detections table

Stores object detection results.

Column	Type	Constraints	Description
id	INTEGER	PRIMARY KEY	Unique detection identifier
image_id	INTEGER	FOREIGN KEY	Reference to images table
model_id	INTEGER	FOREIGN KEY	Reference to models table
class_name	TEXT	NOT NULL	Object class (e.g., "organelle", "membrane_branch")
x_min	REAL	NOT NULL	Bounding box left coordinate (normalized 0-1)
y_min	REAL	NOT NULL	Bounding box top coordinate (normalized 0-1)
x_max	REAL	NOT NULL	Bounding box right coordinate (normalized 0-1)
y_max	REAL	NOT NULL	Bounding box bottom coordinate (normalized 0-1)
confidence	REAL	NOT NULL	Detection confidence score (0-1)
detected_at	TIMESTAMP	DEFAULT CURRENT_TIMESTAMP	When detection was performed
metadata	JSON		Additional metadata (processing time, etc.)

annotations table

Stores manual annotations for training data (future feature).

Column	Type	Constraints	Description
id	INTEGER	PRIMARY KEY	Unique annotation identifier
image_id	INTEGER	FOREIGN KEY	Reference to images table
class_name	TEXT	NOT NULL	Annotated object class
x_min	REAL	NOT NULL	Bounding box left coordinate (normalized)
y_min	REAL	NOT NULL	Bounding box top coordinate (normalized)
x_max	REAL	NOT NULL	Bounding box right coordinate (normalized)
y_max	REAL	NOT NULL	Bounding box bottom coordinate (normalized)
annotator	TEXT		Name of person who created annotation
created_at	TIMESTAMP	DEFAULT CURRENT_TIMESTAMP	Annotation timestamp
verified	BOOLEAN	DEFAULT 0	Whether annotation is verified

Application Architecture

Component Diagram

graph TB
    subgraph GUI Layer
        MW[Main Window]
        TT[Training Tab]
        VT[Validation Tab]
        DT[Detection Tab]
        RT[Results Tab]
        AT[Annotation Tab]
    end
    
    subgraph Business Logic
        YW[YOLO Wrapper]
        INF[Inference Engine]
        DBM[Database Manager]
        CM[Config Manager]
    end
    
    subgraph Data Layer
        DB[(SQLite Database)]
        FS[File System]
        YOLO[YOLOv8 Model]
    end
    
    MW --> TT
    MW --> VT
    MW --> DT
    MW --> RT
    MW --> AT
    
    TT --> YW
    VT --> YW
    DT --> INF
    RT --> DBM
    AT --> DBM
    
    YW --> YOLO
    YW --> DBM
    INF --> YOLO
    INF --> DBM
    DBM --> DB
    CM --> FS
    
    YW -.reads.-> FS
    INF -.reads.-> FS

Key Components

1. YOLO Wrapper (src/model/yolo_wrapper.py)

Encapsulates YOLOv8 operations:

• Load pre-trained YOLOv8s model
• Fine-tune on custom microscopy dataset
• Export trained models
• Provide training progress callbacks
• Calculate validation metrics

Key Methods:

class YOLOWrapper:
    def __init__(self, model_path: str = "yolov8s.pt")
    def train(self, data_yaml: str, epochs: int, callbacks: dict)
    def validate(self, data_yaml: str) -> dict
    def predict(self, image_path: str, conf: float) -> list
    def export_model(self, format: str, output_path: str)

2. Database Manager (src/database/db_manager.py)

Handles all database operations:

• Connection management
• CRUD operations for all tables
• Query builders for filtering and searching
• Transaction management
• Database migrations

Key Methods:

class DatabaseManager:
    def __init__(self, db_path: str)
    def add_detection(self, detection_data: dict) -> int
    def add_image(self, image_data: dict) -> int
    def add_model(self, model_data: dict) -> int
    def get_detections(self, filters: dict) -> list
    def export_detections(self, format: str, output_path: str)

3. Inference Engine (src/model/inference.py)

Manages detection operations:

• Real-time single image detection
• Batch processing with progress tracking
• Result formatting and storage
• Confidence thresholding

Key Methods:

class InferenceEngine:
    def __init__(self, model_path: str, db_manager: DatabaseManager)
    def detect_single(self, image_path: str, conf: float) -> dict
    def detect_batch(self, image_paths: list, conf: float, callback: callable)
    def save_results(self, results: list, model_id: int)

4. GUI Components

Main Window (src/gui/main_window.py)

• Tab container for different functionalities
• Menu bar with File, Tools, Help
• Status bar with progress indicators
• Global settings and repository configuration

Training Tab (src/gui/tabs/training_tab.py)

Features:

• Dataset path selection (YOLO format)
• Training parameters configuration (epochs, batch size, learning rate)
• Start/stop training controls
• Real-time training progress with pyqtgraph plots:
  • Loss curves (training and validation)
  • mAP progression
• Training logs display

Validation Tab (src/gui/tabs/validation_tab.py)

Features:

• Model selection for validation
• Validation dataset selection
• Metrics visualization:
  • Confusion matrix
  • Precision-Recall curves
  • mAP by class
• Results table with per-class metrics

Detection Tab (src/gui/tabs/detection_tab.py)

Features:

• Model selection dropdown
• Detection modes:
  • Single image detection
  • Batch folder processing
  • Real-time camera feed (future)
• Confidence threshold slider
• Image viewer with bounding box overlay
• Detection results panel
• Save to database button

Results Tab (src/gui/tabs/results_tab.py)

Features:

• Searchable detection history table
• Filters by date, model, class, confidence
• Image preview with detections overlay
• Statistics dashboard with pyqtgraph:
  • Detections per class histogram
  • Confidence distribution
  • Detection timeline
• Export options (CSV, JSON, Excel)

Annotation Tab (src/gui/tabs/annotation_tab.py)

Future feature for manual annotation:

• Image browser and viewer
• Drawing tools for bounding boxes
• Class label assignment
• Annotation export to YOLO format
• Annotation statistics

Workflow Diagrams

Training Workflow

sequenceDiagram
    participant User
    participant TrainingTab
    participant YOLOWrapper
    participant DBManager
    participant Database
    
    User->>TrainingTab: Configure training parameters
    User->>TrainingTab: Select dataset (YOLO format)
    User->>TrainingTab: Click "Start Training"
    TrainingTab->>YOLOWrapper: train(data_yaml, epochs, callbacks)
    
    loop Every epoch
        YOLOWrapper-->>TrainingTab: Progress callback (loss, metrics)
        TrainingTab->>TrainingTab: Update plots
    end
    
    YOLOWrapper-->>TrainingTab: Training complete (model_path)
    TrainingTab->>DBManager: add_model(model_info)
    DBManager->>Database: INSERT INTO models
    Database-->>DBManager: model_id
    TrainingTab->>User: Show completion message

Detection Workflow

sequenceDiagram
    participant User
    participant DetectionTab
    participant InferenceEngine
    participant YOLOWrapper
    participant DBManager
    participant Database
    
    User->>DetectionTab: Select model
    User->>DetectionTab: Choose image/folder
    User->>DetectionTab: Set confidence threshold
    User->>DetectionTab: Click "Detect"
    
    DetectionTab->>InferenceEngine: detect_batch(images, conf)
    
    loop For each image
        InferenceEngine->>YOLOWrapper: predict(image_path, conf)
        YOLOWrapper-->>InferenceEngine: detections list
        InferenceEngine->>DBManager: add_image(image_info)
        DBManager->>Database: INSERT INTO images
        Database-->>DBManager: image_id
        
        loop For each detection
            InferenceEngine->>DBManager: add_detection(detection_info)
            DBManager->>Database: INSERT INTO detections
        end
        
        InferenceEngine-->>DetectionTab: Progress update
    end
    
    InferenceEngine-->>DetectionTab: All detections complete
    DetectionTab->>User: Display results

Configuration Management

Application Configuration (config/app_config.yaml)

database:
  path: "data/detections.db"

image_repository:
  base_path: ""  # Set by user
  allowed_extensions: [".jpg", ".jpeg", ".png", ".tif", ".tiff"]

models:
  default_base_model: "yolov8s.pt"
  models_directory: "data/models"

training:
  default_epochs: 100
  default_batch_size: 16
  default_imgsz: 640
  default_patience: 50

detection:
  default_confidence: 0.25
  max_batch_size: 100

visualization:
  colors:
    organelle: "#FF6B6B"
    membrane_branch: "#4ECDC4"
  bbox_thickness: 2

export:
  formats: ["csv", "json", "excel"]
  default_format: "csv"

logging:
  level: "INFO"
  file: "logs/app.log"

Data Flow

Image Repository Structure

image_repository/
├── train/
│   ├── images/
│   │   ├── img001.png
│   │   └── ...
│   └── labels/
│       ├── img001.txt  (YOLO format)
│       └── ...
├── val/
│   ├── images/
│   └── labels/
└── test/
    └── images/

YOLO Format Labels

Each label file contains lines with format:

<class_id> <x_center> <y_center> <width> <height>

All coordinates normalized to 0-1 range.

Error Handling Strategy

1. Database Errors: Graceful degradation, show error dialog, log to file
2. Model Loading Errors: Validate model before loading, clear error messages
3. File Access Errors: Check permissions, validate paths, inform user
4. Training Errors: Catch ultralytics exceptions, save partial progress
5. Memory Errors: Implement batch processing limits, monitor memory usage

Performance Considerations

1. Database:

   • Indexed columns: image_id, model_id, class_name, detected_at
   • Batch inserts for multiple detections
   • Connection pooling

2. Image Processing:

   • Lazy loading for large datasets
   • Thumbnail generation for previews
   • Async operations for batch processing

3. GUI:

   • Worker threads for long operations (training, batch detection)
   • Progress signals for UI updates
   • Image caching for faster display

Testing Strategy

1. Unit Tests:

   • Database operations
   • YOLO wrapper methods
   • Utility functions

2. Integration Tests:

   • End-to-end training workflow
   • Detection and storage pipeline
   • Export functionality

3. GUI Tests:

   • User interaction flows
   • Signal/slot connections
   • Widget state management

Future Enhancements

1. Phase 1 (Core Features):

   • Basic training, validation, detection
   • Results storage and visualization
   • Model management

2. Phase 2 (Enhanced Features):

   • Annotation tool implementation
   • Advanced filtering and search
   • Model comparison tools
   • Batch exports

3. Phase 3 (Advanced Features):

   • Real-time camera detection
   • Multi-model ensemble
   • Cloud storage integration
   • Collaborative annotation

Dependencies

Python Packages

ultralytics>=8.0.0
PySide6>=6.5.0
pyqtgraph>=0.13.0
numpy>=1.24.0
opencv-python>=4.8.0
Pillow>=10.0.0
pyyaml>=6.0
sqlalchemy>=2.0.0
pandas>=2.0.0
openpyxl>=3.1.0

System Requirements

• Python 3.8+
• CUDA-capable GPU (recommended for training)
• 8GB RAM minimum
• 2GB disk space for models and data