Adding test scripts
This commit is contained in:
298
docs/16BIT_TIFF_SUPPORT.md
Normal file
298
docs/16BIT_TIFF_SUPPORT.md
Normal file
@@ -0,0 +1,298 @@
|
|||||||
|
# 16-bit TIFF Support for YOLO Object Detection
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
|
||||||
|
This document describes the implementation of 16-bit grayscale TIFF support for YOLO object detection. The system properly loads 16-bit TIFF images, normalizes them to float32 [0-1], and passes them directly to YOLO **without uint8 conversion** to preserve the full dynamic range and avoid data loss.
|
||||||
|
|
||||||
|
## Key Features
|
||||||
|
|
||||||
|
✅ Reads 16-bit or float32 images using tifffile
|
||||||
|
✅ Converts to float32 [0-1] (NO uint8 conversion)
|
||||||
|
✅ Replicates grayscale → RGB (3 channels)
|
||||||
|
✅ Passes numpy arrays directly to YOLO (no file I/O)
|
||||||
|
✅ Uses Ultralytics YOLOv8/v11 models
|
||||||
|
✅ Works with segmentation models
|
||||||
|
✅ No data loss, no double normalization, no silent clipping
|
||||||
|
|
||||||
|
## Changes Made
|
||||||
|
|
||||||
|
### 1. Dependencies ([`requirements.txt`](../requirements.txt:14))
|
||||||
|
- Added `tifffile>=2023.0.0` for reliable 16-bit TIFF loading
|
||||||
|
|
||||||
|
### 2. Image Loading ([`src/utils/image.py`](../src/utils/image.py))
|
||||||
|
|
||||||
|
#### Enhanced TIFF Loading
|
||||||
|
- Modified [`Image._load()`](../src/utils/image.py:87) to use `tifffile` for `.tif` and `.tiff` files
|
||||||
|
- Preserves original 16-bit data type during loading
|
||||||
|
- Properly handles both grayscale and multi-channel TIFF files
|
||||||
|
|
||||||
|
#### New Normalization Method
|
||||||
|
Added [`Image.to_normalized_float32()`](../src/utils/image.py:280) method that:
|
||||||
|
- Converts image data to `float32`
|
||||||
|
- Properly scales values to [0, 1] range:
|
||||||
|
- **16-bit images**: divides by 65535 (full dynamic range)
|
||||||
|
- 8-bit images: divides by 255
|
||||||
|
- Float images: clips to [0, 1]
|
||||||
|
- Handles various data types automatically
|
||||||
|
|
||||||
|
### 3. YOLO Preprocessing ([`src/model/yolo_wrapper.py`](../src/model/yolo_wrapper.py))
|
||||||
|
|
||||||
|
Enhanced [`YOLOWrapper._prepare_source()`](../src/model/yolo_wrapper.py:231) to:
|
||||||
|
1. Detect 16-bit TIFF files automatically
|
||||||
|
2. Load and normalize to float32 [0-1] using the new method
|
||||||
|
3. Replicate grayscale to RGB (3 channels)
|
||||||
|
4. **Return numpy array directly** (NO file saving, NO uint8 conversion)
|
||||||
|
5. Pass float32 array directly to YOLO for inference
|
||||||
|
|
||||||
|
## Processing Pipeline
|
||||||
|
|
||||||
|
For 16-bit TIFF files:
|
||||||
|
|
||||||
|
1. **Load**: File loaded using `tifffile` → preserves 16-bit uint16 data
|
||||||
|
2. **Normalize**: Convert to float32 and scale to [0, 1]
|
||||||
|
```python
|
||||||
|
float_data = uint16_data.astype(np.float32) / 65535.0
|
||||||
|
```
|
||||||
|
3. **RGB Conversion**: Replicate grayscale to 3 channels
|
||||||
|
```python
|
||||||
|
rgb_float = np.stack([float_data] * 3, axis=-1)
|
||||||
|
```
|
||||||
|
4. **Pass to YOLO**: Return float32 array directly (no uint8, no file I/O)
|
||||||
|
5. **Inference**: YOLO processes the float32 [0-1] RGB array
|
||||||
|
|
||||||
|
### No Data Loss!
|
||||||
|
|
||||||
|
Unlike the previous approach that converted to uint8 (256 levels), the new implementation:
|
||||||
|
- Preserves full 16-bit dynamic range (65536 levels)
|
||||||
|
- Maintains precision with float32 representation
|
||||||
|
- Passes data directly without intermediate file conversions
|
||||||
|
|
||||||
|
## Usage
|
||||||
|
|
||||||
|
### Basic Image Loading
|
||||||
|
|
||||||
|
```python
|
||||||
|
from src.utils.image import Image
|
||||||
|
|
||||||
|
# Load a 16-bit TIFF file
|
||||||
|
img = Image("path/to/16bit_image.tif")
|
||||||
|
|
||||||
|
# Get normalized float32 data [0-1]
|
||||||
|
normalized = img.to_normalized_float32() # Shape: (H, W), dtype: float32
|
||||||
|
|
||||||
|
# Original data is preserved
|
||||||
|
original = img.data # Still uint16
|
||||||
|
```
|
||||||
|
|
||||||
|
### YOLO Inference
|
||||||
|
|
||||||
|
The preprocessing is automatic - just use YOLO as normal:
|
||||||
|
|
||||||
|
```python
|
||||||
|
from src.model.yolo_wrapper import YOLOWrapper
|
||||||
|
|
||||||
|
# Initialize model
|
||||||
|
yolo = YOLOWrapper("yolov8s-seg.pt")
|
||||||
|
yolo.load_model()
|
||||||
|
|
||||||
|
# Perform inference on 16-bit TIFF
|
||||||
|
# The image will be automatically normalized and passed as float32 [0-1]
|
||||||
|
detections = yolo.predict("path/to/16bit_image.tif", conf=0.25)
|
||||||
|
```
|
||||||
|
|
||||||
|
### With InferenceEngine
|
||||||
|
|
||||||
|
```python
|
||||||
|
from src.model.inference import InferenceEngine
|
||||||
|
from src.database.db_manager import DatabaseManager
|
||||||
|
|
||||||
|
# Setup
|
||||||
|
db = DatabaseManager("database.db")
|
||||||
|
engine = InferenceEngine("model.pt", db, model_id=1)
|
||||||
|
|
||||||
|
# Detect objects in 16-bit TIFF
|
||||||
|
result = engine.detect_single(
|
||||||
|
image_path="path/to/16bit_image.tif",
|
||||||
|
relative_path="images/16bit_image.tif",
|
||||||
|
conf=0.25
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
## Testing
|
||||||
|
|
||||||
|
Three test scripts are provided:
|
||||||
|
|
||||||
|
### 1. Image Loading Test
|
||||||
|
```bash
|
||||||
|
./venv/bin/python tests/test_16bit_tiff_loading.py
|
||||||
|
```
|
||||||
|
|
||||||
|
Tests:
|
||||||
|
- Loading 16-bit TIFF files with tifffile
|
||||||
|
- Normalization to float32 [0-1]
|
||||||
|
- Data type and value range verification
|
||||||
|
|
||||||
|
### 2. Float32 Passthrough Test (Most Important!)
|
||||||
|
```bash
|
||||||
|
./venv/bin/python tests/test_yolo_16bit_float32.py
|
||||||
|
```
|
||||||
|
|
||||||
|
Tests:
|
||||||
|
- YOLO preprocessing returns numpy array (not file path)
|
||||||
|
- Data is float32 [0-1] (not uint8)
|
||||||
|
- No quantization to 256 levels (proves no uint8 conversion)
|
||||||
|
- Sample output:
|
||||||
|
```
|
||||||
|
✓ SUCCESS: Prepared source is a numpy array (float32 passthrough)
|
||||||
|
Shape: (200, 200, 3)
|
||||||
|
Dtype: float32
|
||||||
|
Min value: 0.000000
|
||||||
|
Max value: 1.000000
|
||||||
|
Unique values: 399
|
||||||
|
|
||||||
|
✓ SUCCESS: Data has 399 unique values (> 256)
|
||||||
|
This confirms NO uint8 quantization occurred!
|
||||||
|
```
|
||||||
|
|
||||||
|
### 3. Legacy Test (Shows Old Behavior)
|
||||||
|
```bash
|
||||||
|
./venv/bin/python tests/test_yolo_16bit_preprocessing.py
|
||||||
|
```
|
||||||
|
|
||||||
|
This test shows the old behavior (uint8 conversion) - kept for comparison.
|
||||||
|
|
||||||
|
## Benefits
|
||||||
|
|
||||||
|
1. **No Data Loss**: Preserves full 16-bit dynamic range (65536 levels vs 256)
|
||||||
|
2. **High Precision**: Float32 maintains fine-grained intensity differences
|
||||||
|
3. **Automatic Processing**: No manual preprocessing needed
|
||||||
|
4. **YOLO Compatible**: Ultralytics YOLO accepts float32 [0-1] arrays
|
||||||
|
5. **Performance**: No intermediate file I/O for 16-bit TIFFs
|
||||||
|
6. **Backwards Compatible**: Regular images (8-bit PNG, JPEG, etc.) still work as before
|
||||||
|
|
||||||
|
## Technical Notes
|
||||||
|
|
||||||
|
### Float32 vs uint8
|
||||||
|
|
||||||
|
**With uint8 conversion (OLD - BAD):**
|
||||||
|
- 16-bit (65536 levels) → uint8 (256 levels) = **99.6% data loss!**
|
||||||
|
- Fine intensity differences are lost
|
||||||
|
- Quantization artifacts
|
||||||
|
|
||||||
|
**With float32 [0-1] (NEW - GOOD):**
|
||||||
|
- 16-bit (65536 levels) → float32 (continuous) = **No data loss**
|
||||||
|
- Full dynamic range preserved
|
||||||
|
- Smooth gradients maintained
|
||||||
|
|
||||||
|
### Memory Considerations
|
||||||
|
|
||||||
|
For a 2048×2048 single-channel image:
|
||||||
|
|
||||||
|
| Format | Memory | Notes |
|
||||||
|
|--------|--------|-------|
|
||||||
|
| Original 16-bit | 8 MB | uint16 grayscale |
|
||||||
|
| Float32 grayscale | 16 MB | Intermediate |
|
||||||
|
| Float32 RGB | 48 MB | Final (3 channels) |
|
||||||
|
| uint8 RGB (old) | 12 MB | OLD approach with data loss |
|
||||||
|
|
||||||
|
The float32 approach uses ~4× more memory than uint8 but preserves **all information**.
|
||||||
|
|
||||||
|
### Why Direct Numpy Array?
|
||||||
|
|
||||||
|
Passing numpy arrays directly to YOLO (instead of saving to file):
|
||||||
|
|
||||||
|
1. **Faster**: No disk I/O overhead
|
||||||
|
2. **No Quantization**: Avoids PNG/JPEG quantization
|
||||||
|
3. **Memory Efficient**: Single copy in memory
|
||||||
|
4. **Cleaner**: No temp file management
|
||||||
|
|
||||||
|
Ultralytics YOLO supports various input types:
|
||||||
|
- File paths (str): `"image.jpg"`
|
||||||
|
- Numpy arrays: `np.ndarray` ← **we use this**
|
||||||
|
- PIL Images: `PIL.Image`
|
||||||
|
- Torch tensors: `torch.Tensor`
|
||||||
|
|
||||||
|
## For Training with Custom Dataset
|
||||||
|
|
||||||
|
If you need to train YOLO on 16-bit TIFF images, you should create a custom dataset loader similar to the example provided by the user:
|
||||||
|
|
||||||
|
```python
|
||||||
|
import torch
|
||||||
|
import numpy as np
|
||||||
|
import tifffile as tiff
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
class FloatYoloSegDataset(torch.utils.data.Dataset):
|
||||||
|
def __init__(self, img_dir, label_dir, img_size=640):
|
||||||
|
self.img_paths = sorted(Path(img_dir).glob('*'))
|
||||||
|
self.label_dir = Path(label_dir)
|
||||||
|
self.img_size = img_size
|
||||||
|
|
||||||
|
def __len__(self):
|
||||||
|
return len(self.img_paths)
|
||||||
|
|
||||||
|
def __getitem__(self, idx):
|
||||||
|
img_path = self.img_paths[idx]
|
||||||
|
|
||||||
|
# Load 16-bit TIFF
|
||||||
|
img = tiff.imread(img_path)
|
||||||
|
|
||||||
|
# Convert to float32 [0-1]
|
||||||
|
img = img.astype(np.float32)
|
||||||
|
if img.max() > 1.5: # Assume 16-bit if max > 1.5
|
||||||
|
img /= 65535.0
|
||||||
|
|
||||||
|
# Grayscale → RGB
|
||||||
|
if img.ndim == 2:
|
||||||
|
img = np.repeat(img[..., None], 3, axis=2)
|
||||||
|
|
||||||
|
# HWC → CHW for PyTorch
|
||||||
|
img = torch.from_numpy(img).permute(2, 0, 1).contiguous()
|
||||||
|
|
||||||
|
# Load labels...
|
||||||
|
# (implementation depends on your label format)
|
||||||
|
|
||||||
|
return img, labels
|
||||||
|
```
|
||||||
|
|
||||||
|
Then use this dataset with Ultralytics training API or custom training loop.
|
||||||
|
|
||||||
|
## Installation
|
||||||
|
|
||||||
|
Install the updated dependencies:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./venv/bin/pip install -r requirements.txt
|
||||||
|
```
|
||||||
|
|
||||||
|
Or install tifffile directly:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./venv/bin/pip install tifffile>=2023.0.0
|
||||||
|
```
|
||||||
|
|
||||||
|
## Example Test Output
|
||||||
|
|
||||||
|
```
|
||||||
|
=== Testing Float32 Passthrough (NO uint8) ===
|
||||||
|
Created test 16-bit TIFF: /tmp/tmpdt5hm0ab.tif
|
||||||
|
Shape: (200, 200)
|
||||||
|
Dtype: uint16
|
||||||
|
Min value: 0
|
||||||
|
Max value: 65535
|
||||||
|
|
||||||
|
Preprocessing result:
|
||||||
|
Prepared source type: <class 'numpy.ndarray'>
|
||||||
|
|
||||||
|
✓ SUCCESS: Prepared source is a numpy array (float32 passthrough)
|
||||||
|
Shape: (200, 200, 3)
|
||||||
|
Dtype: float32
|
||||||
|
Min value: 0.000000
|
||||||
|
Max value: 1.000000
|
||||||
|
Mean value: 0.499992
|
||||||
|
Unique values: 399
|
||||||
|
|
||||||
|
✓ SUCCESS: Data has 399 unique values (> 256)
|
||||||
|
This confirms NO uint8 quantization occurred!
|
||||||
|
|
||||||
|
✓ All float32 passthrough tests passed!
|
||||||
182
tests/show_yolo_seg.py
Normal file
182
tests/show_yolo_seg.py
Normal file
@@ -0,0 +1,182 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""
|
||||||
|
show_yolo_seg.py
|
||||||
|
|
||||||
|
Usage:
|
||||||
|
python show_yolo_seg.py /path/to/image.jpg /path/to/labels.txt
|
||||||
|
|
||||||
|
Supports:
|
||||||
|
- Segmentation polygons: "class x1 y1 x2 y2 ... xn yn"
|
||||||
|
- YOLO bbox lines as fallback: "class x_center y_center width height"
|
||||||
|
Coordinates can be normalized [0..1] or absolute pixels (auto-detected).
|
||||||
|
"""
|
||||||
|
import sys
|
||||||
|
import cv2
|
||||||
|
import numpy as np
|
||||||
|
import matplotlib.pyplot as plt
|
||||||
|
import argparse
|
||||||
|
from pathlib import Path
|
||||||
|
import random
|
||||||
|
|
||||||
|
|
||||||
|
def parse_label_line(line):
|
||||||
|
parts = line.strip().split()
|
||||||
|
if not parts:
|
||||||
|
return None
|
||||||
|
cls = int(float(parts[0]))
|
||||||
|
coords = [float(x) for x in parts[1:]]
|
||||||
|
return cls, coords
|
||||||
|
|
||||||
|
|
||||||
|
def coords_are_normalized(coords):
|
||||||
|
# If every coordinate is between 0 and 1 (inclusive-ish), assume normalized
|
||||||
|
if not coords:
|
||||||
|
return False
|
||||||
|
return max(coords) <= 1.001
|
||||||
|
|
||||||
|
|
||||||
|
def yolo_bbox_to_xyxy(coords, img_w, img_h):
|
||||||
|
# coords: [xc, yc, w, h] normalized or absolute
|
||||||
|
xc, yc, w, h = coords[:4]
|
||||||
|
if max(coords) <= 1.001:
|
||||||
|
xc *= img_w
|
||||||
|
yc *= img_h
|
||||||
|
w *= img_w
|
||||||
|
h *= img_h
|
||||||
|
x1 = int(round(xc - w / 2))
|
||||||
|
y1 = int(round(yc - h / 2))
|
||||||
|
x2 = int(round(xc + w / 2))
|
||||||
|
y2 = int(round(yc + h / 2))
|
||||||
|
return x1, y1, x2, y2
|
||||||
|
|
||||||
|
|
||||||
|
def poly_to_pts(coords, img_w, img_h):
|
||||||
|
# coords: [x1 y1 x2 y2 ...] either normalized or absolute
|
||||||
|
if coords_are_normalized(coords):
|
||||||
|
coords = [
|
||||||
|
coords[i] * (img_w if i % 2 == 0 else img_h) for i in range(len(coords))
|
||||||
|
]
|
||||||
|
pts = np.array(coords, dtype=np.int32).reshape(-1, 2)
|
||||||
|
return pts
|
||||||
|
|
||||||
|
|
||||||
|
def random_color_for_class(cls):
|
||||||
|
random.seed(cls) # deterministic per class
|
||||||
|
return tuple(int(x) for x in np.array([random.randint(0, 255) for _ in range(3)]))
|
||||||
|
|
||||||
|
|
||||||
|
def draw_annotations(img, labels, alpha=0.4, draw_bbox_for_poly=True):
|
||||||
|
# img: BGR numpy array
|
||||||
|
overlay = img.copy()
|
||||||
|
h, w = img.shape[:2]
|
||||||
|
for cls, coords in labels:
|
||||||
|
if not coords:
|
||||||
|
continue
|
||||||
|
# polygon case (>=6 coordinates)
|
||||||
|
if len(coords) >= 6:
|
||||||
|
pts = poly_to_pts(coords, w, h)
|
||||||
|
color = random_color_for_class(cls)
|
||||||
|
# fill on overlay
|
||||||
|
cv2.fillPoly(overlay, [pts], color)
|
||||||
|
# outline on base image
|
||||||
|
cv2.polylines(img, [pts], isClosed=True, color=color, thickness=2)
|
||||||
|
# put class text at first point
|
||||||
|
x, y = int(pts[0, 0]), int(pts[0, 1]) - 6
|
||||||
|
cv2.putText(
|
||||||
|
img,
|
||||||
|
str(cls),
|
||||||
|
(x, max(6, y)),
|
||||||
|
cv2.FONT_HERSHEY_SIMPLEX,
|
||||||
|
0.6,
|
||||||
|
(255, 255, 255),
|
||||||
|
2,
|
||||||
|
cv2.LINE_AA,
|
||||||
|
)
|
||||||
|
if draw_bbox_for_poly:
|
||||||
|
x, y, w_box, h_box = cv2.boundingRect(pts)
|
||||||
|
cv2.rectangle(img, (x, y), (x + w_box, y + h_box), color, 1)
|
||||||
|
# YOLO bbox case (4 coords)
|
||||||
|
elif len(coords) == 4:
|
||||||
|
x1, y1, x2, y2 = yolo_bbox_to_xyxy(coords, w, h)
|
||||||
|
color = random_color_for_class(cls)
|
||||||
|
cv2.rectangle(img, (x1, y1), (x2, y2), color, 2)
|
||||||
|
cv2.putText(
|
||||||
|
img,
|
||||||
|
str(cls),
|
||||||
|
(x1, max(6, y1 - 4)),
|
||||||
|
cv2.FONT_HERSHEY_SIMPLEX,
|
||||||
|
0.6,
|
||||||
|
(255, 255, 255),
|
||||||
|
2,
|
||||||
|
cv2.LINE_AA,
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
# Unknown / invalid format, skip
|
||||||
|
continue
|
||||||
|
|
||||||
|
# blend overlay for filled polygons
|
||||||
|
cv2.addWeighted(overlay, alpha, img, 1 - alpha, 0, img)
|
||||||
|
return img
|
||||||
|
|
||||||
|
|
||||||
|
def load_labels_file(label_path):
|
||||||
|
labels = []
|
||||||
|
with open(label_path, "r") as f:
|
||||||
|
for raw in f:
|
||||||
|
line = raw.strip()
|
||||||
|
if not line:
|
||||||
|
continue
|
||||||
|
parsed = parse_label_line(line)
|
||||||
|
if parsed:
|
||||||
|
labels.append(parsed)
|
||||||
|
return labels
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
parser = argparse.ArgumentParser(
|
||||||
|
description="Show YOLO segmentation / polygon annotations"
|
||||||
|
)
|
||||||
|
parser.add_argument("image", type=str, help="Path to image file")
|
||||||
|
parser.add_argument("labels", type=str, help="Path to YOLO label file (polygons)")
|
||||||
|
parser.add_argument(
|
||||||
|
"--alpha", type=float, default=0.4, help="Polygon fill alpha (0..1)"
|
||||||
|
)
|
||||||
|
parser.add_argument(
|
||||||
|
"--no-bbox", action="store_true", help="Don't draw bounding boxes for polygons"
|
||||||
|
)
|
||||||
|
args = parser.parse_args()
|
||||||
|
|
||||||
|
img_path = Path(args.image)
|
||||||
|
lbl_path = Path(args.labels)
|
||||||
|
|
||||||
|
if not img_path.exists():
|
||||||
|
print("Image not found:", img_path)
|
||||||
|
sys.exit(1)
|
||||||
|
if not lbl_path.exists():
|
||||||
|
print("Label file not found:", lbl_path)
|
||||||
|
sys.exit(1)
|
||||||
|
|
||||||
|
img = cv2.imread(str(img_path), cv2.IMREAD_COLOR)
|
||||||
|
if img is None:
|
||||||
|
print("Could not load image:", img_path)
|
||||||
|
sys.exit(1)
|
||||||
|
|
||||||
|
labels = load_labels_file(str(lbl_path))
|
||||||
|
if not labels:
|
||||||
|
print("No labels parsed from", lbl_path)
|
||||||
|
# continue and just show image
|
||||||
|
out = draw_annotations(
|
||||||
|
img.copy(), labels, alpha=args.alpha, draw_bbox_for_poly=(not args.no_bbox)
|
||||||
|
)
|
||||||
|
|
||||||
|
# Convert BGR -> RGB for matplotlib display
|
||||||
|
out_rgb = cv2.cvtColor(out, cv2.COLOR_BGR2RGB)
|
||||||
|
plt.figure(figsize=(10, 10 * out.shape[0] / out.shape[1]))
|
||||||
|
plt.imshow(out_rgb)
|
||||||
|
plt.axis("off")
|
||||||
|
plt.title(f"{img_path.name} ({lbl_path.name})")
|
||||||
|
plt.show()
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
109
tests/test_16bit_tiff_loading.py
Normal file
109
tests/test_16bit_tiff_loading.py
Normal file
@@ -0,0 +1,109 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""
|
||||||
|
Test script for 16-bit TIFF loading and normalization.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import numpy as np
|
||||||
|
import tifffile
|
||||||
|
from pathlib import Path
|
||||||
|
import tempfile
|
||||||
|
import sys
|
||||||
|
import os
|
||||||
|
|
||||||
|
# Add parent directory to path to import modules
|
||||||
|
sys.path.insert(0, str(Path(__file__).parent.parent))
|
||||||
|
|
||||||
|
from src.utils.image import Image
|
||||||
|
|
||||||
|
|
||||||
|
def create_test_16bit_tiff(output_path: str) -> str:
|
||||||
|
"""Create a test 16-bit grayscale TIFF file.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
output_path: Path where to save the test TIFF
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
Path to the created TIFF file
|
||||||
|
"""
|
||||||
|
# Create a 16-bit grayscale test image (100x100)
|
||||||
|
# With values ranging from 0 to 65535 (full 16-bit range)
|
||||||
|
height, width = 100, 100
|
||||||
|
|
||||||
|
# Create a gradient pattern
|
||||||
|
test_data = np.zeros((height, width), dtype=np.uint16)
|
||||||
|
for i in range(height):
|
||||||
|
for j in range(width):
|
||||||
|
# Create a diagonal gradient
|
||||||
|
test_data[i, j] = int((i + j) / (height + width - 2) * 65535)
|
||||||
|
|
||||||
|
# Save as TIFF
|
||||||
|
tifffile.imwrite(output_path, test_data)
|
||||||
|
print(f"Created test 16-bit TIFF: {output_path}")
|
||||||
|
print(f" Shape: {test_data.shape}")
|
||||||
|
print(f" Dtype: {test_data.dtype}")
|
||||||
|
print(f" Min value: {test_data.min()}")
|
||||||
|
print(f" Max value: {test_data.max()}")
|
||||||
|
|
||||||
|
return output_path
|
||||||
|
|
||||||
|
|
||||||
|
def test_image_loading():
|
||||||
|
"""Test loading 16-bit TIFF with the Image class."""
|
||||||
|
print("\n=== Testing Image Loading ===")
|
||||||
|
|
||||||
|
# Create temporary test file
|
||||||
|
with tempfile.NamedTemporaryFile(suffix=".tif", delete=False) as tmp:
|
||||||
|
test_path = tmp.name
|
||||||
|
|
||||||
|
try:
|
||||||
|
# Create test image
|
||||||
|
create_test_16bit_tiff(test_path)
|
||||||
|
|
||||||
|
# Load with Image class
|
||||||
|
print("\nLoading with Image class...")
|
||||||
|
img = Image(test_path)
|
||||||
|
|
||||||
|
print(f"Successfully loaded image:")
|
||||||
|
print(f" Width: {img.width}")
|
||||||
|
print(f" Height: {img.height}")
|
||||||
|
print(f" Channels: {img.channels}")
|
||||||
|
print(f" Dtype: {img.dtype}")
|
||||||
|
print(f" Format: {img.format}")
|
||||||
|
|
||||||
|
# Test normalization
|
||||||
|
print("\nTesting normalization to float32 [0-1]...")
|
||||||
|
normalized = img.to_normalized_float32()
|
||||||
|
|
||||||
|
print(f"Normalized image:")
|
||||||
|
print(f" Shape: {normalized.shape}")
|
||||||
|
print(f" Dtype: {normalized.dtype}")
|
||||||
|
print(f" Min value: {normalized.min():.6f}")
|
||||||
|
print(f" Max value: {normalized.max():.6f}")
|
||||||
|
print(f" Mean value: {normalized.mean():.6f}")
|
||||||
|
|
||||||
|
# Verify normalization
|
||||||
|
assert normalized.dtype == np.float32, "Dtype should be float32"
|
||||||
|
assert (
|
||||||
|
0.0 <= normalized.min() <= normalized.max() <= 1.0
|
||||||
|
), "Values should be in [0, 1]"
|
||||||
|
|
||||||
|
print("\n✓ All tests passed!")
|
||||||
|
return True
|
||||||
|
|
||||||
|
except Exception as e:
|
||||||
|
print(f"\n✗ Test failed with error: {e}")
|
||||||
|
import traceback
|
||||||
|
|
||||||
|
traceback.print_exc()
|
||||||
|
return False
|
||||||
|
|
||||||
|
finally:
|
||||||
|
# Cleanup
|
||||||
|
if os.path.exists(test_path):
|
||||||
|
os.remove(test_path)
|
||||||
|
print(f"\nCleaned up test file: {test_path}")
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
success = test_image_loading()
|
||||||
|
sys.exit(0 if success else 1)
|
||||||
150
tests/test_yolo_16bit_float32.py
Normal file
150
tests/test_yolo_16bit_float32.py
Normal file
@@ -0,0 +1,150 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""
|
||||||
|
Test script for YOLO preprocessing of 16-bit TIFF images with float32 passthrough.
|
||||||
|
Verifies that no uint8 conversion occurs and data is preserved.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import numpy as np
|
||||||
|
import tifffile
|
||||||
|
from pathlib import Path
|
||||||
|
import tempfile
|
||||||
|
import sys
|
||||||
|
import os
|
||||||
|
|
||||||
|
# Add parent directory to path to import modules
|
||||||
|
sys.path.insert(0, str(Path(__file__).parent.parent))
|
||||||
|
|
||||||
|
from src.model.yolo_wrapper import YOLOWrapper
|
||||||
|
|
||||||
|
|
||||||
|
def create_test_16bit_tiff(output_path: str) -> str:
|
||||||
|
"""Create a test 16-bit grayscale TIFF file.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
output_path: Path where to save the test TIFF
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
Path to the created TIFF file
|
||||||
|
"""
|
||||||
|
# Create a 16-bit grayscale test image (200x200)
|
||||||
|
# With specific values to test precision preservation
|
||||||
|
height, width = 200, 200
|
||||||
|
|
||||||
|
# Create a gradient pattern with the full 16-bit range
|
||||||
|
test_data = np.zeros((height, width), dtype=np.uint16)
|
||||||
|
for i in range(height):
|
||||||
|
for j in range(width):
|
||||||
|
# Create a diagonal gradient using full 16-bit range
|
||||||
|
test_data[i, j] = int((i + j) / (height + width - 2) * 65535)
|
||||||
|
|
||||||
|
# Save as TIFF
|
||||||
|
tifffile.imwrite(output_path, test_data)
|
||||||
|
print(f"Created test 16-bit TIFF: {output_path}")
|
||||||
|
print(f" Shape: {test_data.shape}")
|
||||||
|
print(f" Dtype: {test_data.dtype}")
|
||||||
|
print(f" Min value: {test_data.min()}")
|
||||||
|
print(f" Max value: {test_data.max()}")
|
||||||
|
print(
|
||||||
|
f" Sample values: {test_data[50, 50]}, {test_data[100, 100]}, {test_data[150, 150]}"
|
||||||
|
)
|
||||||
|
|
||||||
|
return output_path
|
||||||
|
|
||||||
|
|
||||||
|
def test_float32_passthrough():
|
||||||
|
"""Test that 16-bit TIFF preprocessing passes float32 directly without uint8 conversion."""
|
||||||
|
print("\n=== Testing Float32 Passthrough (NO uint8) ===")
|
||||||
|
|
||||||
|
# Create temporary test file
|
||||||
|
with tempfile.NamedTemporaryFile(suffix=".tif", delete=False) as tmp:
|
||||||
|
test_path = tmp.name
|
||||||
|
|
||||||
|
try:
|
||||||
|
# Create test image
|
||||||
|
create_test_16bit_tiff(test_path)
|
||||||
|
|
||||||
|
# Create YOLOWrapper instance
|
||||||
|
print("\nTesting YOLOWrapper._prepare_source() for float32 passthrough...")
|
||||||
|
wrapper = YOLOWrapper()
|
||||||
|
|
||||||
|
# Call _prepare_source to preprocess the image
|
||||||
|
prepared_source, cleanup_path = wrapper._prepare_source(test_path)
|
||||||
|
|
||||||
|
print(f"\nPreprocessing result:")
|
||||||
|
print(f" Original path: {test_path}")
|
||||||
|
print(f" Prepared source type: {type(prepared_source)}")
|
||||||
|
|
||||||
|
# Verify it returns a numpy array (not a file path)
|
||||||
|
if isinstance(prepared_source, np.ndarray):
|
||||||
|
print(
|
||||||
|
f"\n✓ SUCCESS: Prepared source is a numpy array (float32 passthrough)"
|
||||||
|
)
|
||||||
|
print(f" Shape: {prepared_source.shape}")
|
||||||
|
print(f" Dtype: {prepared_source.dtype}")
|
||||||
|
print(f" Min value: {prepared_source.min():.6f}")
|
||||||
|
print(f" Max value: {prepared_source.max():.6f}")
|
||||||
|
print(f" Mean value: {prepared_source.mean():.6f}")
|
||||||
|
|
||||||
|
# Verify it's float32 in [0, 1] range
|
||||||
|
assert (
|
||||||
|
prepared_source.dtype == np.float32
|
||||||
|
), f"Expected float32, got {prepared_source.dtype}"
|
||||||
|
assert (
|
||||||
|
0.0 <= prepared_source.min() <= prepared_source.max() <= 1.0
|
||||||
|
), f"Expected values in [0, 1], got [{prepared_source.min()}, {prepared_source.max()}]"
|
||||||
|
|
||||||
|
# Verify it has 3 channels (RGB)
|
||||||
|
assert (
|
||||||
|
prepared_source.shape[2] == 3
|
||||||
|
), f"Expected 3 channels (RGB), got {prepared_source.shape[2]}"
|
||||||
|
|
||||||
|
# Verify no quantization to 256 levels (would happen with uint8 conversion)
|
||||||
|
unique_values = len(np.unique(prepared_source))
|
||||||
|
print(f" Unique values: {unique_values}")
|
||||||
|
|
||||||
|
# With float32, we should have much more than 256 unique values
|
||||||
|
if unique_values > 256:
|
||||||
|
print(f"\n✓ SUCCESS: Data has {unique_values} unique values (> 256)")
|
||||||
|
print(f" This confirms NO uint8 quantization occurred!")
|
||||||
|
else:
|
||||||
|
print(f"\n✗ WARNING: Data has only {unique_values} unique values")
|
||||||
|
print(f" This might indicate uint8 quantization happened")
|
||||||
|
|
||||||
|
# Sample some values to show precision
|
||||||
|
print(f"\n Sample normalized values:")
|
||||||
|
print(f" [50, 50]: {prepared_source[50, 50, 0]:.8f}")
|
||||||
|
print(f" [100, 100]: {prepared_source[100, 100, 0]:.8f}")
|
||||||
|
print(f" [150, 150]: {prepared_source[150, 150, 0]:.8f}")
|
||||||
|
|
||||||
|
# No cleanup needed since we returned array directly
|
||||||
|
assert (
|
||||||
|
cleanup_path is None
|
||||||
|
), "Cleanup path should be None for float32 pass through"
|
||||||
|
|
||||||
|
print("\n✓ All float32 passthrough tests passed!")
|
||||||
|
return True
|
||||||
|
|
||||||
|
else:
|
||||||
|
print(f"\n✗ FAILED: Prepared source is a file path: {prepared_source}")
|
||||||
|
print(f" This means data was saved to disk, not passed as float32 array")
|
||||||
|
if cleanup_path and os.path.exists(cleanup_path):
|
||||||
|
os.remove(cleanup_path)
|
||||||
|
return False
|
||||||
|
|
||||||
|
except Exception as e:
|
||||||
|
print(f"\n✗ Test failed with error: {e}")
|
||||||
|
import traceback
|
||||||
|
|
||||||
|
traceback.print_exc()
|
||||||
|
return False
|
||||||
|
|
||||||
|
finally:
|
||||||
|
# Cleanup
|
||||||
|
if os.path.exists(test_path):
|
||||||
|
os.remove(test_path)
|
||||||
|
print(f"\nCleaned up test file: {test_path}")
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
success = test_float32_passthrough()
|
||||||
|
sys.exit(0 if success else 1)
|
||||||
126
tests/test_yolo_16bit_preprocessing.py
Normal file
126
tests/test_yolo_16bit_preprocessing.py
Normal file
@@ -0,0 +1,126 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""
|
||||||
|
Test script for YOLO preprocessing of 16-bit TIFF images.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import numpy as np
|
||||||
|
import tifffile
|
||||||
|
from pathlib import Path
|
||||||
|
import tempfile
|
||||||
|
import sys
|
||||||
|
import os
|
||||||
|
|
||||||
|
# Add parent directory to path to import modules
|
||||||
|
sys.path.insert(0, str(Path(__file__).parent.parent))
|
||||||
|
|
||||||
|
from src.model.yolo_wrapper import YOLOWrapper
|
||||||
|
from src.utils.image import Image
|
||||||
|
from PIL import Image as PILImage
|
||||||
|
|
||||||
|
|
||||||
|
def create_test_16bit_tiff(output_path: str) -> str:
|
||||||
|
"""Create a test 16-bit grayscale TIFF file.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
output_path: Path where to save the test TIFF
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
Path to the created TIFF file
|
||||||
|
"""
|
||||||
|
# Create a 16-bit grayscale test image (200x200)
|
||||||
|
# With values ranging from 0 to 65535 (full 16-bit range)
|
||||||
|
height, width = 200, 200
|
||||||
|
|
||||||
|
# Create a gradient pattern
|
||||||
|
test_data = np.zeros((height, width), dtype=np.uint16)
|
||||||
|
for i in range(height):
|
||||||
|
for j in range(width):
|
||||||
|
# Create a diagonal gradient
|
||||||
|
test_data[i, j] = int((i + j) / (height + width - 2) * 65535)
|
||||||
|
|
||||||
|
# Save as TIFF
|
||||||
|
tifffile.imwrite(output_path, test_data)
|
||||||
|
print(f"Created test 16-bit TIFF: {output_path}")
|
||||||
|
print(f" Shape: {test_data.shape}")
|
||||||
|
print(f" Dtype: {test_data.dtype}")
|
||||||
|
print(f" Min value: {test_data.min()}")
|
||||||
|
print(f" Max value: {test_data.max()}")
|
||||||
|
|
||||||
|
return output_path
|
||||||
|
|
||||||
|
|
||||||
|
def test_yolo_preprocessing():
|
||||||
|
"""Test YOLO preprocessing of 16-bit TIFF images."""
|
||||||
|
print("\n=== Testing YOLO Preprocessing of 16-bit TIFF ===")
|
||||||
|
|
||||||
|
# Create temporary test file
|
||||||
|
with tempfile.NamedTemporaryFile(suffix=".tif", delete=False) as tmp:
|
||||||
|
test_path = tmp.name
|
||||||
|
|
||||||
|
try:
|
||||||
|
# Create test image
|
||||||
|
create_test_16bit_tiff(test_path)
|
||||||
|
|
||||||
|
# Create YOLOWrapper instance (no actual model loading needed for this test)
|
||||||
|
print("\nTesting YOLOWrapper._prepare_source()...")
|
||||||
|
wrapper = YOLOWrapper()
|
||||||
|
|
||||||
|
# Call _prepare_source to preprocess the image
|
||||||
|
prepared_path, cleanup_path = wrapper._prepare_source(test_path)
|
||||||
|
|
||||||
|
print(f"\nPreprocessing complete:")
|
||||||
|
print(f" Original path: {test_path}")
|
||||||
|
print(f" Prepared path: {prepared_path}")
|
||||||
|
print(f" Cleanup path: {cleanup_path}")
|
||||||
|
|
||||||
|
# Verify the prepared image exists
|
||||||
|
assert os.path.exists(prepared_path), "Prepared image should exist"
|
||||||
|
|
||||||
|
# Load the prepared image and verify it's uint8 RGB
|
||||||
|
prepared_img = PILImage.open(prepared_path)
|
||||||
|
print(f"\nPrepared image properties:")
|
||||||
|
print(f" Mode: {prepared_img.mode}")
|
||||||
|
print(f" Size: {prepared_img.size}")
|
||||||
|
print(f" Format: {prepared_img.format}")
|
||||||
|
|
||||||
|
# Convert to numpy to check values
|
||||||
|
img_array = np.array(prepared_img)
|
||||||
|
print(f" Shape: {img_array.shape}")
|
||||||
|
print(f" Dtype: {img_array.dtype}")
|
||||||
|
print(f" Min value: {img_array.min()}")
|
||||||
|
print(f" Max value: {img_array.max()}")
|
||||||
|
print(f" Mean value: {img_array.mean():.2f}")
|
||||||
|
|
||||||
|
# Verify it's RGB uint8
|
||||||
|
assert prepared_img.mode == "RGB", "Prepared image should be RGB"
|
||||||
|
assert img_array.dtype == np.uint8, "Prepared image should be uint8"
|
||||||
|
assert img_array.shape[2] == 3, "Prepared image should have 3 channels"
|
||||||
|
assert (
|
||||||
|
0 <= img_array.min() <= img_array.max() <= 255
|
||||||
|
), "Values should be in [0, 255]"
|
||||||
|
|
||||||
|
# Cleanup prepared file if needed
|
||||||
|
if cleanup_path and os.path.exists(cleanup_path):
|
||||||
|
os.remove(cleanup_path)
|
||||||
|
print(f"\nCleaned up prepared image: {cleanup_path}")
|
||||||
|
|
||||||
|
print("\n✓ All YOLO preprocessing tests passed!")
|
||||||
|
return True
|
||||||
|
|
||||||
|
except Exception as e:
|
||||||
|
print(f"\n✗ Test failed with error: {e}")
|
||||||
|
import traceback
|
||||||
|
|
||||||
|
traceback.print_exc()
|
||||||
|
return False
|
||||||
|
|
||||||
|
finally:
|
||||||
|
# Cleanup
|
||||||
|
if os.path.exists(test_path):
|
||||||
|
os.remove(test_path)
|
||||||
|
print(f"Cleaned up test file: {test_path}")
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
success = test_yolo_preprocessing()
|
||||||
|
sys.exit(0 if success else 1)
|
||||||
Reference in New Issue
Block a user