rajlab/condensatepose-model

CondensatePose

CondensatePose is a deep learning model for detecting and segmenting biomolecular condensates in fluorescence microscopy images.

Model Description

This model uses EfficientNetV2 as the encoder with a sophisticated Feature Pyramid Network decoder that includes style-based feature modulation and dual residual blocks for accurate condensate detection.

Architecture Highlights

• Encoder: EfficientNetV2-RW-S (pretrained, adapted for grayscale microscopy)
• Decoder: Multi-scale Feature Pyramid Network with:
  • Style-based feature modulation for adaptive processing
  • Dual residual blocks for deep feature fusion
  • Multi-scale upsampling refinement
  • Spatial attention for focusing on sparse objects
• Output: Binary mask logits + flow field vectors
• Parameters: ~18M trainable parameters

Key Features

• Style Modulation: Adaptive feature processing based on global image context
• Multi-Scale Processing: Captures condensates at various scales
• Spatial Attention: Focuses on sparse condensate regions
• Flow Fields: Cellpose-style flow vectors for instance segmentation
• Single-Channel: Works on grayscale fluorescence images
• Variable Size: Handles different image dimensions

Training Data

Trained on annotated fluorescence microscopy images from the Nimbus Image platform, containing manually annotated biomolecular condensates with diverse:

• Morphologies (spherical to irregular)
• Sizes (10-500 pixels in diameter)
• Intensities (bright to dim)
• Densities (sparse to crowded fields)

Intended Use

Primary Applications

• Detecting biomolecular condensates in live-cell imaging
• Segmenting phase-separated compartments
• Tracking condensate dynamics over time
• Quantifying condensate properties (size, number, morphology, intensity)

How to Use

Installation

pip install torch timm huggingface-hub

Quick Start

from huggingface_hub import hf_hub_download
import torch
import numpy as np
from model import load_condensatepose_model

# Download and load model
model_path = hf_hub_download(
    repo_id="rajlab/condensatepose-model",
    filename="model_weights.pth"
)

model = load_condensatepose_model(model_path, device='cuda')
model.eval()

# Prepare your image (H, W) grayscale
image = ...  # Your microscopy image as numpy array

# Normalize
image = image.astype(np.float32)
if image.max() > 1:
    image = image / 255.0

# Convert to tensor
img_tensor = torch.from_numpy(image).unsqueeze(0).unsqueeze(0)  # (1, 1, H, W)
img_tensor = img_tensor.to('cuda')

# Run inference
with torch.no_grad():
    outputs = model(img_tensor)

# Get outputs
mask_logits = outputs['mask'][0, 0].cpu().numpy()  # (H, W)
flow_vectors = outputs['flows'][0].cpu().numpy()   # (2, H, W)

# Apply sigmoid to get probabilities
mask_prob = 1 / (1 + np.exp(-mask_logits))

# Threshold to get binary mask
binary_mask = (mask_prob > 0.5).astype(np.uint8)

# Generate instance segmentation
from scipy.ndimage import label
instance_mask, num_objects = label(binary_mask)

print(f"Detected {num_objects} condensates")

Model Configuration

{
  "model_type": "condensatepose",
  "architecture": "efficientnetv2-fpn-style-modulation",
  "task": "instance-segmentation",
  "application": "biomolecular-condensate-detection",
  "framework": "pytorch",
  "encoder_variant": "rw_s",
  "encoder_backbone": "efficientnetv2",
  "pyramid_channels": [
    24,
    48,
    64,
    160
  ],
  "pyramid_dim": 32,
  "use_spatial_attention": true,
  "spatial_kernel_size": 11,
  "dropout_rate": 0.15,
  "input_channels": 1,
  "output_channels": 3,
  "patch_size": 256,
  "trained_on": "nimbus_image_data",
  "microscopy_type": "fluorescence",
  "target": "biomolecular_condensates",
  "style_modulation": true,
  "dual_residual_blocks": true,
  "multi_scale_upsampling": true
}