Murdock Aubry
commited on
Update README.md
Browse files
README.md
CHANGED
|
@@ -1,3 +1,53 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
license: mit
|
| 3 |
language:
|
|
|
|
| 1 |
+
This model is part of the research work described in "FeatureFusion: Merging Diffusion Models Through Representation Correlations" by Murdock Aubry and James Bona-Landry.
|
| 2 |
+
|
| 3 |
+
<h1>
|
| 4 |
+
Model Description
|
| 5 |
+
</h1>
|
| 6 |
+
|
| 7 |
+
<h2>Overview</h2>
|
| 8 |
+
This model is an vehicles specialist based on the Stable Diffusion 1.4 architecture.
|
| 9 |
+
|
| 10 |
+
<br>
|
| 11 |
+
<h2>Model Details</h2>
|
| 12 |
+
|
| 13 |
+
Base Model: CompVis/stable-diffusion-v1-4
|
| 14 |
+
Type: Specialist
|
| 15 |
+
Specialization: Vehicles
|
| 16 |
+
Training Data: Vehicles shard
|
| 17 |
+
Model Architecture: UNet-based diffusion model
|
| 18 |
+
|
| 19 |
+
<h2>Limitations</h2>
|
| 20 |
+
|
| 21 |
+
The model has the same limitations as the base Stable Diffusion model
|
| 22 |
+
Best performance is achieved when prompts relate to the model's specialization
|
| 23 |
+
May produce unexpected results for concepts outside its training distribution
|
| 24 |
+
|
| 25 |
+
|
| 26 |
+
<h1>Training</h1>
|
| 27 |
+
|
| 28 |
+
<h2>Training Procedure</h2>
|
| 29 |
+
|
| 30 |
+
Training Data: Pick-a-Pic v1
|
| 31 |
+
Training Method: Finetuning of the UNet component while keeping text encoder and VAE frozen
|
| 32 |
+
|
| 33 |
+
<h2>Hyperparameters:</h2>
|
| 34 |
+
|
| 35 |
+
Optimizer: AdamW
|
| 36 |
+
Learning rate: 1e-6
|
| 37 |
+
Schedule: Cosine with warmup
|
| 38 |
+
Training steps: 5 epochs on 1000 data samples
|
| 39 |
+
Memory optimization: Gradient accumulation (4 steps), attention slicing, VAE slicing, gradient checkpointing
|
| 40 |
+
|
| 41 |
+
<h1>Citation</h1>
|
| 42 |
+
|
| 43 |
+
If you use this model in your research, please cite:<br>
|
| 44 |
+
@article{aubry2024featurefusion,<br>
|
| 45 |
+
title={FeatureFusion: Merging Diffusion Models Through Representation Correlations},<br>
|
| 46 |
+
author={Aubry, Murdock and Bona-Landry, James},<br>
|
| 47 |
+
journal={},<br>
|
| 48 |
+
year={2025}<br>
|
| 49 |
+
}
|
| 50 |
+
|
| 51 |
---
|
| 52 |
license: mit
|
| 53 |
language:
|