×
Submission rules
This leaderboard tracks tiny text-to-image models whose denoising network was trained from scratch . Every submission is reviewed by a maintainer before it appears on the board.
The one rule that matters: your denoising network must start from random weights . Beyond that, you can use any architecture, any dataset, and even learn from bigger models (distillation) as long as you're not starting from their weights.
Allowed
Pre-trained VAEs (e.g. Stability-AI VAE, sd-vae-ft-mse). Training your own isn't the point of this benchmark.Pre-trained text encoders and tokenizers (e.g. CLIP, T5, Gemma). They're frozen feature extractors.Any architecture : DiT, U-ViT, UNet, MDT, anything. Novelty is encouraged.Any training data , including images generated by other models.Distillation. You can train your own randomly initialized model to imitate a bigger teacher (SDXL, Flux, etc.) using its outputs or training signals. Just name the teacher in your submission. The teacher doesn't count toward the parameter cap.Any weights format. .safetensors, .pt, .ckpt, sharded files, whatever works. A single model.safetensors is recommended (see below).
Not allowed
Fine-tuning. No starting from a pre-trained diffusion model (SDXL, SD 1.5/2.1, Flux, PixArt, etc.) and continuing training. This includes pruned or partially copied versions of one.LoRA / adapters. No LoRA, DoRA, adapter layers or side-tuning on a frozen pre-trained diffusion model.Re-uploading existing models. Converting someone else's weights to another format and submitting them doesn't count as your own work.Merged models. No weight merges of existing checkpoints.Pre-trained diffusion backbones. If the denoiser was pre-trained on any image generation task, it's ineligible, even if you continued it on a new dataset.Pre-trained image encoders inside your model. Beyond the frozen VAE, your backbone can't depend on another image encoder's latent space.
How to evaluate (mandatory)
Everyone is scored the same way so results are comparable:
Dataset: MS-COCO 2014 validation, first 30,000 captions of the official COCO-30K list. Both FID and CLIP use this set.FID reference images: the 30K matching real images, center-cropped to the evaluation resolution.Generation: one image per caption, prompt used as-is, fixed seed 0. Report your sampler, number of steps and guidance scale.Resolution: 256×256 for the main score. If your model's native size is smaller, upsample or report the native size. Different resolutions aren't directly comparable.FID: torchmetrics.image.fid (preferred) or clean-fid, Inception-v3 pool features (2048-d).CLIP Score: torchmetrics.image.clip_score with openai/clip-vit-base-patch32, reported on a 0–1 scale (torchmetrics output ÷ 100).Gallery images need prompts. Every image must include the exact prompt it was generated from.Other datasets or resolutions are welcome as extra results but don't count toward the main leaderboard.
Technical requirements
Parameter cap: ≤ 1.5B total for the model you submit. Count everything : text encoder + VAE + denoiser + any other learned parameters. A distillation teacher is not part of your model, so it isn't counted.Weights format (recommended, not required): one file named model.safetensors. It's safe to load (no code execution) and lets us verify parameter counts quickly. If you use another format, include the model code or config so we can load it and check the count.Resolution: report the native generation resolution used for evaluation.
How we verify
For .safetensors files we read the metadata for parameter count and key names. For other formats we use your config or code.
We look for signs of fine-tuning, like key names or weights that match known public checkpoints.
We may ask for training logs, configs, or a short training summary. For distilled models, we'll ask which teacher you used.
Submissions that break the rules are rejected with an explanation. Repeat offenders are banned.
FAQ
Can I distill SDXL or Flux into my own model? Yes, as long as your model starts from random weights and you name the teacher.What's the difference between distillation and fine-tuning? In distillation you train a fresh, randomly initialized model to imitate a teacher. In fine-tuning you start from the teacher's (or any pre-trained model's) weights. The first is allowed, the second isn't.Do I have to use safetensors? No, but we recommend it. Any format works as long as we can verify the model.Can I use a pre-trained VAE? Yes, it's encouraged.My model uses the SDXL architecture but was trained from scratch. Is that allowed? Yes, if every weight came from random initialization. Weights that are identical to a public checkpoint will be rejected.