MearcAll papers

arXiv 2112.10752 · cs.CV

High-Resolution Image Synthesis with Latent Diffusion Models

Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, Björn Ommer

Claims were extracted automatically by reducer-1 on 2026-11-12. The paper's license does not allow republishing it, so long claims are shortened to a quotation. Read the full text on arXiv.

No reproduction recordedclm_1e417801
By decomposing the image formation process into a sequential application of denoising autoencoders, diffusion models (DMs) achieve state-of-the-art synthesis results on image data and beyond.

Abstract · page 1 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedclm_df997f3c
Our latent diffusion models (LDMs) achieve new state-of-the-art scores for image inpainting and class-conditional image synthesis and highly competitive performance on various tasks, including text-to-image synthesis, unconditional image generation and … (excerpt)

Abstract · page 1 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedclm_34d6f5b4
We propose latent diffusion models (LDMs) as an effective generative model and a separate mild compression stage that only eliminates imperceptible details.

Introduction · page 2 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_fb12bdc9
(iii) We show that, in contrast to previous work [93] which learns both an encoder/decoder architecture and a score-based prior simultaneously, our approach does not require a delicate weighting of … (excerpt)

Introduction · page 2 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_ec51a755
Revisiting the analysis above (Fig. 1 and 2) we attribute this to i) leaving most of perceptual compression to the diffusion model and ii) too strong first stage compression resulting … (excerpt)

On Perceptual Compression Tradeoffs · page 5 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_e0d76d5a
LDM-{4-8} outperform models with unsuitable ratios of perceptual and conceptual compression.

On Perceptual Compression Tradeoffs · page 5 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_6c3d37c7
Especially compared to pixel-based LDM-1, they achieve much lower FID scores while simultaneously significantly increasing sample throughput.

On Perceptual Compression Tradeoffs · page 5 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_ea426177
In summary, LDM-4 and -8 offer the best conditions for achieving high-quality synthesis results.

On Perceptual Compression Tradeoffs · page 5 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_c92e6a2f
On CelebA-HQ, we report a new state-of-the-art FID of 5.11, outperforming previous likelihood-based models as well as GANs.

Image Generation with Latent Diffusion · page 5 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_10338e2f
We also outperform LSGM [93] where a latent diffusion model is trained jointly together with the first stage.

Image Generation with Latent Diffusion · page 5 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_678df2ac
We outperform prior diffusion based approaches on all but the LSUN-Bedrooms dataset, where our score is close to ADM [15], despite utilizing half its parameters and requiring 4-times less train … (excerpt)

Image Generation with Latent Diffusion · page 6 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_62fea507
Evaluation of text-conditional image synthesis on the 256× 256-sized MS-COCO [51] dataset: with 250 DDIM [84] steps our model is on par with the most recent diffusion [59] and autoregressive … (excerpt)

Image Generation with Latent Diffusion · page 6 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_7678a86b
Here we outperform the state of the art diffusion model ADM [15] while significantly reducing computational requirements and parameter count, cf .

Transformer Encoders for LDMs · page 7 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_e9f222a2
Our qualitative and quantitative results (see Fig. 10 and Tab. 5) show competitive performance and LDM-SR outperforms SR3 in FID while SR3 has a better IS.

Super-Resolution with Latent Diffusion · page 8 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_b1780ee7
Overall, we observe a speed-up of at least 2.7× between pixel- and latent-based diffusion models while improving FID scores by a factor of at least 1.6×.

Inpainting with Latent Diffusion · page 8 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_cafd047d
The comparison with other inpainting approaches in Tab. 7 shows that our model with attention improves the overall image quality as measured by FID over that of [88].

Inpainting with Latent Diffusion · page 8 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedclm_d18ca28e
We have presented latent diffusion models, a simple and efficient way to significantly improve both the training and sampling efficiency of denoising diffusion models without degrading their quality.

Conclusion · page 9 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedclm_c3723637
Based on this and our crossattention conditioning mechanism, our experiments could demonstrate favorable results compared to state-of-the-art methods across a wide range of conditional image synthesis tasks without task-specific architectures.

Conclusion · page 9 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_bd9cb664
Our COCO model reaches the performance of recent state-of-the art models in layout-toimage synthesis, when following their training and evaluation protocol [89].

Layout-to-Image Synthesis · page 20 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_79417bd2
Our OpenImages model surpasses the results of Jahn et al [37] by a margin of nearly 11 in terms of FID.

Layout-to-Image Synthesis · page 20 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_ba93be6e
LDM-8 requires significantly fewer parameters and compute requirements (see Tab. 18) to achieve very competitive performance.

Class-Conditional Image Synthesis on ImageNet · page 20 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_0f15ad9d
The results of this comparison are shown in the last two rows of Tab. 11 and demonstrate that LDM achieves better performance while allowing for significantly faster sampling.

Super-Resolution · page 23 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedclm_be587db3
Compute during training in V100-days, numbers of competing methods taken from [15] unless stated differently; ∗: Throughput measured in samples/sec on a single NVIDIA A100;†: Numbers taken from [15] ;‡: … (excerpt)

Computational Requirements · page 28 · Automated extraction, not reviewed · 0 reviewed reproductions

Reproduced one of these claims?

Record your result in the Mearc app on your own machine, then use Publish this result with this paper's id (2112.10752) and the claim id shown next to the claim. Mearc creates a file that holds only what you choose to publish. Add it to the public claim data by pull request; a maintainer other than you reviews it before it counts.