Hyper-SD:

    Trajectory Segmented Consistency Model for Efficient Image Synthesis

    ByteDance
    *  Project Lead

    Visual Comparison between Hyper-SD and Other Methods. From the first column to the fourth column, the prompts of these images are (1) A dog wearing a white t-shirt, with the word “hyper” written on it ... (2) Abstract beauty, approaching perfection, pure form, golden ratio, minimalistic, unfinished, ... (3) A crystal heart laying on moss in a serene zen garden ... (4)Anthropomorphic art of a scientist stag, victorian inspired clothing by krenz cushart ... , respectively.

    Real-Time Generation Demo of Hyper-SD.

    Abstract


    Recently, a series of diffusion-aware distillation algorithms have emerged to alleviate the computational overhead associated with the multi-step inference process of Diffusion Models (DMs). Current distillation techniques often dichotomize into two distinct aspects: i) ODE Trajectory Preservation; and ii) ODE Trajectory Reformulation. However, these approaches suffer from severe performance degradation or domain shifts. To address these limitations, we propose Hyper-SD, a novel framework that synergistically amalgamates the advantages of ODE Trajectory Preservation and Reformulation, while maintaining near-lossless performance during step compression. Firstly, we introduce Trajectory Segmented Consistency Distillation to progressively perform consistent distillation within pre-defined time-step segments, which facilitates the preservation of the original ODE trajectory from a higher-order perspective. Secondly, we incorporate human feedback learning to boost the performance of the model in a low-step regime and mitigate the performance loss incurred by the distillation process. Thirdly, we integrate score distillation to further improve the low-step generation capability of the model and offer the first attempt to leverage a unified LoRA to support the inference process at all steps. Extensive experiments and user studies demonstrate that Hyper-SD achieves SOTA performance from 1 to 8 inference steps for both SDXL and SD1.5. For example, Hyper-SDXL surpasses SDXL-Lightning by +0.68 in CLIP Score and +0.51 in Aes Score in the 1-step inference.

    Pipeline


    Hyper-SD take the two-stage Progressive Consistency Distillation. The first stage involves consistency distillation in two separate time segments: [0, T/2] and [T/2 , T] to obtain the two segments consistency ODE. Then, this ODE trajectory is adopted to train a global consistency model in the subsequent stage

    Experiment

    Qualitative comparisons between Hyper-SD and other LoRA-based acceleration approaches on SDXL architecture.

    Qualitative comparisons between Hyper-SD and other LoRA-based acceleration approaches on SD15 architecture.

    Hyper-SD exhibits a remarkable superiority over existing methods that concentrate on acceleration and obtain more user preference on both SD1.5 and SDXL architectures.

    Hyper-SD LoRAs with different steps can be applied to different base models and consistently generate high-quality images

    The unified LoRAs of Hyper-SD are compatible with ControlNet. The examples are conditioned on either scribble or canny images.

    BibTeX

    @misc{ren2024hypersd,
          title={Hyper-SD: Trajectory Segmented Consistency Model for Efficient Image Synthesis}, 
          author={Yuxi Ren and Xin Xia and Yanzuo Lu and Jiacheng Zhang and Jie Wu and Pan Xie and Xing Wang and Xuefeng Xiao},
          year={2024},
          eprint={2404.13686},
          archivePrefix={arXiv},
          primaryClass={cs.CV}
    }
    主站蜘蛛池模板: 精品免费AV一区二区三区| 久久影院亚洲一区| 成人精品视频一区二区| 久久毛片一区二区| 2018高清国产一区二区三区| 国产日本亚洲一区二区三区| 亚洲色精品三区二区一区| 日韩免费无码一区二区视频| 国产成人久久一区二区三区| 中文字幕一区精品| 精品无码人妻一区二区三区品| 亚洲变态另类一区二区三区| 国产一区在线播放| 久久久久一区二区三区| 亚洲熟女综合色一区二区三区| 日本精品视频一区二区三区| 区三区激情福利综合中文字幕在线一区亚洲视频1 | 中文字幕一区二区三区在线不卡| 亚洲一区二区三区无码中文字幕| 亚洲国产福利精品一区二区| 精品在线一区二区| 亚洲一区二区三区免费视频| 制服美女视频一区| 在线视频一区二区| 国产精品一区12p| 亚洲熟妇成人精品一区| 国产精品一区三区| 亚洲a∨无码一区二区| 亚洲av高清在线观看一区二区| 亚洲一区二区三区高清视频| 日韩一区二区三区免费播放| 中文字幕一区二区精品区| 一区五十路在线中出| 国产美女在线一区二区三区| 自慰无码一区二区三区| 国产熟女一区二区三区五月婷| 日韩精品一区二区亚洲AV观看| 日韩视频在线观看一区二区| 一区二区三区在线观看| 国产午夜精品免费一区二区三区 | 91麻豆精品国产自产在线观看一区 |