Hyper-SD:

    Trajectory Segmented Consistency Model for Efficient Image Synthesis

    ByteDance
    *  Project Lead

    Visual Comparison between Hyper-SD and Other Methods. From the first column to the fourth column, the prompts of these images are (1) A dog wearing a white t-shirt, with the word “hyper” written on it ... (2) Abstract beauty, approaching perfection, pure form, golden ratio, minimalistic, unfinished, ... (3) A crystal heart laying on moss in a serene zen garden ... (4)Anthropomorphic art of a scientist stag, victorian inspired clothing by krenz cushart ... , respectively.

    Real-Time Generation Demo of Hyper-SD.

    Abstract


    Recently, a series of diffusion-aware distillation algorithms have emerged to alleviate the computational overhead associated with the multi-step inference process of Diffusion Models (DMs). Current distillation techniques often dichotomize into two distinct aspects: i) ODE Trajectory Preservation; and ii) ODE Trajectory Reformulation. However, these approaches suffer from severe performance degradation or domain shifts. To address these limitations, we propose Hyper-SD, a novel framework that synergistically amalgamates the advantages of ODE Trajectory Preservation and Reformulation, while maintaining near-lossless performance during step compression. Firstly, we introduce Trajectory Segmented Consistency Distillation to progressively perform consistent distillation within pre-defined time-step segments, which facilitates the preservation of the original ODE trajectory from a higher-order perspective. Secondly, we incorporate human feedback learning to boost the performance of the model in a low-step regime and mitigate the performance loss incurred by the distillation process. Thirdly, we integrate score distillation to further improve the low-step generation capability of the model and offer the first attempt to leverage a unified LoRA to support the inference process at all steps. Extensive experiments and user studies demonstrate that Hyper-SD achieves SOTA performance from 1 to 8 inference steps for both SDXL and SD1.5. For example, Hyper-SDXL surpasses SDXL-Lightning by +0.68 in CLIP Score and +0.51 in Aes Score in the 1-step inference.

    Pipeline


    Hyper-SD take the two-stage Progressive Consistency Distillation. The first stage involves consistency distillation in two separate time segments: [0, T/2] and [T/2 , T] to obtain the two segments consistency ODE. Then, this ODE trajectory is adopted to train a global consistency model in the subsequent stage

    Experiment

    Qualitative comparisons between Hyper-SD and other LoRA-based acceleration approaches on SDXL architecture.

    Qualitative comparisons between Hyper-SD and other LoRA-based acceleration approaches on SD15 architecture.

    Hyper-SD exhibits a remarkable superiority over existing methods that concentrate on acceleration and obtain more user preference on both SD1.5 and SDXL architectures.

    Hyper-SD LoRAs with different steps can be applied to different base models and consistently generate high-quality images

    The unified LoRAs of Hyper-SD are compatible with ControlNet. The examples are conditioned on either scribble or canny images.

    BibTeX

    @misc{ren2024hypersd,
          title={Hyper-SD: Trajectory Segmented Consistency Model for Efficient Image Synthesis}, 
          author={Yuxi Ren and Xin Xia and Yanzuo Lu and Jiacheng Zhang and Jie Wu and Pan Xie and Xing Wang and Xuefeng Xiao},
          year={2024},
          eprint={2404.13686},
          archivePrefix={arXiv},
          primaryClass={cs.CV}
    }
    主站蜘蛛池模板: 色精品一区二区三区| 色系一区二区三区四区五区| 国产成人久久一区二区不卡三区| 国产精品成人一区二区| 国产成人精品一区二区A片带套 | 一区二区精品视频| 亚洲欧美国产国产一区二区三区| 日韩国产精品无码一区二区三区| 国产福利一区二区在线视频| 国产精品视频无圣光一区| 亚洲熟妇无码一区二区三区 | 精品视频无码一区二区三区 | 色一情一乱一区二区三区啪啪高| 精品少妇一区二区三区视频| 在线精品国产一区二区三区| 亚洲国产综合精品一区在线播放| 免费国产在线精品一区| 国产传媒一区二区三区呀| 国产亚洲综合精品一区二区三区 | 亚洲国产综合精品中文第一区| 国产av福利一区二区三巨 | 国产精品一区二区不卡| 一区二区在线观看视频| 91午夜精品亚洲一区二区三区| 一区二区三区日韩| 亚洲日韩一区二区三区| 福利国产微拍广场一区视频在线 | 无码国产精品一区二区免费式影视| 人体内射精一区二区三区| 日韩精品一区二区三区色欲AV| 亚洲制服丝袜一区二区三区| 亚洲高清一区二区三区电影| 冲田杏梨AV一区二区三区| 成人区人妻精品一区二区不卡| 精品国产一区二区三区av片| 精品国产一区二区三区AV性色| 中文字幕一区在线| 麻豆亚洲av熟女国产一区二| 亚洲欧美日韩中文字幕在线一区 | 欧美人妻一区黄a片| 免费一区二区三区在线视频|