基于多模态大模型的透明印花素材生成方法

Transparent Printing Material Generation Method Based on Large Multimodal Models

摘要:

针对现有的图像生成模型无法满足纺织领域对生成多样透明印花素材专业要求的问题，提出了一种基于多模态大模型的透明印花素材生成方法；采用美学评分预测器构建高质量印花素材数据集，并使用多模态大语言模型BLIP3进行数据集的标签语义生成；通过多尺度分桶训练的方式微调SD模型，并改进VAE模型将图像透明信息引入到图像生成空间中，使得能够直接生成高质量的透明印花素材。实验结果表明，所设计方法在文生图、图生图两种模式下都能生成内容和风格多样的透明印花素材，并且生成素材的边缘细节明显好于深度学习图像分割模型的结果。

Abstract:

Aiming at the problem that existing image generation models cannot meet the professional requirements for generating diverse transparent printing materials in the textile field, a transparent printing material generation method based on large multi-modal models is proposed. First, an aesthetic score predictor is used to construct high-quality printing material data set. Second, the large multi-modal language model BLIP3 is employed to generate label semantics of the data set. Third, the SD model is fine-tuned through multi-scale bucket training, and VAE is improved to introduce image transparency information into the image generation space, and enable the direct generation of high quality transparent printing material. Experimental results show that the designed method can generate transparent printed materials with diverse contents and styles in both text-to-image and image-to-image modes, and the edge details of the generated materials are significantly better than those of the deep learning image segmentation model.