AWPortrait-FL 是在 FLUX.1-dev 上使用 AWPortrait-XL 的训练集和近 2000 张具有极高美学品质的时尚摄影照片进行微调的。它在构图和细节方面有显著改进，皮肤和纹理更加细腻和逼真。由DynamicWang在AWPlanet训练。
先上代码
import torch
from diffusers import FluxPipeline
pipe = FluxPipeline.from_pretrained("./black-forest-labs/FLUX.1-dev", torch_dtype=torch.bfloat16)
pipe.load_lora_weights('./Shakker-Labs/AWPortrait-FL', weight_name='AWPortrait-FL-lora.safetensors')
pipe.fuse_lora(lora_scale=0.9)
pipe.to("cuda")
prompt = "close up portrait, Amidst the interplay of light and shadows in a photography studio,a soft spotlight traces the contours of a face,highlighting a figure clad in a sleek black turtleneck. The garment,hugging the skin with subtle luxury,complements the Caucasian model's understated makeup,embodying minimalist elegance. Behind,a pale gray backdrop extends,its fine texture shimmering subtly in the dim light,artfully balancing the composition and focusing attention on the subject. In a palette of black,gray,and skin tones,simplicity intertwines with profundity,as every detail whispers untold stories."
image = pipe(prompt,
num_inference_steps=24,
guidance_scale=3.5,
width=768, height=1024,
).images[0]
image.save(f"example.png")
注意，这里和HF上官方的样例有所区别
- 单独下载了模型到本地
- 修复了一个weight_name错误的小bug
详细步骤
1. 创建Conda环境及依赖安装
conda create -n portrait python=3.10 -y
conda activate portrait
conda install pytorch==2.0.1 torchvision==0.15.2 torchaudio==2.0.2 pytorch-cuda=11.7 -c pytorch -c nvidia
conda install -c conda-forge transformers huggingface_hub
conda install chardet
pip3 install diffusers accelerate sentencepiece protobuf
pip3 install -U PEFT
2. 下载模型文件及lora到本地
参考教程下载模型文件到本地，总共约占用120GB空间
3. 生成图片
运行脚本生成图片即可。大概20秒一张图，测试环境是
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 515.105.01 Driver Version: 515.105.01 CUDA Version: 11.7 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 NVIDIA A800-SXM... On | 00000000:5B:00.0 Off | 0 |
| N/A 58C P0 407W / 400W | 36964MiB / 81920MiB | 99% Default |
| | | Disabled |
+-------------------------------+----------------------+----------------------+
- Mac上mps模式无法使用，cpu生成的时候极其缓慢并且风扇工作声音特别大
