ModelsLab/ControlNetXL (CNXL) - Huchenlei-PuLID

PuLID is an ip-adapter alike method to restore facial identity. It uses both insightface embedding and CLIP embedding similar to what ip-adapter faceid plus model does. However, there is an extra process of masking out the face from background environment using facexlib before passing image to CLIP. PuLID also uses Eva CLIP instead of normal CLIP. In the attn overrides, PuLID also does something more than IPAdapter, as it zero pads the tensor and adds to ortho of hidden states.

PuLID uses both evaclip embedding and insightface embedding as proj module input. So in order to use the feature, you need to install https://github.com/huchenlei/sd-webui-controlnet-evaclip extension.

ComfyUI implementations:

• https://github.com/cubiq/PuLID_ComfyUI provided by cubiq, native ComfyUI implementation

• https://github.com/ZHO-ZHO-ZHO/ComfyUI-PuLID-ZHO provided by ZHO, diffusers-based implementation

Original repository: https://huggingface.co/huchenlei/ipadapter_pulid