CharGen
Fast and Fluent Portrait Modification
CharGen — Fast and Fluent Portrait Modification
Overview
Interactive editing of character images with diffusion models remains challenging due to the inherent trade-off between fine-grained control, generation speed, and visual fidelity. We introduce CharGen, a character-focused editor that combines attribute-specific Concept Sliders, trained to isolate and manipulate attributes such as facial feature size, expression, and decoration with the StreamDiffusion sampling pipeline for more interactive performance. To counteract the loss of detail that often accompanies accelerated sampling, we propose a lightweight Repair Step that reinstates fine textures without compromising structural consistency.
Method
CharGen combines attribute-specific Concept Sliders with StreamDiffusion's real-time pipeline to enable interactive character editing. The system addresses three key challenges: (1) fine-grained attribute control through pretrained Concept Sliders, (2) interactive inference speed via StreamDiffusion integration, and (3) detail restoration through a lightweight Repair Step.
Results
CharGen achieves two-to-fourfold faster edit turnaround compared to standard diffusion methods while maintaining precise attribute control through pretrained Concept Sliders. The lightweight Repair Step effectively restores high-frequency details lost during accelerated sampling, with the Repair Slider approach providing the most consistent detail enhancement.
Single Attribute Modifications — CharGen provides precise control for localized adjustments like makeup and facial features, though it shows limitations with strong transformations such as aging effects. Unlike existing methods that provide discrete editing steps, CharGen is the only approach enabling truly continuous control through progressive slider adjustments.
Multi-Attribute Editing — CharGen's LoRA merging approach enables simultaneous modification of multiple attributes while maintaining consistency. Our method's interactive nature allows interactive adjustment of multiple sliders, providing immediate visual feedback for complex editing scenarios.
Progressive Editing — CharGen maintains the original input throughout the editing process by only adjusting slider parameters for progressive modifications. This approach eliminates quality degradation while enabling fluent attribute transitions.
Refinement Comparison — StreamDiffusion's acceleration introduces detail loss that we address through a lightweight Repair Step. We explore three approaches: standard Stable Diffusion, Repair Slider integration, and ControlNet-based repair. Our Repair Slider approach provides the best balance of detail enhancement and structural preservation, successfully restoring high-frequency details without compromising the original image structure.
Citation
More Information
Open Positions
Interested in persuing a PhD in computer graphics?
Never miss an update
Join us on Twitter / X for the latest updates of our research group and more.
Acknowledgements
Funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) under Germany's Excellence Strategy – EXC number 2064/1 – Project number 390727645. This work was supported by the German Research Foundation (DFG): SFB 1233, Robust Vision: Inference Principles and Neural Mechanisms, TP 02, project number: 276693517. This work was supported by the Tübingen AI Center. The authors thank the International Max Planck Research School for Intelligent Systems (IMPRS-IS) for supporting Jan-Niklas Dihlmann.