This is an announcement of this new model: https://huggingface.co/crumb/bloom-560m-RLHF-SD2-prompter-aesthetic. Let’s ask ChatGPT to give a quick summary.
Can you give me a description of this code?
This code appears to be a reinforcement learning loop that trains a transformer model to generate text and then uses that text to generate an image with a stable diffusion pipeline. The generated text and images are then displayed and the user is asked to provide feedback on the quality of the generated content. The feedback is then used to fine-tune the transformer model.
For the first iteration of the model, I’m using a BLOOM-560 model fine-tuned to generate Stable Diffusion (v1) prompts, I used RLHF to, over the course of one night, hunched over Google Colab, fine-tune a model to make high quality images from the base Stable Diffusion V2 model. Now you can use it for free to augment your prompting workflow @ https://huggingface.co/crumb/bloom-560m-RLHF-SD2-prompter
I would process 16 images at a time, scoring each on a scale of 0–1 (0 being worst, 1 being best) and updating with a learning rate of 0.001(very high). I didn’t come to that scale immediately though and never wanted to stick to one scale, so I had to make myself a graphic.
I made use of bitsandbytes Adam8bit optimizer as well as only fine-tuning the Biases and LayerNorm weights of the BLOOM model, because of speed and memory constraints.
The result is a good prompt extender that can make images a little better! I have another addition.
For the second iteration of the model I took aesthetic models from https://github.com/crowsonkb/simulacra-aesthetic-models to automatically score images, set an arbitrary threshold of 6.3 which will score an image as 0.25, and a threshold of 7. which will score an image as 1. Now I don’t have to hand-score all these images! It was super simple to add this replacement and let it just run until Google Colab crashes.
You can use this model as well https://huggingface.co/crumb/bloom-560m-RLHF-SD2-prompter-aesthetic
