Things I’ve made this year. I’m writing this so I can have my own catalog of important code that I’d want to come back to. To either revise or to Frankenstein into new projects. I’m going to do this every year around this time to hopefully document my journey deeper into the ML rabbit hole.
Doohickey, A diffusers-based wrapper for stable diffusion, one of (if not the? someone fact check) the first public notebooks to incorporate CLIP guidance, loading of any model from the Huggingface Hub (diffusers or CompVis style) and more.
RLHF Tuned Prompt Generator https://twitter.com/aicrumb/status/1598248104981319681
Quick Inversion, approximate textual inversion in seconds instead of hours.
Experiments in Adversarial Noise for corrupting an image upon encoding of the VQGAN used in Stable Diffusion. (In response to a tweet about an “anti-ai” filter) https://colab.research.google.com/github/aicrumb/doohickey/blob/main/Adversarial_Attacks_to_AI_Art_Tools.ipynb
Experiments with pre-training transformers on algorithmically generated texts from scratch. (note: on a laptop 3060, couldn’t make it much bigger in a reasonable time without a big machine)
Meta-Opt of RGB Tensors for Txt2Img
Whatchamacallit is a small simple alternative wrapper for simplified use of Stable Diffusion with diffusers pipelines.
Genetic Algorithm for creating a prompt in accordance with user preferences.
WebUI for creating new concepts from existing tokens with sliders representing how much of each token to mix in
Experiment with hivemind’s 8bit transformer finetuning before load_in_8bit existed.. now this is redundant 🙃whoopsy
Wrapper around OpenAI’s CLIP for few/0-shot classification of images
Bert for spam detection / arbitrary few-shot text classification
WIP robust notebook for fine-tuning (almost) *any* decoder-only transformer on the HF Hub for causal language modeling on any dataset (coming soon? to a Colab notebook near you?)
Other Assorted Models/Files:
- Finetuned SD Models
https://huggingface.co/doohickey/doodad-v1-3 https://huggingface.co/crumb/dalle-dreambooth
https://huggingface.co/crumb/eva-fusion-v2.22
https://huggingface.co/crumb/icon-diffusion-v1-1
- CLIP-related
https://huggingface.co/crumb/midjourney-textual-inversions
(a group of cinematic styles for generating images that is automatically pre-loaded in Doohickey)
https://huggingface.co/crumb/ViT-L-14-Token-Embeddings
(preprocessed every single token into its full embedding for ViT-L/14, for use in search functions for optimizing)
There are countless other side-projects I ran this year (there’s an implementation of https://arxiv.org/pdf/2211.01910.pdf somewhere) but I’m pretty sure this hits all the main points.
