{"id":1382,"date":"2023-06-21T09:27:59","date_gmt":"2023-06-21T03:27:59","guid":{"rendered":"http:\/\/magiteker.com\/?p=1382"},"modified":"2023-06-21T09:36:02","modified_gmt":"2023-06-21T03:36:02","slug":"resource-training-stable-diffusion-with-dreambooth","status":"publish","type":"post","link":"https:\/\/magiteker.com\/index.php\/2023\/06\/21\/resource-training-stable-diffusion-with-dreambooth\/","title":{"rendered":"Resource: Training Stable Diffusion with Dreambooth"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\">&#8211;<a href=\"https:\/\/huggingface.co\/blog\/dreambooth\">Blog Post<\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">This article describes experiments with different Learning Rates in training models using Dreambooth.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><a href=\"https:\/\/huggingface.co\/blog\/dreambooth#summary-of-initial-results\"><\/a> FTA: Summary of Initial Results<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">To get good results training Stable Diffusion with Dreambooth, it&#8217;s important to tune the learning rate and training steps for your dataset.<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>High learning rates and too many training steps will lead to overfitting. The model will mostly generate images from your training data, no matter what prompt is used.<\/li>\n\n\n\n<li>Low learning rates and too few steps will lead to underfitting: the model will not be able to generate the concept we were trying to incorporate.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">Faces are harder to train. In our experiments, a learning rate of <code>2e-6<\/code> with <code>400<\/code> training steps works well for objects but faces required <code>1e-6<\/code> (or <code>2e-6<\/code>) with ~1200 steps.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Image quality degrades a lot if the model overfits, and this happens if:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>The learning rate is too high.<\/li>\n\n\n\n<li>We run too many training steps.<\/li>\n\n\n\n<li>In the case of faces, when no prior preservation is used, as shown in the next section.<\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"<p>&#8211;Blog Post This article describes experiments with different Learning Rates in training models using Dreambooth. FTA: Summary of Initial Results To get good results training Stable Diffusion with Dreambooth, it&#8217;s important to tune the learning rate and training steps for your dataset. Faces are harder to train. In our experiments, a learning rate of 2e-6 [&hellip;]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_feature_clip_id":0,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2},"jetpack_post_was_ever_published":false},"categories":[21,67],"tags":[68,71,69],"class_list":["post-1382","post","type-post","status-publish","format-standard","hentry","category-code","category-stablediffusionai","tag-ai","tag-dreambooth","tag-stable-diffusion"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p6MIgq-mi","jetpack-related-posts":[{"id":1386,"url":"https:\/\/magiteker.com\/index.php\/2023\/06\/21\/resource-dreambooth-training-readme\/","url_meta":{"origin":1382,"position":0},"title":"Resource: Dreambooth Training README","author":"Frank O'Hanlon","date":"June 21, 2023","format":false,"excerpt":"- Github README describing proper methods for constructing a dataset.According to the README Regularization images allow for prior preservation of a models existing dataset. Regularization images should be generated before training from the base model. Image classification should be a generic term ( i.e. person, cat, dog, man, woman ).\u2026","rel":"","context":"In &quot;Code&quot;","block_context":{"text":"Code","link":"https:\/\/magiteker.com\/index.php\/category\/code\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/magiteker.com\/wp-content\/uploads\/2023\/06\/image-1.png?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/magiteker.com\/wp-content\/uploads\/2023\/06\/image-1.png?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/magiteker.com\/wp-content\/uploads\/2023\/06\/image-1.png?resize=525%2C300&ssl=1 1.5x, https:\/\/i0.wp.com\/magiteker.com\/wp-content\/uploads\/2023\/06\/image-1.png?resize=700%2C400&ssl=1 2x"},"classes":[]},{"id":1411,"url":"https:\/\/magiteker.com\/index.php\/2023\/06\/24\/dreambooth-kohya-ss-install\/","url_meta":{"origin":1382,"position":1},"title":"Dreambooth Kohya SS Install","author":"Frank O'Hanlon","date":"June 24, 2023","format":false,"excerpt":"- Github Above is the repository for Kohya SS Dreambooth, to install simply clone the repository to a separate folder using Git then run the install.bat file. Note: For better training speed ( iterations per second ) using RTX 40xx GPU's it's advised to use CUDA 11.8 so that Xformers\u2026","rel":"","context":"In &quot;Code&quot;","block_context":{"text":"Code","link":"https:\/\/magiteker.com\/index.php\/category\/code\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":1437,"url":"https:\/\/magiteker.com\/index.php\/2023\/07\/08\/tbd-experiment-multiple-classifications\/","url_meta":{"origin":1382,"position":2},"title":"TBD Experiment: Multiple Classifications","author":"Frank O'Hanlon","date":"July 8, 2023","format":false,"excerpt":"Hypothesis: Dreambooth can utilize multiple classifications in regularization to improve model training while maintaining model style and data. Each classification should correspond to a tag that is common throughout the training data.","rel":"","context":"In &quot;Stable Diffusion&quot;","block_context":{"text":"Stable Diffusion","link":"https:\/\/magiteker.com\/index.php\/category\/tutorials\/stablediffusionai\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":1350,"url":"https:\/\/magiteker.com\/index.php\/2023\/06\/21\/stable-diffusion-experiments\/","url_meta":{"origin":1382,"position":3},"title":"Stable Diffusion Experiments","author":"Frank O'Hanlon","date":"June 21, 2023","format":false,"excerpt":"AI module experiments","rel":"","context":"In &quot;Stable Diffusion&quot;","block_context":{"text":"Stable Diffusion","link":"https:\/\/magiteker.com\/index.php\/category\/tutorials\/stablediffusionai\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":1413,"url":"https:\/\/magiteker.com\/index.php\/2023\/06\/26\/experiment-high-accuracy-training\/","url_meta":{"origin":1382,"position":4},"title":"Experiment: High Accuracy Training","author":"Frank O'Hanlon","date":"June 26, 2023","format":false,"excerpt":"Utilizing Kohya_ss GUI I'm experimenting with various settings sourced from different tutorials in the hopes of gathering a basic configuration that can generate accurate renders while using as little time and VRAM as possible during training. Methodology: 1.) To quickly iterate through configurations I'm using a small dataset of 5\u2026","rel":"","context":"In &quot;Stable Diffusion&quot;","block_context":{"text":"Stable Diffusion","link":"https:\/\/magiteker.com\/index.php\/category\/tutorials\/stablediffusionai\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":1441,"url":"https:\/\/magiteker.com\/index.php\/2023\/07\/21\/retrieval-based-voice-conversion\/","url_meta":{"origin":1382,"position":5},"title":"Retrieval Based Voice Conversion","author":"Frank O'Hanlon","date":"July 21, 2023","format":false,"excerpt":"-Github Retrieval Based Voice Conversion is a technique for analyzing speech recordings and training an AI model to emulate vocal patterns. Using this system it is possible to create text to speech models with more natural voices. These models can also be utilized to sing lyrics to songs or recite\u2026","rel":"","context":"In &quot;Stable Diffusion&quot;","block_context":{"text":"Stable Diffusion","link":"https:\/\/magiteker.com\/index.php\/category\/tutorials\/stablediffusionai\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]}],"_links":{"self":[{"href":"https:\/\/magiteker.com\/index.php\/wp-json\/wp\/v2\/posts\/1382","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/magiteker.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/magiteker.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/magiteker.com\/index.php\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/magiteker.com\/index.php\/wp-json\/wp\/v2\/comments?post=1382"}],"version-history":[{"count":3,"href":"https:\/\/magiteker.com\/index.php\/wp-json\/wp\/v2\/posts\/1382\/revisions"}],"predecessor-version":[{"id":1385,"href":"https:\/\/magiteker.com\/index.php\/wp-json\/wp\/v2\/posts\/1382\/revisions\/1385"}],"wp:attachment":[{"href":"https:\/\/magiteker.com\/index.php\/wp-json\/wp\/v2\/media?parent=1382"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/magiteker.com\/index.php\/wp-json\/wp\/v2\/categories?post=1382"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/magiteker.com\/index.php\/wp-json\/wp\/v2\/tags?post=1382"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}