Train a LoRA
Last updated
Last updated
First, let's briefly introduce what LoRA is. LoRA stands for Low-Rank Adaptation. It allows you to use low-rank adaptation technology to quickly fine-tune diffusion models. Simply put, the LoRA training model makes it easier to train Stable Diffusion on different concepts, such as characters or a specific style. These trained models can then be exported and used by others in their own generations.
In this section, we will demonstrate how to Train a LoRA on defusion.ai.
Find the "Train a LoRA" button.
Go to https://app.defusion.ai/trainModel or click the "Train a LoRA" button on the page
Start with "Create Your LoRA"
Based on the type of LoRA you want to train, choose one of Anime, Real Person, or Free Form, and then enter a name for your LoRA in the box below (you can still change it after training).
Add Training Data
Upload Dataset
The quality of the training set is crucial for the success of the model.
If you're training a set of NFT styles, just upload 10 to 20 images of that Collection. For training a specific person or style, use clear images with a well-defined subject.
The images should have a minimum pixel size of 512x512 and a maximum of 3000x3000, with larger sizes being more beneficial for the model's learning.
Additionally, contrary to common belief, it's not necessary to ensure consistent sizes or proportions for each image.
Captioning the Images
Captioning the images is aimed at facilitating the model's better comprehension of the visual content.
We offer an "automatic tagging" feature, which helps you quickly complete the captioning process.
After clicking "auto tag," you'll see a popup, where we strongly recommend you fill in the "Prefix to add to BllP caption."
The prefix will be added as the first word of each caption, serving as a "trigger word" for this LoRA. Here, we recommend you enter a non-existent word, for example, instead of "apple," use "appl." Otherwise, the model might confuse your images with real apples in the future.
After clicking submit, each model will be captioned successfully. If you'd like, you can review each annotation to ensure it matches the image. Alternatively, you can simply move on by clicking "next."
Edit Parameters
On the "Review and Submit" page, we need to check the parameters for LoRA training. If you're not familiar with these parameters, don't worry. You can simply use the default configuration we've set up for you, which should still yield good training results.
For the "instant prompt," we suggest using the same text as the "prefix" from the previous page. (Remember? The trigger word.) As for the "class prompt," you can tailor it according to the type of training set you've submitted, such as anime, man, women, and so on.
If you wish to fine-tune the model parameters, hover over the parameter names for tooltips that explain their functions. You can also refer to our training experiment records for more detailed information.
Submit and await the surprises
Now you're all set. Click on the submit button, and your submission will enter the task queue. Once the task starts, we'll provide real-time feedback on the remaining time based on your training parameters. You can check the progress on the "My Models" page.
Publish the LoRA
Once your LoRA is trained, you'll need to perform a simple public action to make it visible to others. Follow the instructions below to complete this step.
Select Model File
Once your LoRA is trained, you'll need to perform a simple public action to make it visible to others. First, on the "Training" tab of the "My Models" page, locate your LoRA and click "Go Set."
For each epoch, a LoRA file and its corresponding sample image will be generated. Choose your favorite set, as it will showcase the image results trained by your upcoming LoRA publication. Click "Publish".
Create a Post
This step is to generate sample images for your model before you officially publish it. These images will be displayed on your LoRA's details page in the future.
So there are two ways to create post images. The first is by directly uploading, which requires you to download LoRA and generate images through other methods. The second option is our on-site text-to-image tool, for which you have 12 free image generation opportunities.
Once you have all the sample images ready, click "submit."
Edit Model
On this page, you can set the LoRA's Name and edit the Creator Note. Once you've finished editing, click "submit" to complete the LoRA's publication.
However, there's one area where you'll need to fill in yourself, and that's the "instant prompt" and "class prompt."
