LLM in Comfy. Automatic prompts!

DOWNLOAD example workflow in bottom of post!

Video guide here: https://youtu.beMYSjvwD_s

With these nodes you will connect a LLM to Comfy and get great prompts for your images. Especially great for Flux, which is trained for natural language.

How to install

Go into your manager
Select "Custom Nodes Manager"

Search for Searge LLM and press Install

Download https://huggingface.co/MaziyarPanahi/Mistral-7B-Instruct-v0.3-GGUF/resolve/main/Mistral-7B-Instruct-v0.3.Q4_K_M.gguf
Create the directory and place model in /ComfyUI/models/llm_gguf/

Restart your Comfy
DONE!

These are the nodes available and how they connect.

From the generate string, you connect to your CLIP Text encoder (Prompt). To be able to connect to the CLIP Text Encode, you can right click on that node and select "Convert widget to input"

As per the instructions from Searge's Github:
Configure the Searge_LLM_Node with the necessary parameters within your ComfyUI project to utilize its capabilities fully:

• text: The input text for the language model to process.

• model: The directory name of the model within models/llm_gguf you wish to use.

• max_tokens: Maximum number of tokens for the generated text, adjustable according to your needs.

• instructions: The instructions for the language model to generate a prompt. It supports the placeholder {prompt} to insert the prompt from the text input. Example: Generate a prompt from "{prompt}"

Advanced Options Node

The Searge_AdvOptionsNode offers a range of configurable parameters allowing for precise control over the text generation process and model behavior.

The default values on this node are also the defaults that Searge_LLM_Node uses when no Searge_AdvOptionsNode is connected to it.

Below is a detailed overview of these parameters:

• Temperature (temperature): Controls the randomness in the text generation process. Lower values make the model more confident in its predictions, leading to less variability in output. Higher values increase diversity but can also introduce more randomness. Default: 1.0.

• Top-p (top_p): Also known as nucleus sampling, this parameter controls the cumulative probability distribution cutoff. The model will only consider the top p% of tokens with the highest probabilities for sampling. Reducing this value helps in controlling the generation quality by avoiding low-probability tokens. Default: 0.9.

• Top-k (top_k): Limits the number of highest probability tokens considered for each step of the generation. A value of 0 means no limit. This parameter can prevent the model from focusing too narrowly on the top choices, promoting diversity in the generated text. Default: 50.

• Repetition Penalty (repetition_penalty): Adjusts the likelihood of tokens that have already appeared in the output, discouraging repetition. Values greater than 1 penalize tokens that have been used, making them less likely to appear again. Default: 1.2.

These parameters provide granular control over the text generation capabilities of the Searge_LLM_Node, allowing users to fine-tune the behavior of the underlying models to best fit their application requirements.

ALTERNATE Manual install from GIT Url

Paste & install https://github.com/SeargeDP/ComfyUI_Searge_LLM.git

If this gives you an error about permissions, you need to change your security settings.
Go into \ComfyUI\custom_nodes\ComfyUI-Manager\
Open config.ini
Change security_level = weak
Save the file