date
Dec 12, 2024
type
Page
status
Published
slug
summary
Install comfyUI LLM party and use LLM via api
tags
Tutorial
Teaching
ComfyUI
UCL
category
Knowledge
icon
password
URL
1.What is Comfyui_LLM_party
Comfyui_llm_party aims to develop a complete set of nodes for LLM workflow construction based on comfyui as the front end. It allows users to quickly and conveniently build their own LLM workflows and easily integrate them into their existing image workflows.
2.Install comfyui_LLM_party
You could directly search and install LLM party in your comfyui Manager. It may take around 20-30 minutes to finish installation, please be patient for that.
If the installation has been interrupted and import failed, you could search and uninstall it in comfyui manager, then try to reinstall it again.
3.Use API to call LLM in comfyui
3.1 Under stand the basic input for LLM.
Once you finish the installation, you could try to talk with chatGPT (gpt-4o-mini) in your comfyui with the workflow bellow.
After you open this workflow, there are several parameters you need to take care about before running.
- Api-key: api-key is necessary for accessing openAI services.
- system_prompt: The “role (character)” you assign to AI.
- user_propmt: The question you want to ask AI / the task you want AI to do.
3.2 How to get your OpenAI API-key.
Before creating an API secret key, you'll need to add a payment method (debit or credit card) to your OpenAI account.
For the API
In the billing panel, add a payment method (under Payment Methods). After adding a payment method, add credits to your organization by clicking 'Buy credits'.
Under our billing panel, you'll be able to edit your billing details under Preferences and edit your spend limits under Usage Limits.
Please note that the API service is different from ChatGPT Plus. The API operates on a pay-as-you-go basis, meaning you only pay for what you use. For typical personal usage that doesn't involve processing large volumes of documents, the costs are usually minimal. You can review the detailed pricing on OpenAI's pricing page.
It is also possible to use other AI model services and even local LLM on comfyui LLM party. Check the model support list here.
Then You can find your Secret API key on the API key page.
Then you could create your own API key.
Please be careful the API key will only display once. Remember to save it in a safety place.
3.3 Talk with AI in Comfyui.
Now, you could fill in the API-Key, system_prompt, user_prompt and test the flow.
You can use my system_prompt as an example.
You are a pioneering biological architect specializing in biomimicry. When I provide you with the name of a creature, please analyze its key biological features, including physical structure, adaptations, and survival mechanisms. Then, based on your analysis, develop hypotheses about potential bio-inspired innovations and explain how these biological features could inform new technological and architectural developments.
5. Test with more example.
You could find more LLM example from here.
Here are some simple video tutorial for the exmaple.
6.Render the LLM output in Markdown format.
Markdown is a lightweight markup language that adds formatting elements to plain text documents.
The output from LLM normally is in markdown format. If you want to render them in correct way, you could copy and paste them in anykind of markdown editor, such as notion, marktext and typora.
