Running the Application
Last updated
Was this helpful?
Last updated
Was this helpful?
Before you run , let's examine the code.
1. Importing the Required Modules
ChatSecret
: Handles communication with the LLM, such as sending messages and receiving responses.
Secret
: Manages secure access to LLM models and provides metadata about available models.
2. Initializing the Secret Client
Creates an instance of Secret
, which is used to interact with the Secret backend to fetch available LLM models and their connection details.
3. Fetching Models and URLs
get_models()
: Retrieves a list of all LLM models registered with the backend.
get_urls(model=models[0])
: For the first model in the list (models[0]
), fetches a list of instance URLs where the model can be accessed.
base_url=urls[0]
: Specifies the first URL in the list as the endpoint to connect to the LLM.
model=models[0]
: Specifies the model to use (e.g., Llama or Deepseek, etc).
temperature=1.0
: Configures the "creativity" of the model's responses. Higher values produce more varied outputs, while lower values make responses more deterministic.
Messages are defined as a list of tuples:
("system", ...)
: Instructions for the model about its role (e.g., "therapist").
("human", ...)
: A message from the user (e.g., expressing sadness about missing their cat).
invoke(messages, stream=False)
: Sends the messages to the LLM for processing.
messages
: The list of chat messages.
stream=False
: Indicates that the response should be returned all at once rather than streamed in parts.
node_url
If you experience issues with the default node_url
(SECRET_NODE_URL_DEFAULT
config variable), instead of declaring the client like this:
you can manually specify a node_url
when instantiating the Secret
client:
Alternatively, you can set the node_url
via an environment variable:
To run the sample application:
Upon successful installation, you should see a response:
The Secret AI SDK provides developers with an early glimpse into integrating confidential LLMs, enabling secure and innovative applications. As a developer preview, it is designed for testing and exploration, not production use. Feedback is encouraged to help refine the SDK for future releases.
For more details on available endpoints and a list of LCD nodes, refer to the official Secret Network documentation:
Congrats on your first integration with Secret AI SDK!
Join the group on Telegram to share feedback and get code assistance