Skip to content

Invoke LLM

Invoke a Large Language Model (LLM) to generate text based on a given instruction.

This endpoint supports general-purpose text generation, rewriting, summarization, question answering, and other LLM tasks. It can be used together with other endpoints to build powerful LLM applications.

Example Use Cases
Use Case instruction input
Rewrite text "Rewrite the following text in first-person plural (using 'we' and 'us') as a customer service agent." "You can contact AwesomeCompany at 512-512-5252. They are happy to assist you with any questions you may have."
Answer a question "You are a knowledgeable assistant. Answer questions clearly and concisely." "What is the capital of France?"
Summarization "Summarize the following article in one paragraph." "The Great Wall of China is one of the most famous landmarks..."
Creative writing "You're a creative writer. Write a short sci-fi story under 300 words." (empty)
Explain a concept "Explain the concept of black holes in simple terms." (empty)
Format conversion "Convert the following text into a bullet-point list." "The benefits of exercise include improved cardiovascular health, stronger muscles, better mental health, and increased energy levels."
Grammar correction "Fix grammar and spelling errors in the following text." "He dont know where is the nearest train station."
Translate text "Translate the following text from English to French." "Hello, how are you?"
Resume/CV improvement "Rewrite the following resume summary in a more professional tone." "I worked as a software dev, I made apps, and I know Python."
Social media post "Generate a short and engaging tweet about AI advancements." (empty)
Legal text simplification "Rewrite the following contract clause in plain language." "The indemnifying party shall hold harmless and indemnify the indemnified party against any and all liabilities..."

Best Practices:

  • Be specific with instruction
    • Instead of "Rewrite this text", use "Rewrite this text in a professional tone."
  • For open-ended tasks, omit input parameter
    • "Write a futuristic sci-fi story in under 300 words."

Endpoint

POST /v20241104/invoke-llm

API Request

  • Required Parameters

    instruction string required

    Describes what the LLM should do. This acts as the system message, in LLM terms.

    Info

    Examples:

    "instruction": "Rewrite the following text in first-person plural (using 'we' and 'us') as a customer service agent."

    "instruction": "You're a creative writer. Write a short sci-fi story under 300 words."

    MinLength: 1   MaxLength: 2000

    Optional Parameters

    input string

    The content the instruction applies to. If omitted, the LLM will generate an output based only on instruction.

    Info

    Example:

    "input": "You can contact AwesomeCompany at 512-512-5252. They are happy to assist you with any questions you may have."

    MinLength: 1   MaxLength: 10000

    max_output_tokens integer

    Specifies the maximum number of tokens the LLM can generate in its output, which sets an upper limit on the length of the generated answer. Default is 512.

    Info
    • Lower values reduce LLM output token usage and response time - but may result in less informative answers, or answers that are cut off.

    Min: 128   Max: 4096

    temperature float

    Controls the randomness of the answer generated by the LLM. Default is 0.5.

    Info

    Lower values lead to a more predictable and conservative wording of the answer. Higher values lead to a more creative (and potentially less coherent) wording.

    Min: 0   Max: 1

  • POST /v20241104/invoke-llm

    curl -X POST "https://api.gainly.ai/v20241104/invoke-llm" \
  -H "Content-Type: application/json" \
  -H "X-API-Key: YOUR_API_KEY_HERE" \  # (1)!
  -d '{
    "instruction": "Rewrite the following text in first-person plural (using we and us) as a customer service agent.",
    "input": "You can contact AwesomeCompany at 512-512-5252. They are happy to assist you with any questions you may have."
  }'
    1. Replace YOUR_API_KEY_HERE with the value of your API key.
    # Prompt for AI coding assistants/IDEs (e.g., ChatGPT, Claude, GitHub Copilot, Cursor, Windsurf)

Using the Gainly API:
1. Write code to call the invoke_llm operation (see OpenAPI spec: https://api.gainly.ai/v20241104/openapi.json)
2. Implement authentication using the header "X-API-Key" as described in the docs: https://docs.gainly.ai/latest/api-reference/authentication/
3. Implement rate limit handling as described in the docs: https://docs.gainly.ai/latest/api-reference/rate-limits/
4. Implement error handling
5. Handle the response according to the InvokeLlmResults schema in the OpenAPI spec

    using System.Net.Http;
using System.Text.Json;
using System.Text;

var client = new HttpClient();

var url = "https://api.gainly.ai/v20241104/invoke-llm";

var payload = new {
    instruction = "Rewrite the following text in first-person plural (using 'we' and 'us') as a customer service agent.",
    input = "You can contact AwesomeCompany at 512-512-5252. They are happy to assist you with any questions you may have."
};

var content = new StringContent(
    JsonSerializer.Serialize(payload),
    Encoding.UTF8,
    "application/json"
);

client.DefaultRequestHeaders.Add("X-API-Key", "YOUR_API_KEY_HERE"); // (1)!

var response = await client.PostAsync(url, content);
var result = await response.Content.ReadAsStringAsync();
Console.WriteLine(result);
    1. Replace YOUR_API_KEY_HERE with the value of your API key.
    package main

import (
    "bytes"
    "encoding/json"
    "fmt"
    "net/http"
)

func main() {
    url := "https://api.gainly.ai/v20241104/invoke-llm"

    payload := map[string]interface{}{
        "instruction": "Rewrite the following text in first-person plural (using 'we' and 'us') as a customer service agent.",
        "input": "You can contact AwesomeCompany at 512-512-5252. They are happy to assist you with any questions you may have.",
    }

    jsonData, _ := json.Marshal(payload)

    req, _ := http.NewRequest("POST", url, bytes.NewBuffer(jsonData))
    req.Header.Set("Content-Type", "application/json")
    req.Header.Set("X-API-Key", "YOUR_API_KEY_HERE") // (1)!

    resp, _ := http.DefaultClient.Do(req)
    defer resp.Body.Close()

    var result map[string]interface{}
    json.NewDecoder(resp.Body).Decode(&result)
    fmt.Println(result)
}
    1. Replace YOUR_API_KEY_HERE with the value of your API key.
    import java.net.http.HttpClient;
import java.net.http.HttpRequest;
import java.net.http.HttpResponse;
import java.net.URI;

var client = HttpClient.newHttpClient();

var url = "https://api.gainly.ai/v20241104/invoke-llm";

var payload = """
    {
        "instruction": "Rewrite the following text in first-person plural (using 'we' and 'us') as a customer service agent.",
        "input": "You can contact AwesomeCompany at 512-512-5252. They are happy to assist you with any questions you may have."
    }
    """;

var request = HttpRequest.newBuilder()
    .uri(URI.create(url))
    .header("Content-Type", "application/json")
    .header("X-API-Key", "YOUR_API_KEY_HERE") // (1)!
    .POST(HttpRequest.BodyPublishers.ofString(payload))
    .build();

var response = client.send(request, HttpResponse.BodyHandlers.ofString());
System.out.println(response.body());
    1. Replace YOUR_API_KEY_HERE with the value of your API key.
    const axios = require('axios');  // or: import axios from 'axios';

const url = 'https://api.gainly.ai/v20241104/invoke-llm';

const payload = {
    instruction: "Rewrite the following text in first-person plural (using 'we' and 'us') as a customer service agent.",
    input: "You can contact AwesomeCompany at 512-512-5252. They are happy to assist you with any questions you may have."
};

const headers = {
    'Content-Type': 'application/json',
    'X-API-Key': 'YOUR_API_KEY_HERE' // (1)!
};

axios.post(url, payload, { headers })
    .then(response => console.log(response.data))
    .catch(error => console.error('Error:', error.message));
    1. Replace YOUR_API_KEY_HERE with the value of your API key.
    <?php

$client = new \GuzzleHttp\Client();

$url = 'https://api.gainly.ai/v20241104/invoke-llm';

$payload = [
    'instruction' => "Rewrite the following text in first-person plural (using 'we' and 'us') as a customer service agent.",
    'input' => "You can contact AwesomeCompany at 512-512-5252. They are happy to assist you with any questions you may have."
];

$response = $client->request('POST', $url, [
    'json' => $payload,
    'headers' => [
        'Content-Type' => 'application/json',
        'X-API-Key' => 'YOUR_API_KEY_HERE' # (1)!
    ],
]);

echo $response->getBody();
    1. Replace YOUR_API_KEY_HERE with the value of your API key.
    import requests

url = "https://api.gainly.ai/v20241104/invoke-llm"

payload = {
    "instruction": "Rewrite the following text in first-person plural (using 'we' and 'us') as a customer service agent.",
    "input": "You can contact AwesomeCompany at 512-512-5252. They are happy to assist you with any questions you may have."
}

headers = {
    "Content-Type": "application/json",
    "X-API-Key": "YOUR_API_KEY_HERE" # (1)!
}

response = requests.post(url, json=payload, headers=headers)
data = response.json()
print(data)
    1. Replace YOUR_API_KEY_HERE with the value of your API key.
    require 'json'
require 'uri'
require 'net/http'
require 'openssl'

url = URI('https://api.gainly.ai/v20241104/invoke-llm')

http = Net::HTTP.new(url.host, url.port)
http.use_ssl = true

request = Net::HTTP::Post.new(url)
request['Content-Type'] = 'application/json'
request['X-API-Key'] = 'YOUR_API_KEY_HERE' # (1)!
request.body = {
    instruction: "Rewrite the following text in first-person plural (using 'we' and 'us') as a customer service agent.",
    input: "You can contact AwesomeCompany at 512-512-5252. They are happy to assist you with any questions you may have."
}.to_json

response = http.request(request)
puts response.read_body
    1. Replace YOUR_API_KEY_HERE with the value of your API key.

API Response

{
    "object": "invoke_llm_result",
    "url": "/v20241104/invoke-llm",
    "data": [
        {
            "output": "We can be reached at 512-512-5252. We are happy to assist you with any questions you may have.",
            "stop_reason": "end_turn"
        }
    ],
    "instruction": "Rewrite the following text in first-person plural (using 'we' and 'us') as a customer service agent.",
    "input": "You can contact AwesomeCompany at 512-512-5252. They are happy to assist you with any questions you may have.",
    "max_output_tokens": 512,
    "temperature": 0.5,
    "model": "model_3",
    "token_usage": {
        "semantic_tokens": 0,
        "llm_tokens": {
            "llm_output_tokens": 33,
            "llm_input_tokens": 70,
            "model": "model_3"
        }
    },
    "livemode": false
}

Output

output represents the AI-generated answer for the user's query.

1
2
3
4
5
6
"data": [
    {
        "output": "We can be reached at 512-512-5252. We are happy to assist you with any questions you may have.",
        "stop_reason": "end_turn"
    }
],

Stop Reason

stop_reason indicates why the model stopped generating tokens while responding to the user's query.

  • end_turn: The model successfully completed generating the answer.
  • max_tokens: The model ran out of tokens, suggesting that you may need to do one or more of the following:
    • Increase the max_output_tokens value in your API request.
    • Add a maximum length of text as a part of the instruction parameter in your API request.
  • not_available: Reason is not available (rare cases).

Token Usage

token_usage indicates the number of tokens used to process the query and generate an answer.

"token_usage": {
    "semantic_tokens": 0,
    "llm_tokens": {
        "llm_output_tokens": 33,    // determined by the amount of text the LLM has to write
        "llm_input_tokens": 70,     // determined by the amount of text the LLM has to read
        "model": "model_3"
    }
}