Tool Calling (previously Function Calling) is a great way to introduce ShuttleAI into your project seamlessly.

Tool calling gives you the ability to give the model certain information about a function and the model will choose how to use that information to generate a function-ready formatted respnse for you automatically.

Not all models are trained with tool calling data, only the following are capable of this feature: shuttle-tools, gemini-pro, mixtral-8x7b, mistral-7b, gpt-3.5-turbo, gpt-4-0613, gpt-3.5-turbo-1106, gpt-3.5-turbo-0613, gpt-3.5-turbo-0125.

​
What is a Tool?

Currently there is only one type of tool, a “function”; this is subject to change in the future.

A “function” lets you give certain information of your function and its requiremnts, and then receive a formatted response ready for that function.

​
How should a Tool look?

Below we have a basic tool utilizing our get_current_weather function

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_current_weather",
            "description": "Get the current weather in a given location",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {
                        "type": "string",
                        "description": "The city and state, e.g. San Francisco, CA",
                    },
                    "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]},
                },
                "required": ["location"],
            },
        },
    }
]

Lets also make the function that we will call as well:

Keep in mind this is a basic example implementation and you would need to use your own weather API to get accurate results yourself.

import random

def get_current_weather(location: str, unit: str = "fahrenheit"):
    """ Get the current weather in a given location. """
    weather_data = {
        "tokyo": (75, 85),
        "san francisco": (55, 65),
        "paris": (65, 75)
    }
    location = location.lower()
    if location in weather_data:
        temp_range = weather_data[location]
        temperature = random.randint(*temp_range)
    else:
        temperature = "unknown"
    return {
        "location": location.title(),
        "temperature": temperature
    }

Now that we have our tools and functions set up, lets make our request:

from shuttleai import *

...
response = await shuttle.chat_completion(
    model="gemini-pro",
    messages=[{"role": "user", "content": "What is the current weather like in Los Angeles California?"}],
    tools=tools,
    tool_choice="auto"
)
print(response)
...

This will result in something similar to the following:

{
    "choices": [
        {
            ...
            "finish_reason": "tool_calls",
            "message": {
                "content": null,
                "role": "assistant",
                "tool_calls": [
                    {
                        "function": {
                            "arguments": "{\"location\":\"Los Angeles, CA\"}",
                            "name": "get_current_weather"
                        },
                        "id": "call_3GjyYbPEskNsP67fkjyXjI1g",
                        "type": "function"
                    }
                ]
            }
        }
    ],
    ...
}

With this response, we can easily use Python’s built in json module with import json and using json.loads(response['choices'][0]['message']['tool_calls']['function']['arguments']) to get the required properties for your function.

ShuttleAI supports both Parallel Tool Calling and Streamed Tool Calling!

For a full example using Tools, check out Web Access!

ModelsWeb access