How much does the Llama 3.1 8B Instruct API cost?

On FastMetal, Llama 3.1 8B Instruct is billed per token: $0.0211 per 1M input tokens and $0.0422 per 1M output tokens on a US-dollar account, before tax. Usage is drawn from a prepaid balance; there is no subscription or monthly fee.

Can I call Llama 3.1 8B Instruct with the OpenAI SDK?

Yes. Point base_url at https://api.fastmetal.ai/v1 and pass "llama-3.1-8b-instruct" as the model; existing OpenAI-style code works unchanged, including streaming, tool calls and structured output.

What is the context window of Llama 3.1 8B Instruct?

131,072 tokens, shared between the prompt and the response.

What do I need to try Llama 3.1 8B Instruct?

Create an account and add credit; the model is then available both in the browser chat and over the API. There is no contract or minimum spend.

Back to Models

Llama 3.1 8B Instruct

Meta's latest class of model (Llama 3.1) launched with a variety of sizes & flavors. This 8B instruct-tuned version is fast and efficient. It has demonstrated strong performance compared to...

7/23/2024

131,072 tokens

Input: $0.0211/M

Output: $0.0422/M

Try in Chat Get API Key

Specifications

Modalities

Input

text

Output

text

Supported Parameters

frequency_penalty

logit_bias

logprobs

max_tokens

min_p

presence_penalty

repetition_penalty

response_format

seed

stop

structured_outputs

temperature

tool_choice

tools

top_k

top_logprobs

top_p

Max Output Tokens

117,964

Data policy

Prompt retention: Unknown — we could not confirm
Training: Not used for training

"Unknown" does not mean "safe". It means we could not confirm it.

Whether a provider trains on prompts is a declared value from our terms with them. Retention is determined from the upstream listing for every host this model can reach. Neither is guessed.

Measured performance

Time to first response (p50)2.6s

Throughput (p50)40 tokens/s

Measured through FastMetal's own gateway over the last 30 days (657 first-response samples).

Throughput is sampled once a day: the median of 27 observations over the last 30 days.

Time to first response is measured to the first streamed chunk; throughput counts all output tokens, including reasoning tokens. These are observations, not a performance guarantee.

Code Examples

curl https://api.fastmetal.ai/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -d '{
    "model": "llama-3.1-8b-instruct",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

How Llama 3.1 8B Instruct actually answers

Real responses to our standard prompts, recorded on FastMetal.

Count the number of 'r's in 'strawberry'

Count the number of 'r's in 'strawberry'. Explain your reasoning step by step.

To count the number of 'r's in 'strawberry', I will break down the word into its individual letters and examine each one. Here's the step-by-step process: 1. The word 'strawberry' consists of the following letters: s-t-r-a-w-b-e-r-r-y. 2. I will start by counting the letters that are not 'r'. The word has 1 's', 1 't', 1 'a', 1 'w', 1 'b', 1 'e', 1 'y'. 3. Next, I will count the number of 'r's. The word has 2 'r's, one of which is part of the letter combination 'rr'. Therefore, there are 2 'r's in the word 'strawberry'.

9.7s response169 output tokens

Debug This Error

I'm getting the following error in my Node.js application: TypeError: Cannot read properties of undefined (reading 'map') at UserList (/app/components/UserList.js:12:25) at renderWithHooks (/app/node_modules/rea…

The error `TypeError: Cannot read properties of undefined (reading 'map')` typically occurs when you're trying to access a property (in this case, `map`) on an object that is `undefined`. In your case, the `users` prop passed to the `UserList` component is `undefined`. This is likely because the parent component that renders `UserList` is not passing the `users` prop, or the prop is not being properly initialized. To fix this error, you can add a simple check to ensure that `users` is not `undefined` before trying to call `map` on it: ```javascript function UserList({ users }) { if (!users) return null; // or return a default message, e.g. "No users found" return ( <ul> {us…

7.9s response292 output tokens

Code Review

Please review the following Python function and suggest improvements for readability, performance, and best practices: def get_data(url, retries=3): import requests import time for i in range(retries):…

Here are some suggestions for improving the function's readability, performance, and best practices: **Readability Improvements** 1. **Function name**: Consider renaming `get_data` to `fetch_active_items` to better reflect its purpose. 2. **Variable naming**: Use more descriptive variable names, such as `response` instead of `r`, and `active_items` instead of `result`. 3. **Code organization**: Break up the function into smaller, more focused sections. This will improve readability and make it easier to understand the logic. 4. **Type hints**: Add type hints for function parameters and return types to make the code more self-documenting. **Performance Improvements** 1. **Use a more effic…

29.7s response533 output tokens

Compare these answers side by side with other models →

Frequently asked questions

How much does the Llama 3.1 8B Instruct API cost?: On FastMetal, Llama 3.1 8B Instruct is billed per token: $0.0211 per 1M input tokens and $0.0422 per 1M output tokens on a US-dollar account, before tax. Usage is drawn from a prepaid balance; there is no subscription or monthly fee.
Can I call Llama 3.1 8B Instruct with the OpenAI SDK?: Yes. Point base_url at https://api.fastmetal.ai/v1 and pass "llama-3.1-8b-instruct" as the model; existing OpenAI-style code works unchanged, including streaming, tool calls and structured output.
What is the context window of Llama 3.1 8B Instruct?: 131,072 tokens, shared between the prompt and the response.
What do I need to try Llama 3.1 8B Instruct?: Create an account and add credit; the model is then available both in the browser chat and over the API. There is no contract or minimum spend.