sambanovacloud

SambaNovaCloud

caution

You are currently on a page documenting the use of SambaNovaCloud models as text completion models. We recommend you to use the chat completion models.

You may be looking for SambaNovaCloud Chat Models .

SambaNova's SambaNova Cloud is a platform for performing inference with open-source models

Overview

Integration details

Class	Package	Local	Serializable	JS support	Package downloads	Package latest
SambaNovaCloud	langchain_community	❌	beta	❌

This example goes over how to use LangChain to interact with SambaNovaCloud models

Setup

Credentials

To access ChatSambaNovaCloud models you will need to create a SambaNovaCloud account, get an API key and set it as the SAMBANOVA_API_KEY environment variable:

import getpass
import os

if "SAMBANOVA_API_KEY" not in os.environ:
    os.environ["SAMBANOVA_API_KEY"] = getpass.getpass()

Installation

The integration lives in the langchain-community package. We also need to install the sseclient-py package this is required to run streaming predictions

%pip install --quiet -U langchain-community sseclient-py

Invocation

Call SambaNovaCloud models directly from LangChain!

from langchain_community.llms.sambanova import SambaNovaCloud

llm = SambaNovaCloud(
    model="Meta-Llama-3.1-70B-Instruct",
    max_tokens_to_generate=1000,
    temperature=0.01,
    # top_k = 50,
    # top_p = 1.0
)

print(llm.invoke("Why should I use open source models?"))

API Reference:SambaNovaCloud

**Advantages of Open Source Models**

Using open source models can bring numerous benefits to your project or organization. Here are some reasons why you should consider using open source models:

### 1. **Cost-Effective**

Open source models are free to use, modify, and distribute. This can significantly reduce the costs associated with developing and maintaining proprietary models.

### 2. **Community Support**

Open source models are often maintained by a community of developers and users who contribute to their improvement. This community support can lead to faster bug fixes, new feature additions, and better documentation.

### 3. **Transparency and Customizability**

Open source models provide complete transparency into their architecture and implementation. This allows you to customize and fine-tune the model to suit your specific needs.

### 4. **Faster Development**

By leveraging pre-trained open source models, you can accelerate your development process. You can focus on fine-tuning the model for your specific use case rather than building one from scratch.

### 5. **Improved Security**

Open source models are often reviewed and audited by a large community of developers, which can help identify and fix security vulnerabilities.

### 6. **Interoperability**

Open source models can be easily integrated with other open source tools and frameworks, promoting interoperability and reducing vendor lock-in.

### 7. **Access to State-of-the-Art Technology**

Many open source models are developed by top researchers and institutions, providing access to state-of-the-art technology and techniques.

### Example Use Cases

* **Computer Vision**: Use open source models like TensorFlow's Object Detection API or OpenCV's pre-trained models for image classification, object detection, and segmentation tasks.
* **Natural Language Processing**: Leverage open source models like spaCy or Stanford CoreNLP for text processing, sentiment analysis, and language translation tasks.
* **Speech Recognition**: Utilize open source models like Kaldi or Mozilla's DeepSpeech for speech-to-text applications.

**Getting Started**

To get started with open source models, explore popular repositories on GitHub or model hubs like TensorFlow Hub or PyTorch Hub. Familiarize yourself with the model's documentation, and experiment with pre-trained models before fine-tuning them for your specific use case.

By embracing open source models, you can accelerate your development process, reduce costs, and tap into the collective knowledge of the developer community.

# Streaming response

from langchain_community.llms.sambanova import SambaNovaCloud

llm = SambaNovaCloud(
    model="Meta-Llama-3.1-70B-Instruct",
    max_tokens_to_generate=1000,
    temperature=0.01,
    # top_k = 50,
    # top_p = 1.0
)

for chunk in llm.stream("Why should I use open source models?"):
    print(chunk, end="", flush=True)

API Reference:SambaNovaCloud

**Advantages of Open Source Models**

Using open source models can bring numerous benefits to your project or organization. Here are some reasons why you should consider using open source models:

### 1. **Cost-Effective**

Open source models are free to use, modify, and distribute. This can significantly reduce the costs associated with developing and maintaining proprietary models.

### 2. **Community Support**

Open source models are often maintained by a community of developers and users who contribute to their improvement. This community support can lead to faster bug fixes, new feature additions, and better documentation.

### 3. **Transparency and Customizability**

Open source models provide complete transparency into their architecture and implementation. This allows you to customize and fine-tune the model to suit your specific needs.

### 4. **Faster Development**

By leveraging pre-trained open source models, you can accelerate your development process. You can focus on fine-tuning the model for your specific use case rather than building one from scratch.

### 5. **Improved Security**

Open source models are often reviewed and audited by a large community of developers, which can help identify and fix security vulnerabilities.

### 6. **Interoperability**

Open source models can be easily integrated with other open source tools and frameworks, promoting interoperability and reducing vendor lock-in.

### 7. **Access to State-of-the-Art Technology**

Many open source models are developed by top researchers and institutions, providing access to state-of-the-art technology and techniques.

### Example Use Cases

* **Computer Vision**: Use open source models like TensorFlow's Object Detection API or OpenCV's pre-trained models for image classification, object detection, and segmentation tasks.
* **Natural Language Processing**: Leverage open source models like spaCy or Stanford CoreNLP for text processing, sentiment analysis, and language translation tasks.
* **Speech Recognition**: Utilize open source models like Kaldi or Mozilla's DeepSpeech for speech-to-text applications.

**Getting Started**

To get started with open source models, explore popular repositories on GitHub or model hubs like TensorFlow Hub or PyTorch Hub. Familiarize yourself with the model's documentation, and experiment with pre-trained models before fine-tuning them for your specific use case.

By embracing open source models, you can accelerate your development process, reduce costs, and tap into the collective knowledge of the developer community.

API reference

For detailed documentation of all SambaNovaCloud llm features and configurations head to the API reference: https://python.langchain.com/api_reference/community/llms/langchain_community.llms.sambanova.SambaNovaCloud.html

Related

• LLM conceptual guide
• LLM how-to guides