Search for a command to run...
Accessing over 100 LLMs using a single LiteLLM SDK
Talks about artificial intelligence, building SaaS solutions, product management, personal finance, freelancing, business, system design, programming and tech career tips
Decoding the recent developments in the world of Generative Artificial Intelligence like ChatGPT and Midjourney, their real-world use cases, and limitations.
How to craft a top selling coloring book with Generative AI?
Introduction 
Making llms argue for answers to problems that confuse them
As someone who practically lives in their browser, Iβve spent years chasing the perfect browsing experience. I wanted a tool thatβs sleek, powerful, and respects my time and privacy. After falling in love with Arc for its innovative features, I thoug...
π£οΈ Introduction Since you started coding, youβve typed letters, numbers, even emojisβbut how does your computer get it? The answerβs character encoding, the magic that turns βA,β βδΈ,β or βπβ into binary. This guide takes you from ASCIIβs baby steps...
The modern technique of image interaction with AI
Since early 2024, the landscape of Language Models has become increasingly complex with multiple providers and APIs. Developers often struggle to integrate different LLMs into their applications efficiently. LiteLLM simplifies this process by providing a unified interface for all your LLM needs.
LiteLLM is a powerful open-source toolkit that revolutionizes how developers interact with Large Language Models (LLMs). Think of it as a universal translator for LLMs β it allows your application to communicate with any supported language model using a single, consistent interface. π
Universal Model Access π
Seamlessly connect to 100+ LLMs from major providers
Use the same code for OpenAI, Anthropic, Ollama, and other providers
Switch between models without rewriting your application logic
Simplified Development π¨βπ»
Write once, deploy anywhere approach
Consistent completion format across all models
Built-in error handling and retry mechanisms
Automatic model fallbacks for improved reliability
Resource Management π
Monitor usage across different projects
Track costs for each model integration
Built-in load balancing capabilities
Optimize resource allocation automatically
Enterprise-Ready Features β‘
Support for multiple API keys
Customizable retry logic
Detailed logging and monitoring
Production-grade reliability
curl https://ollama.ai/install.sh | sh
sudo apt-get update && sudo apt-get install -y cuda-drivers
This installs the necessary CUDA drivers for GPU acceleration.
echo 'debconf debconf/frontend select Noninteractive' | sudo debconf-set-selections
nohup ollama serve &
Note: This command:
Runs Ollama server in the background
Continues running after terminal closure (via nohup)
Remains active until explicitly terminated
pip install litellm
ollama list
Initial output will be empty:
NAME ID SIZE MODIFIED
Browse available models at ollama.com/library.
For this tutorial, we'll use Llama 3.2:
ollama pull llama3.2:3b-instruct-fp16
Model Specifications:
3B parameter model
Instruction-tuned variant
FP16 precision for optimized memory usage
Suitable for consumer GPUs
import os
from google.colab import userdata
# Set API keys as environment variables
os.environ["ANTHROPIC_API_KEY"] = userdata.get("ANTHROPIC_API_KEY")
os.environ["OPENAI_API_KEY"] = userdata.get("OPENAI_API_KEY")
from litellm import completion
response = completion(
model="ollama/llama3.2:3b-instruct-fp16",
messages=[{
"content": "Write an API in ExpressJS to get the latest weather stats for a city",
"role": "user"
}],
api_base="http://localhost:11434"
)
print(response['choices'][0]['message']['content'])
# OpenAI GPT-4
response = completion(
model="gpt-4",
messages=[{
"content": "Write an API in ExpressJS to get the latest weather stats for a city",
"role": "user"
}]
)
# Anthropic Claude
response = completion(
model="claude-3-opus-20240229",
messages=[{
"content": "Write an API in ExpressJS to get the latest weather stats for a city",
"role": "user"
}]
)