5.1 Ollama
Open source models can be run locally on your own computer given sufficient compute and storage resources. The easiest tool to run models locally is Ollama. List of models directly supported by Ollama are listed on their website. Resource requirements for models are listed on their GitHub repo.
ollama list
ollama pull llama3.1
ollama show llama3.1
ollama run llama3.1
ollama run llava "What's in this image? /Users/jmorgan/Desktop/smile.png"
Ollama can be used with GUIs as well as integrate programatically with Python and R.
According to Ollama, you should have at least 8 GB of RAM available to run the 7B models, 16 GB to run the 13B models, and 32 GB to run the 33B models.
5.2 Fabric
Fabric allows command-line access to LLM models.
fabric --listvendors
fabric --listmodels
fabric --listpatterns
fabric --setup
fabric --changeDefaultModel llama3.1:8b
fabric --model mistral
fabric -y https://www.youtube.com/watch\?v\=Ed4vBY2IQXg > transcript.txt
cat transcript.txt | fabric -sp summarize
cat transcript.txt | fabric -sp extract_wisdom
pbpaste | fabric -sp extract_wisdom
cat transcript.txt | fabric --model mistral:latest -sp summarize
pbpaste | fabric -sp improve_prompt
fabric -y https://www.youtube.com/watch\?v\=e-gwvmhyU7A -m llama3.1:8b -sp summarize
pbpaste | fabric -m llama3.1:8b -sp summarize
echo "An idea that coding is like speaking with rules." | fabric -sp write_essay
Fabric’s prebuilt prompts (patterns) are listed here. Few of the interesting ones:
- analyze_claims
- create_summary
- create_art_prompt
- extract_wisdom
- extract_references
- extract_ideas
- extract_main_idea
- extract_insights
- improve_prompt
- improve_writing
- improve_academic_writing
- summarize
- summarize_meeting
- translate
- write_essay
fabric --listpatterns
fabric --listpatterns | grep "improve"
fabric --listpatterns | grep "extract"
fabric --listpatterns | grep "summarize"
fabric --listpatterns | grep "write"
fabric --listpatterns | grep "translate"
fabric --listpatterns | grep "create"
fabric --listpatterns | grep "analyze"
Install yt-dlp to allow fabric to download and use youtube videos.
fabric -y https://www.youtube.com/watch?v=zKndCikg3R0 -sp summarize
5.3 R
R packages that are helpful for interacting with LLMs.
Package | Interfaces | Features |
---|---|---|
ellmer | openai,anthropic,gemini etc | query,chat,text->image,vision |
openai samterfa | openai | query,chat,text->image,vision |
openai irudnyts | openai | query,chat,text->image,vision |
rollama | ollama | query,chat,vision |
ollamar | ollama | query,chat |
tidychatmodels | openai,mistral,ollama | query,chat |
huggingfaceR | huggingface | |
audio.whisper | whisper | audio->text |
5.4 UI
Graphical user interfaces for LLMs
Tool | Description |
---|---|
Msty | Remote/local models, chat branching |
Anything LLM | |
OpenWebUI | |
LM Studio | |
Poe |
5.5 Image
Tools for generating images from text.
Tool | Category | Description |
---|---|---|
Microsoft Designer | Text to Image | Generation and editing using Dall-E |
Ideogram | Text to Image | |
Leonardo AI | Text to Image | Advanced reference and style control |
Adobe Firefly | Text to Image | Reference and style control |
Midjourney | Text to Image |
5.6 Other
Other generative AI tools
Tool | Category | Description |
---|---|---|
Perplexity | Text to text | AI driven search |
Zapier | Text to text | Automate workflows |
Google AI Studio | Text to text | Task helper |
Eleven labs | Text to speech | Text to speech |
Runway | Text to video | Video generation |
Udio | Text to Speech | Audio generation |
5.7 Builder
Building LLM Powered Applications
Modern applications leverage specialized frameworks and tools to streamline LLM integration. For example, you can use Python-based libraries like LangChain and Haystack to orchestrate your models, while tools such as LlamaIndex help with efficient information retrieval. Integrated solutions like SanaLabs combine AI agents with intuitive GUIs for rapid development.
These tools make it easier to create chatbots, search interfaces, and other interactive systems powered by LLMs.
Tool | Description |
---|---|
LangChain | LLM agent framework, Python |
Haystack | LLM agent framework |
LlamaIndex | LLM agent framework |
SanaLabs | AI agents integration with many tools, GUI, Web interface |
LlamaIndex | LLM agent framework |
LlamaHub | LLM agent framework |
My AI | LLM agent framework |
AG UI | Agent-User interaction protocol |