5  Tools

Running and using Generative AI

5.1 Ollama

Open source models can be run locally on your own computer given sufficient compute and storage resources. The easiest tool to run models locally is Ollama. List of models directly supported by Ollama are listed on their website. Resource requirements for models are listed on their GitHub repo.

ollama list
ollama pull llama3.1
ollama show llama3.1
ollama run llama3.1
ollama run llava "What's in this image? /Users/jmorgan/Desktop/smile.png"

Ollama can be used with GUIs as well as integrate programatically with Python and R.

According to Ollama, you should have at least 8 GB of RAM available to run the 7B models, 16 GB to run the 13B models, and 32 GB to run the 33B models.

5.2 Fabric

Fabric allows command-line access to LLM models.

fabric --listvendors
fabric --listmodels
fabric --listpatterns
fabric --setup
fabric --changeDefaultModel llama3.1:8b
fabric --model mistral

fabric -y https://www.youtube.com/watch\?v\=Ed4vBY2IQXg > transcript.txt
cat transcript.txt | fabric -sp summarize
cat transcript.txt | fabric -sp extract_wisdom
pbpaste | fabric -sp extract_wisdom
cat transcript.txt | fabric --model mistral:latest -sp summarize
pbpaste | fabric -sp improve_prompt
fabric -y https://www.youtube.com/watch\?v\=e-gwvmhyU7A -m llama3.1:8b -sp summarize
pbpaste | fabric -m llama3.1:8b -sp summarize
echo "An idea that coding is like speaking with rules." | fabric -sp write_essay

Fabric’s prebuilt prompts (patterns) are listed here. Few of the interesting ones:

  • analyze_claims
  • create_summary
  • create_art_prompt
  • extract_wisdom
  • extract_references
  • extract_ideas
  • extract_main_idea
  • extract_insights
  • improve_prompt
  • improve_writing
  • improve_academic_writing
  • summarize
  • summarize_meeting
  • translate
  • write_essay
fabric --listpatterns
fabric --listpatterns | grep "improve"
fabric --listpatterns | grep "extract"
fabric --listpatterns | grep "summarize"
fabric --listpatterns | grep "write"
fabric --listpatterns | grep "translate"
fabric --listpatterns | grep "create"
fabric --listpatterns | grep "analyze"

Install yt-dlp to allow fabric to download and use youtube videos.

fabric -y https://www.youtube.com/watch?v=zKndCikg3R0 -sp summarize

5.3 R

R packages that are helpful for interacting with LLMs.

Package Interfaces Features
ellmer openai,anthropic,gemini etc query,chat,text->image,vision
openai samterfa openai query,chat,text->image,vision
openai irudnyts openai query,chat,text->image,vision
rollama ollama query,chat,vision
ollamar ollama query,chat
tidychatmodels openai,mistral,ollama query,chat
huggingfaceR huggingface
audio.whisper whisper audio->text

5.4 UI

Graphical user interfaces for LLMs

Tool Description
Msty Remote/local models, chat branching
Anything LLM
OpenWebUI
LM Studio
Poe

5.5 Image

Tools for generating images from text.

Tool Category Description
Microsoft Designer Text to Image Generation and editing using Dall-E
Ideogram Text to Image
Leonardo AI Text to Image Advanced reference and style control
Adobe Firefly Text to Image Reference and style control
Midjourney Text to Image

5.6 Other

Other generative AI tools

Tool Category Description
Perplexity Text to text AI driven search
Zapier Text to text Automate workflows
Google AI Studio Text to text Task helper
Eleven labs Text to speech Text to speech
Runway Text to video Video generation
Udio Text to Speech Audio generation

5.7 Builder

Building LLM Powered Applications

Modern applications leverage specialized frameworks and tools to streamline LLM integration. For example, you can use Python-based libraries like LangChain and Haystack to orchestrate your models, while tools such as LlamaIndex help with efficient information retrieval. Integrated solutions like SanaLabs combine AI agents with intuitive GUIs for rapid development.

These tools make it easier to create chatbots, search interfaces, and other interactive systems powered by LLMs.

Tool Description
LangChain LLM agent framework, Python
Haystack LLM agent framework
LlamaIndex LLM agent framework
SanaLabs AI agents integration with many tools, GUI, Web interface
LlamaIndex LLM agent framework
LlamaHub LLM agent framework
My AI LLM agent framework
AG UI Agent-User interaction protocol