The impact from smaller opensource LLMs like Llama3-8B and Phi-3 could be large. They are not necessarily the best and smartest models but can be easily integrated in software on every device and platform. Also they can be finetuned, improved with RAG to function better for specific tasks and in specific contexts. Exciting times ahead. #opensource#LLM#AI#Llama#Phi
So #Steeve got a major upgrade recently. He moved from a #gptneo (2.4B) model to a #llama2 (7B) model. Trained on 300k messages from our private chat history, Steeve is way more capable of following the conversation now. He used to have some "favorite phrases" he would say a lot, and I'm seeing less of that. His vision and reading models also got upgraded, so he gets more detail about the links and memes we share. Long live Steeve! :steeve:
⚠️ @forrestbrazeal on the inside threat to OSS
🍴Vicki Boykis says Redis is forked
👻 @johnonolan says Ghost is federating
🦙 Meta Engineering announces Llama 3
❓ @eieio's questions to ask when you don't want to work
🎙 hosted by @jerod
(1/3) Last Friday, I was planning to watch Masters of the Air ✈️, but my ADHD had different plans 🙃, and I ended up running a short POC and creating a tutorial for getting started with Ollama Python 🚀. The settings are available for both Docker 🐳 and locally.
TLDR: It is straightforward to run LLM models locally with the Ollama Python library. Models with up to ~7B parameters run smoothly with low compute resources.
(3/3) The tutorial will get you to run Ollama inside a dockerized container. Yet, there are some missing pieces, such as mounting LLM models from the local environment to avoid downloading the models during the build time. I plan to explore this topic sometime in the coming weeks.
(2/3) The tutorial focuses on the following topics:
✅ Setting up Ollama server 🦙
✅ Setting up Python environment 🐍
✅ Pulling and running LLM (examples of Mistral, Llama2, and Vicuna)
FYI - #llama is NOT#opensource. The license is categorically not open source. Among other things, the llama 2 and 3 licenses explicitly violate Field of Endeavor.
I see all sorts of blogs and marketing materials claiming things are "open source" because they used llama somewhere. Please do not take these claims at face value.
The Code Llama 34b model isn't half bad! Been toying around with it integrated into clion having it explain my own code to me and generate small functions and it's been so far around 90% successful, with most of the errors being minor, the bug detection does have a decent amount of false positives though. I also like that it's aware enough of api's to give doc links
Bonus points for it going off on a tangent once on why console applications are better than gui.
Anyone out there dabbling with on-prem AI? All of the numbers that I’m seeing for RAM requirements on 7b, 13b, 70b models seem to be correct for a 1-user scenario but I’m curious what folks are seeing for 2, 10, or 50 users.
Meta released today Llama 3, the next generation of the Llama model. LLama 3 is a state-of-the-art open-source large language model. Here are some of the key features of the model: 🧵👇🏼