joe, (edited ) Yesterday, we played with Llama 3 using the Ollama CLI client (or REPL). Today I figured that we would play with it using the Ollama API. The Ollama API is documented on their Github repo. Ollama has a client that runs when you run
ollama run llama3
and a service that can be accessed from something like MindMac, Amallo, or Enchanted. The service is what starts when you runollama serve
.In our first Llama 3 post, we asked the model for “a comma-delimited list of cities in Wisconsin with a population over 100,000 people”. Using Postman and the completion API endpoint, you can ask the same thing.
You will notice the
stream
parameter is set tofalse
in the body. If the value is false, the response will be returned as a single response object, rather than a stream of objects. If you are using the API with a web application, you will want to ask the model for the answer as JSON and you will probably want to provide an example of how you want the answer formatted.You can use Node and Node-fetch to do the same thing.
If you run it from the terminal, it will look like this:
Have any questions, comments, etc? Please feel free to drop a comment, below.
https://jws.news/2024/lets-play-more-with-llama-3/
#AI #Amallo #Enchanted #llama3 #LLM #MindMac #NodeJs #Ollama #Postman