LLaVA (Large Language-and-Vision Assistant) was updated to version 1.6 in February. I figured it was time to look at how to use it to describe an image in Node.js. LLaVA 1.6 is an advanced vision-language model created for multi-modal tasks, seamlessly integrating visual and textual data. Last month, we looked at how to use the official Ollama JavaScript Library. We are going to use the same library, today.
Basic CLI Example
Let’s start with a CLI app. For this example, I am using my remote Ollama server but if you don’t have one of those, you will want to install Ollama locally and replace const ollama = new Ollama({ host: 'http://100.74.30.25:11434' }); with const ollama = new Ollama({ host: 'http://localhost:11434' });.
To run it, first run npm i ollama and make sure that you have "type": "module" in your package.json. You can run it from the terminal by running node app.js <image filename>. Let’s take a look at the result.
Its ability to describe an image is pretty awesome.
Basic Web Service
So, what if we wanted to run it as a web service? Running Ollama locally is cool and all but it’s cooler if we can integrate it into an app. If you npm install express to install Express, you can run this as a web service.
The web service takes posts to http://localhost:4040/describe-image with a binary body that contains the image that you are trying to get a description of. It then returns a JSON object containing the description.
It's been almost a decade since I've done a live coding stream. This will be fun!
Today I'll be migrating my website from React to Lit, which is a lightweight framework built around web components. I have the scaffolding set up mostly, so now it's time to get this done.
Come watch. Ask questions in chat! You don't need to create an account, just a username is needed to participate.
What is really missing in this global #SuperElectionYear is a map of active and formerly prominent politicians per country with a Fedi presence (self-updating, preferably.)
I think, though, data is hard to come by (tried this 2 yrs ago for the US and EU for the DeadBirdSite, but stopped.)
@HistoPol@stefan@libraries@parks@lighthouses
Not for individual politicians or parties, but @govdirectory is aiming to get all public organizations (like government agencies and authorities) in the world and their contact data in one place, through Wikidata.
trying to implement a form submission progress bar in js, but XHR follows the success redirect without telling me (I want to access it and redirect the browser).
Fetch can opt out of that but doesn't have a progress api!
I know he didn't explain his position in details, so a 1800-word article sounds a little unfair, but I think dry and sharp statements need adequate context and analysis.
@MaxArt2501@simevidas Sorry, re-reading this and realizing what you're asking. Payloads of that size are very common with React apps. Do they need that much? Of course not. But it would mean rebuilding them without React.
And back to my original point: a lot of that HTML doesn't and shouldn't require JS at all. JS is in fact the problem.
@cferdinandi@MaxArt2501@simevidas I think Massimo is generally making the point that some web sites require JS for core activities, not as a design choice, but out of technical necessity. (My own example: media player). Once you accept that JS is required on a site, you make different decisions about web component design. Designing WCs so there’s a noJS fallback - as you promote, Chris - is a valid choice, it’s just not the only sensible choice.