happyborg,
@happyborg@fosstodon.org avatar

Try a local using 's :

  1. Get the 4.3GB file:
    curl -LO https://huggingface.co/jartine/llava-v1.5-7B-GGUF/resolve/main/llava-v1.5-7b-q4-server.llamafile

  2. Make executable

  3. Run it

  4. Visit http://127.0.0.1:8080/

You're welcome!

This is a text and image capable LLM, so ~ 4

h/t @simon

More in Simon's blog: https://simonwillison.net/2023/Nov/29/llamafile/

happyborg,
@happyborg@fosstodon.org avatar

@simon
Has anyone found a use for #Llama #LLM (see above)?

Serious question. It's interesting to play with but I haven't found anything I'd use it for yet. The only thing it seems it may be good at is summarising a news page and articles filtered by subject - though I'm not sure that's really useful to me.

heysupratim,

@happyborg @simon The mental model with decently large models (>7b params ) is you have to treat them as just nicer thought assistants.

Basically an improved focused search and text generator.

Anytime you require generation of some volume of textual data, just use your LLM as a take off point

happyborg,
@happyborg@fosstodon.org avatar

@heysupratim

> Anytime you require generation of some volume of textual data, just use your LLM as a take off point

Can you elaborate. I don't see how that would help me tbh. IME it's valuable to sit with a task until I am ready to write and then it writes itself.

I've never accepted that gmail or other response generators are doing anything other than taking creativity and autonomy away from humans and moulding them into components of a machine which has anti-human goals.

Easy is not good.

heysupratim,

@happyborg yeah those are discussions that i am sure no-one would be able to justify their own sides for ever.

Over time, i have just hardened my skin to just accept the changing dynamics of how we interact with our computer. I totally get though it can be a personal values based decision but try giving it a chance in the next few years, who knows you might like it then

mkhoury,

@happyborg @simon I use it to do passive things like categorizing, topic extraction and summarization on some RSS feeds. Not super useful with 4k context window, but I see some ways to extend it which I haven't played with yet. I also haven't really played with fine tuning.

It's nice that I can run it on my computer. As long as I find passive/async use cases, it's a great tool.

happyborg,
@happyborg@fosstodon.org avatar

@mkhoury
Yes, the one thing I've found so far that might be useful.

I asked it to summarise a news page, visiting the linked articles and summarising them. It seemed to do a reasonable job - also filtering out all the graphics, ads etc as a bonus. Pretty basic stuff though.

In another session when I hadn't asked it to do anything like that it blurted out that it was unable to access the internet and couldn't do what I asked. 🤷‍♂️

Nobody would use or release this crap if it wasn't human like.

  • All
  • Subscribed
  • Moderated
  • Favorites
  • llm
  • DreamBathrooms
  • mdbf
  • ethstaker
  • magazineikmin
  • GTA5RPClips
  • rosin
  • thenastyranch
  • Youngstown
  • osvaldo12
  • slotface
  • khanakhh
  • kavyap
  • InstantRegret
  • Durango
  • provamag3
  • everett
  • cisconetworking
  • Leos
  • normalnudes
  • cubers
  • modclub
  • ngwrru68w68
  • tacticalgear
  • megavids
  • anitta
  • tester
  • JUstTest
  • lostlight
  • All magazines