In #homeassistant, using #nodered to make an API call to a #llamacpp server running #mistral 7B model. I create a prompt that asks it to summarize all the data in my house from the sensors. The results are pretty impressive for such a little model. Now I get a customized rundown, Jarvis style.
Useful? Probably not. But cool as hell. :cool_skelly:
Note that I send human readable descriptions of 150+ sensors in the house. Items like the following:
"The Kitchen light is on."
"The weather right now is cloudy. The temperature is 55F and the humidity is 38%"
That is a lot of information that little model is sorting through and doing a fairly good job at identifying likely scenarios given the time, weather, and contact sensor data. Shows a fairly good ability for the model to form a "mental model" of the house on its own.
@wagesj45 This is really neat! I like that you could hook it up to tts and ask assist for a summary and have it read it out too 😁
Have you considered sending more context for things like 'The kitchen light has been on for 4 hours', or 'The kitchen temperature is x degrees, a change of +-y degrees in the last hour'? I wonder if it could infer trends given enough information.
@PossiblyMax Good idea! I think grabbing update times would help quite a bit. You'd have to weight that with the idea of adding more tokens to your prompt. I have enough context (32k) but I'm running on CPU. With my 190 sensors, the whole thing takes about 4 minutes to run. Plenty when you're running it on a schedule, but would really slow down an assistant interaction. I have the information in the JSON I get from HA, I'd just have to take the time to write out the printing logic.
@PossiblyMax Yup, just through in a string of something like "(Last changed X days, Y minutes, Z seconds ago.)" and it upped the calculation time about 8.5 minutes. I'm sure I'll keep pidlin' to refine the prompt over time, but this looks like a worst case scenario.
Add comment