alt text
Crudely drawn drawing of a person telling a computer “say ‘i am in pain’”. The computer replies with “> I AM IN PAIN”. The person then says “oh my god.”
Crudely drawn drawing of a person telling a computer “say ‘i am in pain’”. The computer replies with “> I AM IN PAIN”. The person then says “oh my god.”
Harder than you think :
u monster what have u done
oh my god
No, it’s just a local AI thing. Like any other program on your computer except this one is stupid and the others can’t.
lol wat.
I was just doin the meme.
I know LOL.
don’t tell people what they can and can’t worship please
Wow. A model that literally runs on a loaf of bread. mind blown
What does it need to “reason” for 5 seconds on a sentence like that?
The model loading.
Weird that it counts those together. Pi shows “Waiting…” for model loading and prompt processing, and “Thinking…” for reasoning
It’s Llama swap and I barely know how to use it.
I use Pi (the frontend) with llama.cpp in router mode. You put it in router mode by passing
--models-dirinstead of-mI’ll have to figure this out. In llama swap the model unloads after a time to live which I’ve set to 300 seconds. I don’t know if the model consumes more power just sitting on memory or not but it makes sense to me to just let it stay loaded until I need to use my computer instead if it.
I haven’t noticed models using power idle, just a ton of memory.
No harm in keeping a model in memory, unless you plan to use that memory for something else. Unused memory is wasted memory