Ok. I haven’t tried it so I’ll take your word for it. I’m just offering an easier alternative since the topic was “getting started”

Iirc it can run anything llama.cpp can because it just uses that under the hood.

There’s also llamafile, super simple: download and run it.

Markdown is not respected in post preview (can’t attach an image apparently, but many posts on news communities will have a quote)