Hi all
I've heard a bit about installing and running an AI model locally. So I googled around and found this guide:
https://itsfoss.com/ollama-setup-linux/
It's nothing fancy but a bit of fun. The model mentioned (llama2) needs to download about 4GB before you can run it.
The output comes out at a good reading speed on my RTX3070. From what I can gather you need a good GFX card to get decent output.
Looks interesting but I don't know if I have a use for it ?
Thanks for the info, I have set it up on my Truenas Scale setup, and have tried it out, came up with some weird stories lol.
Cheers
Quote from: Dingo on May 16, 2025, 06:30:43 PMLooks interesting but I don't know if I have a use for it ?
Yeah, there's only so much you can ask from it! Some people go ga-ga over it. Me? Meh.
Quote from: tazzduke on May 16, 2025, 06:39:15 PMThanks for the info, I have set it up on my Truenas Scale setup, and have tried it out, came up with some weird stories lol.
Cheers
Ha ha yes that's very true. I asked it to produce some C code to print "Hello world!", which it got right.
I then also asked it for some C code to calculate PI to n decimal places. The first program compiled and produced output, but the algorithm was weak. 2nd attempt it compiled, but its output was 0.0!!! Oh dear...
Some more models for ollama
https://ollama.com/library
Interesting keep us updated with how it goes. I am considering trying gemma3 and adding custom data sets, teaining the models for reasoning on specific topics. Was about to retire my Dell-SFF but might make a good AI test rig.