Install a local AI model on Linux

[jave808]

Hi all

I've heard a bit about installing and running an AI model locally. So I googled around and found this guide:

https://itsfoss.com/ollama-setup-linux/

It's nothing fancy but a bit of fun. The model mentioned (llama2) needs to download about 4GB before you can run it.

The output comes out at a good reading speed on my RTX3070. From what I can gather you need a good GFX card to get decent output.

[Dingo]

Looks interesting but I don't know if I have a use for it ?

[tazzduke]

Thanks for the info, I have set it up on my Truenas Scale setup, and have tried it out, came up with some weird stories lol.

Cheers

[jave808]

Looks interesting but I don't know if I have a use for it ?

Yeah, there's only so much you can ask from it! Some people go ga-ga over it. Me? Meh.

[jave808]

Thanks for the info, I have set it up on my Truenas Scale setup, and have tried it out, came up with some weird stories lol.

Cheers

Ha ha yes that's very true. I asked it to produce some C code to print "Hello world!", which it got right.

I then also asked it for some C code to calculate PI to n decimal places. The first program compiled and produced output, but the algorithm was weak. 2nd attempt it compiled, but its output was 0.0!!! Oh dear...

[jave808]

Some more models for ollama

https://ollama.com/library

[turbo0]

Interesting keep us updated with how it goes. I am considering trying gemma3 and adding custom data sets, teaining the models for reasoning on specific topics. Was about to retire my Dell-SFF but might make a good AI test rig.