Your own ChatGPT, offline: AI without the cloud on your laptop

[Redaktion]

An AI chatbot doesn't have to run in the cloud. With LM Studio or Ollama, you can run it right on your laptop, without an internet connection and without your data ever leaving your device. What your hardware needs to be capable of, which model is right for you, and how to get started in ten minutes.

https://www.notebookcheck.net/Your-own-ChatGPT-offline-AI-without-the-cloud-on-your-laptop.1341872.0.html

[hehehaha]

have been trying this on my own free time. my overall experience, not worth it. unless maybe you have a strong powerful pc.
i'm running off a mini home server that hosts other stuff so my available ram is limited and i'm trying to run it on a system that consumes as low power as possible.
overall i'm able to run only the smallest of models that are usually outdated and replies so slowly that i would not be bothered to even read the final result which is most often inaccurate and out of date.
simply opening icognito and using google's ai is much faster, the results are far more accurate as well. also google's search ai mode is free to use.

it's a fun project but ai today is slow, inaccurate, consumes a ton of power so it is expensive to use and impractical unless maybe you have a powerful enough system and a ton of storage space.

[Vertigo1055]

I've been using the LM Studio Program as well as ComfyUI using Local Only Models. Up front I use a 13900K/64GB Sys/2TB Working Drive and 2TB DATA Drive. These 2 Drives are separate from the C: SYS Drive and Isolated in order that they have specific permission to access only DLL and SYS Files needed. The rest (the workhorse) is the GPU, a RTX 4090. Let's talk POWER. For context my avg. Power Bill was around $200 as a Gamer that plays 8 to 12 hours a day every day. I decided to install the new 5090 I have for a "Burn In" Test. With either Folding@Home or LM Studio the amount of power used went over double. But, I used it for a solid month. For what I do, Batch Texture Processing once a Material is created and some Pictures of my Chihuahua in various Meme style Photos, the costs outweighed the usefulness. With RTX Chat I could go through my own files / DATA and with LM Studio and ComfyUI I could do the rest just fine on the 4090. The cost on the 4090 was about a +50% increase compared to Gaming. BUT, this was only a 30 Day run on each card with no manual Optimization.