have been trying this on my own free time. my overall experience, not worth it. unless maybe you have a strong powerful pc.
i'm running off a mini home server that hosts other stuff so my available ram is limited and i'm trying to run it on a system that consumes as low power as possible.
overall i'm able to run only the smallest of models that are usually outdated and replies so slowly that i would not be bothered to even read the final result which is most often inaccurate and out of date.
simply opening icognito and using google's ai is much faster, the results are far more accurate as well. also google's search ai mode is free to use.
it's a fun project but ai today is slow, inaccurate, consumes a ton of power so it is expensive to use and impractical unless maybe you have a powerful enough system and a ton of storage space.