Quote from: RobertJasiek on December 14, 2023, 16:07:32According to the user A, LLMs need much VRAM or unified memory but I do not know if improved / altered LLM nets might work mostly on RAM, too.
Robert, why do you keep trying to communicate with stupid bot A? Just ignore him, as if he's not on topic and that's it. The best thing you can do.
Complex neural networks simply require HBM3 memory as RAM, but a stupid bot And lies all the time about the need for VRAM. Neural networks don't need VRAM, they need the fastest memory available on the market. But the RAM in Apple hardware, although 1.5-2 times faster than in the x86 camp, is still inferior to the dedicated VRAM in video cards by up to 10 times. And of course, HBM3, which is used on serious neural networks in servers, loses up to 10 times.
Actually, there is only one problem - there is very little memory (terabytes, hundreds of terabytes are needed) and speeds of terabytes per second, which only HBM3 can provide. Obviously, neither Apple nor consumer versions of x86 are in any way suitable for serious neural networks and will not be very suitable yet for a long time due to slowdowns in both productivity and technical processes.
Once upon a time in the 90s, a computer with a performance of 1 Teraflops (with 64-bit precision) was considered the height of progress, but now many ordinary people have it. This is what led to the overall progress and development in software as we see it now. But a further leap (towards smart expert systems) requires hardware thousands of times more powerful than what is currently available to the average person. Everything is very simple. As before, advanced technologies consume energy (forced) like a whole power plant and cost a lot of money, which is what NVidia and AMD are now making money on, but someday it will be in a pocket gadget if the IT world finds a way out of the silicon impasse.
Samsung recently promised simultaneous speech translation in 2 languages using a local neural network in the S24. I don't believe in this and I'm sure that Samsung will screw up for a simple reason - the local resources of top-end smartphones are completely insufficient for accurate translation from one language to another, especially on free topics. If this were possible with low consumption and power of hardware, the Google translator would not produce such wild nonsense in elementary sentences.
I'm afraid that Samsung will soon quarrel with many people who have entrusted their conversations to such a smartphone. Well, it's kind of like how idiots who believed in a working "autopilot" die. I turn out to be right everywhere, later, years later, seeing everything in advance, when the crowd finally realizes how complicated it all is and how modern technologies are not ready for it.
Hundreds of billions of dollars were poured into autopilots (or rather, businessmen from startups sawed it up), and the result was a complete failure. There will also be simultaneous speech interpreters in the next 20-30 years. People, of course, will use it (they are greedy for new products without going into too much detail), but along with mistakes and quarrels, there will also be a wave of disappointment. But Samsung will be able to temporarily stand out from its competitors and sell more. Goal achieved...