Psst, NikoB, check out this disgraceful M1 Ultra running shameful 65B parameter language model on puny 64Gb RAM at around idiotic 50% CPU load.

youtube.com/watch?v=ysA7xg6nevY

As a reminder, M3 Max is 1.5x more powerful than M1 Ultra.

Quote from: NikoB on November 24, 2023, 18:47:56For any model to work, it must have access to huge repositories of information that it analyzes.

Hahaha, no, dummy. You need to do your homework again. 65 billion parameters language model requires only 32Gb RAM and 280Gb on your SSD to run locally, you can run it right now right here on your own pc, google llama-65B. GPT3 is 175 billion parameters but it's also much better optimized for RAM usage and parallelization.

There are no models today that run outside of capabilities of nVidia GPUs, latest of which had 94Gb RAM in GPT3 days.

Quote from: NikoB on November 24, 2023, 18:47:56No modern PC, much less a measly smartphone, is capable of performing GPT3 on the entire volume of data collected.

Bs

Again, a stupid troll and an amateur in IT got into a topic about which he understands nothing.

For any model to work, it must have access to huge repositories of information that it analyzes.

First of all, this is a limitation on ram and disk system, if there is not enough ram.

No modern PC, much less a measly smartphone, is capable of performing GPT3 on the entire volume of data collected. There is simply nowhere to place them for operational analysis of the request.

Quote from: NikoB on November 24, 2023, 13:23:06Robert, GPT, even version 3.0, requires hundreds of terabytes of RAM and giant storage facilities for analyzing information.

Bs.
You could easily run GPT4 inferences locally, if anyone gave you a model. It's not even too consuming. Data centers are only needed because there are millions simultaneous requests worldwide. Inside each server there are about 8 quite ordinary A100/H100 cards with 80/94Gb RAM, each of those capable of running a hundred to thousand simultaneous different requests aka inferences in parallel. Each inference is about 1 to 10Wh, depending on complexity.

Llama models are freely available though so you can play with language models locally anytime lol, without HUNDREDS OF TERABYTES bs.

Quote from: RobertJasiek on November 23, 2023, 18:40:43Specialised neural nets are used today on local computers but maybe NikoB's "20 years" are needed for things like ChatGPT to run well locally.

Robert, GPT, even version 3.0, requires hundreds of terabytes of RAM and giant storage facilities for analyzing information. I doubt that even in 20 years it will be possible to do this on a smartphone, given the obvious slowdown in performance growth per 1W and the problem of capacity scaling for the same technological reasons.

That is why I understand well that a minimally valuable local neural network is not even a matter of the next 30 years. Until the necessary technological breakthroughs accumulate.

As was the case with cell phones, the Internet and everything else, you need a fundamental and technological base. It accumulates gradually. and when all the input components for the success of a new technology can already be available on the market in industrial quantities, there is an explosive growth of a new technology sublimating all these components and then a phase shift in civilization. Which will seem incredible to the predecessors of this event.

Any sufficiently advanced technology is indistinguishable from magic (c). Arthur Clarke.

The time of developed neural networks accessible to the individual and making his life completely different from the current one in terms of capabilities (that same phase shift) has not yet come and is very far away. This is approximately the same level of task as permanent colonies on Mars at the present stage of development of civilization, if not more difficult.

And getting to the stars at the current stage seems like pure fantasy, especially within the framework of one human life, as we now imagine it. Although remember what happened in the 50-60s, what euphoria there was with space and what happened in the end when humanity realized how primitive its technologies were in reality for such a super-task.

Each next step becomes more and more difficult - because...requires more and more scientific research, skilled labor, and more and more resources, as well as risks for civilization in case of fatal mistakes. And humanity is mired in the 21st century in total corruption, the growing real stupidity of the population and showdowns with each other over food and a place in the sun.

Specialised neural nets are used today on local computers but maybe NikoB's "20 years" are needed for things like ChatGPT to run well locally.

Quote from: ArsLoginName on November 23, 2023, 13:36:00No gains from TSMC 3 nm either.

Improvements are a balance of performance/consumption/more modules on chip. Apple balanced 3nm improvements to be something like "more performance and better GPU at around the same power consumption".

Quote from: NikoB on November 23, 2023, 16:57:32And the fact that Qualcomm (like Apple with the M3 line) was forced to back down with some real downgrade

Bs

I am more concerned about the introduction of ever greater totalitarian restrictions on the Android platform than about Apple's problems.

And the fact that Qualcomm (like Apple with the M3 line) was forced to back down with some real downgrade, in view of the negative trends that I described above and wrote earlier, is quite obvious. It will only get worse and worse in terms of what developers of technical processes and chips based on them can offer to the end consumer in terms of performance per 1W and general quality improvements.

Actually, no one needs a special increase in performance in terms of actually used tasks, and as I wrote earlier, for really useful expert systems based on neural systems (which are passed off to fools as "AI"), performance should increase by several orders of magnitude at once, plus the ram and long-term storage in a smartphone should grow just as much - an adequate person will not rely on the use of "cloud AI" if he is smart and understands the risks. And for local neural networks of this level of power, the hardware is simply not ready and will be ready for at least another 20 years, and then only with the favorable development of civilization in these years, for which there is no longer any hope.. Obviously, in the next 20 years this is physically unattainable, which means the IT industry will become self-sufficient in its desire to develop horizontally (rather than vertically by real progress), generally increasingly selling "rhinestones" rather than real new function levels. We have gone through this more than once outside of IT, when the technological reserves are running out, but still need to sell a lot...

Quote from: ArsLoginName on November 23, 2023, 13:36:00No gains from TSMC 3 nm either. Android based phones with Qualcomm chips on the same TSMC process node (4 nm) need 5000-6000 mAhr batteries in order to claim 'longest battery life' all while iPhones are < 4000 mAhr.

Firstly, this is not the same technical process as Apple, which is always one step ahead thanks to its position. And secondly, she is the owner of an ecosystem (by the way, with a bunch of moronic totalitarian restrictions due to which I have been ignoring them for 13 years), where she can carry out almost perfect optimization, unlike the combination of Qualcomm on the worst technical process + Android from Google + shells and curves tuning from a specific smartphone manufacturer.

When a competitor to Apple appears with exactly the same access to TSMC, and with its own entire ecosystem, which even Google does not have, then we'll see.

Android is ten times more flexible and free than iOS, but this is its drawback in terms of the quality of optimization of the platform as a whole.

Great news in my book. Hopefully, when these release it'll cause the prices of older SG8G2 phones to drop further as no longer in as much demand (even though it's still a very decent SoC) and everyone going for the latest and greatest.

Quote from: A on November 22, 2023, 21:17:59The only metric that matters is battery life of the device. All these 'tests' are just for hype.

Battery life and heat when being used because both keep costs down. It costs less and simpler to not need a vapor chamber at all. Less weight for consumer to carry around too. I'd imagine a Dimensity 9200 based phone has a lower BOM to only cool 2.65 W than a SD8G3 with 6.25 W. 70% of the performance at 42% of the max power.  A Prius will never beat an Accord or Camry hybrid in a 0-60 mph (0-100 kmph) drag race; but its ultimate goal was to be the most efficient while still being 'mostly as fast.

Additionally if you look at Tom's Hardware cell phone battery life test, you will see Apple phones having the most battery life per Whr/mAhr of battery capacity. No gains from TSMC 3 nm either. Android based phones with Qualcomm chips on the same TSMC process node (4 nm) need 5000-6000 mAhr batteries in order to claim 'longest battery life' all while iPhones are < 4000 mAhr.

Too bad Samsung won't make a Dimensity 9200/8100 based phone in the US. Or how about a SD 865 (not 865+) based mid/entry level on 4/5 nm vs 7 nm. Dimensity 8100/SD 865 on 5 nm give 48.3-50% of the performance of leading edge all while consuming 1.36 W max (22%) of the maximum power.

Quote from: A on November 22, 2023, 21:17:59The only metric that matters is battery life of the device.

This 👍

The only metric that matters is battery life of the device. All these 'tests' are just for hype.

Wow, d9300 in trouble...

What's surprising about this? I wrote many times before that TSMC itself warned about the onset of problems. Well, Qualcomm was forced (as its client) to cheat with consumption for the sake of improved results. If you leave it the way it is, everything will be much worse. Performance growth per 1W is slowing down quickly even with Arm, which has been the engine of progress in the smartphone market for the last 15 years.

But they are selling performance to the crowd, trying to hide the loss of energy efficiency, as Intel has successfully done for the last 8 years.

Of course, Intel was much simpler - on desktops especially, in laptops it is already miserably losing to AMD/Apple (but its cores are still the most energy efficient, it's just that their factories are far behind TSMC/Samsung in terms of technical processes) and for a long time, but in smartphones this cheating with increased consumption for the sake of a visible increase in performance will quickly be limited by the battery capacity and the quickly falling operating time of the smartphone or clearly noticeable performance drawdowns for those consumers who need it in practice. The majority simply will not notice that they are simply being deceived - in order to maintain battery life, the new Qualcomm chips will quietly sharply reduce performance in normal work, with a much smaller gap than the old ones.

But the technically illiterate majority of buyers, who generally will not notice the catch, will still buy into the marketing. And this is precisely what is important for smartphone manufacturers. Although sales of smartphones are also falling across the planet, people are less and less understanding why they need to change smartphones so often. There is practically no difference with models of 3-5 years ago at the level of user comfort and usability. Rather, smartphones have been degrading in capabilities compared to older ones for several years now, and prices, on the contrary, are rising like crazy. It all ends with the reluctance of consumers to change smartphones - and a political order to their politicians to force manufacturers to support old models for at least 5-10 years.