QuotePerformance: ThinkPad X12 with i3 only for simple tasks

I see big words with nothing behind them but some artificial benchmark.
Will it run smoothly chrome + zoom + MS Office? Chrome + cloud IDE? Probably yes because I had L13 @ 14nm i3 and that X12 10nm i3 is just from another universe of performance.

Quote from: thevisi0nary on July 15, 2022, 13:38:44That's the problem, giving this tablet a 75% gaming score objectively makes zero sense when a device that is three times more powerful is given 88%. The numbers either mean something or they don't. I could say that a Hummer gets good gas mileage because it's good enough for me, that doesn't make it true.

This is due to the weighted score. The gaming score the X12 got is actually 59 %. But convertibles are not meant to be gaming systems, and they almost always get low gaming scores. So, the 59 % is recalculated based on the fact that convertibles on average perform badly in the gaming section, which results in the end score of 75 %.

The Flow Z13 is not rated as a convertible, it is rated as a gaming system. If it was rated as a convertible, the gaming score probably would be 100 %.

While that may seem counter intuitive, consider that for a convertible, gaming performance is only weighted as a 4 in our rating, while for a gaming system, it is weighted as a 20. So the 88 % of the Flow Z13 will have a much larger impact on the total score than the 75 % of the ThinkPad X12.

Again, you can find the different weighting here: https://www.notebookcheck.net/Our-Rating-Criteria.16002.0.html

Quote from: Benjamin Herzig on July 15, 2022, 11:18:09The score is meant to be an objective assessment of the quality of any reviewed device. It is not like the score from other review sites that just use it to condense their subjective opinion into a number.

That's the problem, giving this tablet a 75% gaming score objectively makes zero sense when a device that is three times more powerful is given 88%. The numbers either mean something or they don't. I could say that a Hummer gets good gas mileage because it's good enough for me, that doesn't make it true.

In our free world, we can profit from using several review sites:)

I trust most of the raw review data, ignore the overall NBC percentages and use my own decision criteria, of which some are requirements (such as "no notch") and others are thresholds (such as "Cinebench R15 Single >= 93").

A reviewer's (hopefulls correct) subjective points are helpful because one can compare them to one's own checklist and tick any applicable checkboxes.

Of course. That is why I wrote "is meant to be". You can only attempt to be objective, but it is impossible to ever be truly objective.

Personally, I prefer that approach to the "this device is bad, because it doesn't fit with my preferences" way of thinking. Alas, you are free to choose which approach you like better and which review sites you want to trust and which you don't want to trust - also an arbitrary, subjective decision.

Quote from: Benjamin Herzig on July 15, 2022, 11:18:09The score is meant to be an objective assessment of the quality of any reviewed device. [...] A score, which is meant to be an objective measurement of the quality of any given devices, weighted according to the category of the device

Repetition of this argument does not create objectivity. There remain the arbitrary choice which values to measure at all versus which to ignore and the arbitrary weights. Only once those arbitrary decisions are made the subsequent calculation step itself can be objective.

Arbitrary combined_with objective results_in arbitrary.

Quote from: thevisi0nary on July 15, 2022, 04:47:18What is the overall rating supposed to convey about the device?

The score is meant to be an objective assessment of the quality of any reviewed device. It is not like the score from other review sites that just use it to condense their subjective opinion into a number.

In this case, what the score tells you is that the ThinkPad X12 is an overall very good convertible. It reaches 88 % due to the build quality, quality of input devices, heat management, low fan noise and weight categories. It could have an even better score with better performance, which is why the i7 version has a better score.

In the end though, the score is only one component of the review, the text written is another. And while the score does not factor in the pricing or the subjective opinion of the author, the written verdict does.

So that is what our reviews offer: A score, which is meant to be an objective measurement of the quality of any given devices, weighted according to the category of the device (in this case, convertible). And a written verdict, which is a combination of the objective quality of the device mixed with the subjective judgement from the author. The review is only complete if you take both into consideration.

If you are frustrated with our weighted score, you can also look at the unweighted average score. Without the categorization as a convertible, this ThinkPad X12 achieves a score of 75 %. The i7 version of the X12 gets 78 %, and the Surface Pro 8 achieves 81 %. In the convertible category, the weighting diminishes the importance of the performance and increases the importance of things like the system weight.

If you want to know more about our rating: https://www.notebookcheck.net/Our-Rating-Criteria.16002.0.html

Quote from: thevisi0nary on July 15, 2022, 04:47:18I appreciate all the testing you guys do, the raw data is incredibly useful and I'm here reading all the time and I thank you for it. But when I browse these reviews, I have to disregard the scores and just browse through the data because ultimately they don't mean anything useful for the reader.

Welcome to the club:)

Quote from: Benjamin Herzig on July 15, 2022, 00:28:18The rating is mostly the same as the i7 version of the ThinkPad X12, which unsurprisingly is pretty much the same device, just with a different CPU. The rating is already 1 % lower, because of the worse performance and screen brightness. But these rating categories alone won't destroy the rating on their own, especially not on a convertible like this where performance is less important than for, say, a gaming laptop.

Also, pricing plays no role in our rating (because it is volatile and region specific)

What is the overall rating supposed to convey about the device? If the i7 version is the same device with a far faster CPU, how is it only 1% better? What is this score supposed to inform about the devices?

If price is no factor then that makes even less sense. So it's within 12% of the very best Windows tablet at any price point?

I get that it's a weighted average, so then the category averages don't really make sense. This device has a 1280p IPS display with sub 340 nits, 65% P3 coverage, almost definitely 60hz, but it get's a 90% in the display category? The Surface Pro 8 has a 120hz display, 40% higher resolution, 440 nits, higher P3 coverage and has the same score, how does this make sense? How is that useful to the reader?

This thing has a 75% gaming score getting 30 fps at 720p medium, the Flow z13 gets 144 fps on 1080p medium and it has an 88%?? So it's more than three times as fast but only 13% better at gaming?

The scores should be a comparable metric between devices, if two devices of the same type (two laptops or two convertibles, etc) have the same score or are within a few % of each other, the reader is going to assume they are of comparable quality.

Like I said in the last post, I appreciate all the testing you guys do, the raw data is incredibly useful and I'm here reading all the time and I thank you for it. But when I browse these reviews, I have to disregard the scores and just browse through the data because ultimately they don't mean anything useful for the reader.

I am not saying the device is poor, just that this specific processor choice makes no sense.

The rating is mostly the same as the i7 version of the ThinkPad X12, which unsurprisingly is pretty much the same device, just with a different CPU. The rating is already 1 % lower, because of the worse performance and screen brightness. But these rating categories alone won't destroy the rating on their own, especially not on a convertible like this where performance is less important than for, say, a gaming laptop.

Also, pricing plays no role in our rating (because it is volatile and region specific).

I seriously appreciate all the work that you guys do, but your overall rating system really needs to be revamped. At 88% this thing is not even in the same stratosphere as some other laptops given a similar rating. The 2020 M1 air at the same price point is a mere 3% higher, the overall score isn't useful for anyone.

XPS 9520 - 89%
2020 M1 Air - 91%
Surface Pro 8 - 89%
Thinkpad X12 - 88%???

In your own words - "ThinkPad X12 Detachable with i3 makes little sense".

Please figure out an aggregate score that communicates useful info, because a device the reviewer they themselves say is poor scoring within 3% of the above devices I've listed does not tell us anything useful.