Monday, December 15, 2014

MotoG : Pure CPU Benchmarks, the limits

To point out pure CPU Benchmark flaws, we'll observe how does a moto G (Falcon or Peregrine, this has no relevance) behaves with Linpack Benchmark when running on Stock 4.4.4 Motorola ROM or CM11-20141124-nightly (4.4.4).

CyanogenMod Logo

We have seen previously that Linpack didn't use all cores and was limited to 3 threads in multi-thread benchmarking (when comparing s200 and s400 SoC).

Now, we will compare MFLOPS performance between two ROMs that has similar user experience (smoothness mostly)

Stock 4.4.4 Motorola ROM vs CM11
both @1190MHz, Interactive governor.

Stock Motorola features a nearly doubled score for both Single and Multi-Thread, but does that mean it is really better optimized?

  • In a way, yes, CM11 is made to be compatible with most devices, so we encounter similar problem seen with Windows against iOS : 
    • An OS that fits all against a fully optimized one made for a few specific hardware. Motorola obviously optimized their software to get the best of this great device
    • If you compare with the MotoG vs MotoE test, you'll see that the MotoG running CM11 has lower results than MotoE running stock ROM ...
  • On the other hand, that's not all a matter of optimization :
    • CM11 features KSM, zRAM, and many things that use CPU cycles to compress memory (zRAM) or merging merge-able things (KSM) ==> this reduces CPU pure performance but increase multi-tasking ability.
    • The war of versatility and customization against slickness and stability...
    • Honestly i can't see difference in user-felt performances...

Actually, i don't see any issues by using CM11 as a daily driver on my XT1032 (Falcon), it is not because a Benchmark say something that it is true in "real life"...

  • Don't use CM11 on Moto G if you want to show off with high CPU benchmark results ...
  • Stock Motorola ROM is close enough to AOSP to be acceptable as daily driver
  • Xposed helps you to get the best of stock ROM
  • CM11 is great for end-user customization

The real conclusion is the following :

Don't trust Benchmarks as incontestable truth, they helps to compare devices and  ROMs, but never really tell you how good devices/ROMs are.