nBench and webOSmark benchmarking: Which Kernel is the fastest? Comparison

[ghostinator]

This test involves all Kernels available in the public and ALPHA and BETA testing Feeds for the HP Touchpad.
How to get these kernels you must go to Testing Feeds - WebOS Internals and read EVERYTHING

Be sure to take care of the team who makes webOS how great it truly is. DONATE!!!!!
http://donate.webos-internals.org/


*** Latest Kernel Builds being benchmarked now but this time with lithium benchmark found in the app catalog. ***
I will bench a few kernels at select speeds with lithium, both latest versions and the ones i used previously to compare any version specific speed improvements

Every Speed was tested with the governor set to performance.

I literally had to put my Touchpad in a ziploc bag and in the freezer for over an hour to get the temp low enough to run stable long enough to benchmark.



***The Lower the score the better performance ****


F4 Phantom Kernel Takes the Lead for 1.8GHz Performance.


Warthog Takes the Lead at 1.7GHz


Warthog Takes a considerable lead @ 1.512GHz! I reran all kernels and benched each one over 20 times each and averaged the results since there was such a drastic difference.


at 1.118GHz, No one brings it home like the Palm Default Kernel.

I also wanted the performance changes (if any) of the original F15C I used in the benchmarks below to the latest Alpha release.

Versions benched:
3.0.2-42
Latest at time of testing: 3.0.2-61
So roughly different builds in between.


Looks like uNiXpSyChO knows how to optimize quite well!

Props to Webos-Internals for creating all this kernel goodness and ScienceApps developer for making Lithium ENYO Benchmark that is found in the Appstore. For more information about his application visit http://forums.precentral.net/hp-touc...-touchpad.html




Older nBench and webOSMark Benchmarks Below

Exact Kernel Versions used in benchmarking:
Palm Kernel (3.0.2-1)
UberKernel (3.0.2-36)
WarthogKernel (3.0.2-42)
F15C (3.0.2-42)
F4 Phantom (3.0.2-46)

The Process: Freshly doctored TP with webOS 3.0.2 installed. Preware and govnah installed. Installed the kernel, rebooted, set speed to max supported by kernel on performance governor. run nbench 5 times and averaged score. Rebooted the TP, verified clock speed and governor settings, then ran webOSMark 5 times and averaged results. Between testing each kernel, the current one was uninstalled, palm default installed over it rebooted, new kernel installed, then rebooted again. I didn't do battery life or any of that because it's irrelevant to me. I was curious about the performance in numbers, so i did the tests made some graphs and decided to share. Enjoy!






Obviously the F15C and the F4 are pretty much tied since they run at the same clock speed. Also, If you install F15C and F4 kernels and notice that Govnah isn't showing your second CPU active don't go harass uNiXpSyChO! a) the second cpu is turned on and off as needed alot quicker on his kernels compared to others and govnah doesn't always catch it. b) powerd might have turned it off it deemed necessary.

As soon as i find a good stable voltage to run my touchpad at 1.944GHz, i will update these with results. Until then my kernel of choice is F15C @ 1.836 GHz and settings are as follows in govnah
governor: Ondemand
min freq: 192mhz
max freq: 1.836GHz
sampling rate: 0.01 Sec
up threshold: 65%
ignore nice load: false
powersave bias: 0

I/O Scheduler: NOOP
TCP Congestion: VENO


Sources for Tests:
webOSMark: Load this link directly into your touchpad's browser and it'll make the app catalog appear with an option to download it
http://developer.palm.com/appredirec...oder.webosmark

nBench: Installed optware from Preware, went into terminal used command "ipkg-opt install nbench-byte" then ran nbench



Update: Each Kernel ran at the same speed and governor settings to show any performance differences between them. Only nBench was ran and not webOSMark/.

[drfp]

Doesn't over clocking Kill your battery life?

[ghostinator]

You can't go fast if your trying to save gas...

[Abyssul]

Originally Posted by drfp

Doesn't over clocking Kill your battery life?

The amount difference is negligible. Most battery use is from the data connection and graphics chip. CPU is 3rd on that list of the many things that consume the battery (speakers, RAM, flash, screen, LEDs, bluetooth).

These graphs are pretty interesting.

[jbennett]

Nice work, thanks. If you have the time, it would be interesting to see a comparison of the different kernels at the same clock speed - like the four "homebrew" kernels all running at 1.5 GHz, or all five kernels at 1.2 GHz. This would give a clearer indication of how much better optimized each kernel is for performance, regardless of clock speed.

--John

[ghostinator]

The results were neglible if i remember correctly. then again i only played with 1.5ghz, maybe 1.118ghz will be different, who knows. I'll work on it for ya though

[kkhanmd]

how does a overclocked TP compare to IP-2 benchmarks?

[indivisibletears]

nice! Hopefully this will answer a lot of repeat questions.

Were there ever any concerns with temperature???

[ghostinator]

Only time i've ever seen it get too hot was after i left it sitting my passenger seat during a short trip here in FL in the summer, it did a luna reboot. I never see it get hot at all really, not compared to my pre - with the sr71 kernel @ 1.2ghz atleast. Maybe things will get funky when i up the voltage on it some more to try to tackle the 1.9ghz.

I have no way of comparing these numbers to an ipad. i know webOSmark is a definite no, but i'll look into nbench

[artitumis]

I really wonder how much we can push the TouchPad and keep it stable. Good work onthe benches and for giving me a new Govnah profile to play with.

[GeekOutDog]

What governor settings were used for each of these?

[beardedspoooon]

Originally Posted by GeekOutDog

What governor settings were used for each of these?

This.

[rwhitby]

Originally Posted by ghostinator

That's pretty much what we expect from our WebOS Internals kernels. The experimental ones in which we're pushing the performance limits are the fastest, and the stable public release is around the point where we have zero reports of crashes or random reboots.

So our experimental->stable development process is producing results as we expect it to

-- Rod

[GeekOutDog]

How are you drawing any conclusion about stability and random reboots from looking at a bunch of charts with benchmarks that all completed successfully?

Not to knock any of the hard work developing these kernels... it's just that I think that you've found a really strange way of interpreting the charts you just quoted.

[docnas]

Have you tried the lithium benchmark app? I tried the beta version on my TP and it seems to be giving good results.

» Lithium BenchMark MOJO & ENYO Version ScienceApps | HP webOS

and also about the 2nd cpu i havent seen it run using any of the benchmarking software. During regular use i can see the log of cpu2 going up to 1.7 but not on running benchmarks. Not sure if they were optimized for dual core cpu's as i think this is the first dualcore CPU using WebOS.

[docnas]

The charts will give you an idea about what kind of performance gains you can get using the different kernels. Now as for stability thats up to u and your TP , some TPs run stable at 1.9 some cant pass 1.7 so you have to find your own balance of pushing it /keeping it stable.

[morydees]

Originally Posted by GeekOutDog

How are you drawing any conclusion about stability and random reboots from looking at a bunch of charts with benchmarks that all completed successfully.

Actually he's not. Re-read what he said carefully. He didn't draw any conclusions from the graph directly, instead saying that was what they "expected to see" given the nature of "pushing limits". It may have been stating the obvious (with a small dose of self-promotion) but there's nothing wrong with that!

As for his second statement about stability, he used his own evidence by saying "we have zero reports of crashes or random reboots". In other words, interpreting the chart wasn't used to draw those conclusions, it was the overall great feedback from the community

[ghostinator]

governor was set to performance. Since stability varies per TP didn't include it. I'm working on testing each kernel at the same frequencies to see any other performance differences, so far very inconclusive.

[geekpeter]

find it a tad strange seeing benchmarks for differing kernels with different speeds, wouldn't it be best to have them all set at the same speeds to se if there's any difference at all.

comparing a 1.2ghz stock to a 1.8ghz anything would ofc show that the stock sucks, same for a 1.5ghz {insert any kernel name here} etc.

slap them all on the stocks ghz level and try again perhaps?

[rwhitby]

Originally Posted by GeekOutDog

How are you drawing any conclusion about stability and random reboots from looking at a bunch of charts with benchmarks that all completed successfully?

Not to knock any of the hard work developing these kernels... it's just that I think that you've found a really strange way of interpreting the charts you just quoted.

I drew no conclusions from the charts other than the benchmark results shown on the charts, and neither should you.

Stability and random reboot reports happen in the experimental kernel threads and the support email address which lands in my inbox. It's also part of our development process - we push the boundaries to the limit in the testing feeds and release what has no problems in the stable feed. Since I'm in charge of that decision point, you may assume I'm using all the information available to me, not just four benchmark performance results.

-- Rod