Results 1 to 5 of 5
  1.    #1  
    The m500 benchmarks with benchmark 2.0 as only 160% or so.

    So does the m505.

    The visor platinum clocks in at 204%

    And when you crank up a visor platinum to 45 mhz via afterburner, you get 325%!!!

    With my old palm iiixe, cranked up to 45 mhz, (it was stable) I got 260%

    What gives?

    Benchmark author says his program is compatible with os 4.0
  2. #2  
    Never trust benchmark programs.

    I'm speaking in general, not about this case specifically. Unless you analyze what tests are being performed and how each test is weighted into the overall result, all that you have is a number.

    For instance even though I have seen the numbers that show how much faster my Prism is, I believe that certain operations on my M505 are much faster. I have seen this in games such as Zap 2016. On my M505, Zap plays much more smoothly and responsively. I thought this was true the first time that I played it on my M505, but wasn't sure. So I went back to my Prism and tried it there. Much less smooth and much less responsive. This tells me that Benchmark 2 is not telling the whole story.

    If everything in the different devices had identical relative performance, or if the benchmark tested every component of the system and weighted it's results to model the application that you are interested in, then the benchmark would have real meaning.

    I do know that all of the different benchmark programs give different results. to prove this perform a simple test.

    Run Benchmark 2.0.
    Run QuickBits and enable all of the speedups - note the claimed speed increase.
    Run Benchmark 2.0 again and see if you get the same percentage of speedup.

    If you believe this post, you won't be too surprised by the results.
  3. #3  
    According to the Benchmark docs, it does a "memory-intensive Fibonacci computation" to test CPU and bus bandwidth. I'd assume the CPU would be equivalent to all the other 33MHz Dragonballs, so that leaves the m505 lacking in bus speed (according to Benchmark). But then again I would think a game like Zap does a lot of memory accesses. Changes in OS4.0?
  4. #4  
    That's the problem. A "memory-intensive Fibonacci computation" is a meaningless statement by itself. How is it implemented, and why is that the only test performed. They aren't doing specific memory tests, floating point tests, integer tests, graphics tests in each color depth, etc.

    By only performing a single test, all that they are doing is demonstrating how PDAs do on that test. Now admittedly this is better than nothing, but not much. Go look at the type of benchmark suites that are used to test PC performance. They perform a multitude of tests that exercise specific subsystems and explain what they mean to real world applications. I'm not saying that the good folks at Quartus are even interested in going to this much trouble. I'm just trying to get the point across that you can not trust benchmarks.

    Gather around children, it's story time. A few years ago I put in my time at a major semiconductor company that at the time (but not now), was a major player in the PC graphics market. In the graphics organization was a group of engineers whose sole job was to analyze benchmark programs that were used by various magazines. Once the test was understood, these people would do whatever they could to subvert them. The benchmark score was king. The changes made had very little to do with real-world performance, and in some cases actually slowed down the product for real-world use. BTW, talking to friends who are still in the graphics industry, this is still common.

    According to Benchmark, my Visor Deluxe has a score of 141. A Palm m105 has a score of 100. If I use Cruise Control, the m105 score reaches 149. First of all, the only thing that makes a Visor faster than the m105 out of the box is removing the wait states for reading and writing memory. This is also the change that is made by Cruise Control. Removing wait states does not make the device half again as fast. If all that you are doing is accessing memory, as in a "memory-intensive Fibonacci computation", you might see this kind of speedup. If what you are doing is non-memory intensive calculations or accessing the OS to draw the screen (ROM accesses are verrry slow), you aren't going to see anywhere near this kind of performance increase.

    If there were other changes in Palm OS4 to make screen output faster, which is what I am observing on my m505, this benchmark won't reflect it because it is only testing one specific attribute of the system performance.
  5. Rob
    Rob is offline
    Rob's Avatar
    531 Posts
    Global Posts
    533 Global Posts
    Originally posted by bradhaak
    For instance even though I have seen the numbers that show how much faster my Prism is, I believe that certain operations on my M505 are much faster. I have seen this in games such as Zap 2016. On my M505, Zap plays much more smoothly and responsively.
    I can back you up on this one, Brad. On my Prism, I can consistently get 100% alien killage at the highest speed for the first few levels...on my m505, I'm having trouble even on the slowest setting getting 100% on the first level! It's just much faster on the m505.

Posting Permissions