RPi 3 vs. UDOO X86 CPU benchmarks

Discussion in 'UDOO X86' started by Markus Laire, Jun 17, 2017.

Tags:
  1. Markus Laire

    Markus Laire Active Member

    Joined:
    Mar 9, 2017
    Messages:
    225
    Likes Received:
    92
    As there has been some discussion about how good the CPU of UDOO X86 (all versions) actually is, I thought it might be good to have a thread for actual benchmark results.

    I'll add shortly results of RPi3 and UDOO X86 Ultra for "sysbench --test=cpu --cpu-max-prime=20000 --num-threads=4 run".

    If anyone knows other good benchmarks to try, please tell.
     
    ImLagging and waltervl like this.
  2. Maurice

    Maurice Active Member

    Joined:
    Oct 13, 2015
    Messages:
    394
    Likes Received:
    87
    Of the Advanced:
    sysbench 0.4.12: multi-threaded system evaluation benchmark

    Running the test with following options:
    Number of threads: 4

    Doing CPU performance benchmark

    Threads started!
    Done.

    Maximum prime number checked in CPU test: 20000


    Test execution summary:
    total time: 14.4013s
    total number of events: 10000
    total time taken by event execution: 57.5842
    per-request statistics:
    min: 5.70ms
    avg: 5.76ms
    max: 15.85ms
    approx. 95 percentile: 5.85ms

    Threads fairness:
    events (avg/stddev): 2500.0000/15.60
    execution time (avg/stddev): 14.3961/0.00
     
  3. waltervl

    waltervl UDOOer

    Joined:
    Dec 12, 2015
    Messages:
    2,314
    Likes Received:
    580
    On my Udoo X86 Advanced+ on Ubuntu 16.04 LTS
    Code:
    sysbench 0.4.12:  multi-threaded system evaluation benchmark
    
    Running the test with following options:
    Number of threads: 4
    
    Doing CPU performance benchmark
    
    Threads started!
    Done.
    
    Maximum prime number checked in CPU test: 20000
    
    
    Test execution summary:
        total time:                          14.6706s
        total number of events:              10000
        total time taken by event execution: 58.6516
        per-request statistics:
             min:                                  5.77ms
             avg:                                  5.87ms
             max:                                 40.63ms
             approx.  95 percentile:               5.80ms
    
    Threads fairness:
        events (avg/stddev):           2500.0000/27.74
        execution time (avg/stddev):   14.6629/0.00
    

    Just for the laughs, on my Udoo Neo:
    Code:
    Test execution summary:
        total time:                          610.5784s
        total number of events:              10000
        total time taken by event execution: 610.5547
        per-request statistics:
             min:                                 60.28ms
             avg:                                 61.06ms
             max:                                141.77ms
             approx.  95 percentile:              61.82ms
    
    Threads fairness:
        events (avg/stddev):           10000.0000/0.00
        execution time (avg/stddev):   610.5547/0.00
    Edit: Udoo X86 results added.
     
    Last edited: Jun 29, 2017
  4. Markus Laire

    Markus Laire Active Member

    Joined:
    Mar 9, 2017
    Messages:
    225
    Likes Received:
    92
    Code:
    sysbench --test=cpu --cpu-max-prime=20000 --num-threads=4 run
    
    UDOO X86 Ultra (Debian Stretch, sysbench 0.4.12)
    Code:
    Test execution summary:
        total time:                          12.1223s
        total number of events:              10000
        total time taken by event execution: 48.4716
        per-request statistics:
             min:                                  4.82ms
             avg:                                  4.85ms
             max:                                 18.75ms
             approx.  95 percentile:               4.86ms
    
    Threads fairness:
        events (avg/stddev):           2500.0000/1.22
        execution time (avg/stddev):   12.1179/0.00
    
    Raspberry Pi 3 Model B (Raspbian Jessie, sysbench 0.4.12)
    Code:
    Test execution summary:
        total time:                          119.6545s
        total number of events:              10000
        total time taken by event execution: 478.4663
        per-request statistics:
             min:                                 47.69ms
             avg:                                 47.85ms
             max:                                133.20ms
             approx.  95 percentile:              47.93ms
    
    Threads fairness:
        events (avg/stddev):           2500.0000/4.12
        execution time (avg/stddev):   119.6166/0.02
    
    So Ultra vs. rpi3 is about 10x speed difference in this test, as said in Kickstarter campaign.
     
    Fuzzyroll, ImLagging, modjo and 2 others like this.
  5. yobneb

    yobneb UDOOer

    Joined:
    Jun 17, 2017
    Messages:
    2
    Likes Received:
    2
    Here it is for an "older" up board (1).
    Code:
     sysbench --test=cpu --cpu-max-prime=20000 --num-threads=4 run
    sysbench 0.4.12:  multi-threaded system evaluation benchmark
    
    Running the test with following options:
    Number of threads: 4
    
    Doing CPU performance benchmark
    
    Threads started!
    Done.
    
    Maximum prime number checked in CPU test: 20000
    
    
    Test execution summary:
        total time:                          15.0586s
        total number of events:              10000
        total time taken by event execution: 60.2131
        per-request statistics:
             min:                                  5.98ms
             avg:                                  6.02ms
             max:                                 11.64ms
             approx.  95 percentile:               6.01ms
    
    Threads fairness:
        events (avg/stddev):           2500.0000/22.53
        execution time (avg/stddev):   15.0533/0.00
    
     
    Fuzzyroll and Markus Laire like this.
  6. Ethin E.

    Ethin E. New Member

    Joined:
    Mar 30, 2017
    Messages:
    16
    Likes Received:
    7
    UDOO Basic, FreeBSD 11-RELEASE-p1
    sysbench 0.4.12
    Code:
    sysbench --test=cpu --cpu-max-prime=20000 --num-threads=4 run
    
    Code:
    Test execution summary:
        total time:                          29.5172s
        total number of events:              10000
        total time taken by event execution: 118.0323
        per-request statistics:
             min:                                 11.36ms
             avg:                                 11.80ms
             max:                                 58.81ms
             approx.  95 percentile:              12.25ms
    
    Threads fairness:
        events (avg/stddev):           2500.0000/91.52
        execution time (avg/stddev):   29.5081/0.00
    
    Raspberry Pi 3 Raspbian Jessie Linux kernel 4.4.38-v7+ GNU/Linux
    sysbench 0.4.12
    Code:
    sysbench --test=cpu --cpu-max-prime=20000 --num-threads=4 run
    
    Code:
    Test execution summary:
        total time:                          119.5216s
        total number of events:              10000
        total time taken by event execution: 477.9701
        per-request statistics:
             min:                                 47.69ms
             avg:                                 47.80ms
             max:                                121.42ms
             approx.  95 percentile:              47.90ms
    
    Threads fairness:
        events (avg/stddev):           2500.0000/4.53
        execution time (avg/stddev):   119.4925/0.02
    
     
    Last edited: Jun 17, 2017
    Fuzzyroll and Markus Laire like this.
  7. Markus Laire

    Markus Laire Active Member

    Joined:
    Mar 9, 2017
    Messages:
    225
    Likes Received:
    92
    Interesting that Basic is so much slower than Advanced or Ultra.

    Code:
    Board                    Total time
    ----------------------   -----------
    UDOO X86 Ultra            12.1 s
    UDOO X86 Advanced         14.4 s
    UDOO X86 Basic            29.5 s
    Raspberry Pi 3 Model B   120   s
    
     
  8. Ethin E.

    Ethin E. New Member

    Joined:
    Mar 30, 2017
    Messages:
    16
    Likes Received:
    7
    That might be due to the operating system. It would be interesting if someone could run sysbench on the Basic with Linux.
     
  9. Ethin E.

    Ethin E. New Member

    Joined:
    Mar 30, 2017
    Messages:
    16
    Likes Received:
    7
    So I also ran it on my Raspberry Pi Zero W. For consistency, I still ran it with 4 threads, even though it only has a single core.

    Raspberry Pi Zero W Raspbian Jessie Linux kernel 4.9.24+ GNU/Linux
    sysbench 0.4.12
    Code:
    sysbench --test=cpu --cpu-max-prime=20000 --num-threads=4 run
    
    Code:
    Test execution summary:
        total time:                          908.0136s
        total number of events:              10000
        total time taken by event execution: 3631.8165
        per-request statistics:
             min:                                327.64ms
             avg:                                363.18ms
             max:                                470.53ms
             approx.  95 percentile:             390.65ms
    
    Threads fairness:
        events (avg/stddev):           2500.0000/0.00
        execution time (avg/stddev):   907.9541/0.04
    [code]
    And with a single thread, so it's more comparable to the Neo, below. The runtime is obviously similar, but the per-request statistics are different:
    Code:
    sysbench --test=cpu --cpu-max-prime=20000 --num-threads=1 run
    
    Code:
    Test execution summary:
        total time:                          906.4023s
        total number of events:              10000
        total time taken by event execution: 906.3717
        per-request statistics:
             min:                                 90.56ms
             avg:                                 90.64ms
             max:                                114.02ms
             approx.  95 percentile:              90.71ms
    
    Threads fairness:
        events (avg/stddev):           10000.0000/0.00
        execution time (avg/stddev):   906.3717/0.00
    
     
    Last edited: Jun 17, 2017
    Fuzzyroll and waltervl like this.
  10. sirrab

    sirrab UDOOer

    Joined:
    Jul 26, 2014
    Messages:
    264
    Likes Received:
    32
    Just as a reference my main desktop Xeon X3370 Ubuntu 16.04....
    Running the test with following options:
    Number of threads: 4
    Doing CPU performance benchmark
    Threads started!
    Done.
    Maximum prime number checked in CPU test: 20000
    Test execution summary:
    total time: 5.6209s
    total number of events: 10000
    total time taken by event execution: 22.4697
    per-request statistics:
    min: 2.17ms
    avg: 2.25ms
    max: 11.17ms
    approx. 95 percentile: 2.33ms
    Threads fairness:
    events (avg/stddev): 2500.0000/71.91
    execution time (avg/stddev): 5.6174/0.00
     
    Fuzzyroll, Ethin E. and waltervl like this.
  11. waltervl

    waltervl UDOOer

    Joined:
    Dec 12, 2015
    Messages:
    2,314
    Likes Received:
    580
  12. srohmen

    srohmen New Member

    Joined:
    Apr 19, 2017
    Messages:
    5
    Likes Received:
    6
    Hi there,

    One has to be careful when comparing benchmark results. I have read a blog entry and checked the profiling results and assembly by myself. There are quite huge differences in the produced machine code depending on which compiler version or optimization I used. I do not know if you compiled sysbench by your own or the one from the linux distribution. In particular on BSD the sysbench binary can have differences. Also the pthread implementation can have flaws. Just to mention: One should state the details of the benchmarking environment more accurately, otherwise you are comparing two different things without knowing.
     
    Markus Laire likes this.
  13. Markus Laire

    Markus Laire Active Member

    Joined:
    Mar 9, 2017
    Messages:
    225
    Likes Received:
    92
    Yes, true benchmarking is quite hard to do correctly.
     
  14. waltervl

    waltervl UDOOer

    Joined:
    Dec 12, 2015
    Messages:
    2,314
    Likes Received:
    580
    I agree but it is better than having nothing other than some remarks of someone claiming some performance issues. It will also indicate that an RPi ist something different than a Udoo X86.

    It also will give some opportunities to compare the Udoo X86's we already seen a strange result but it will need confirmation.
     
  15. Iggy

    Iggy New Member

    Joined:
    Jun 28, 2017
    Messages:
    1
    Likes Received:
    0
    Hello,

    I am also disappointed in the speed of my x86 Advanced Plus. Comparing to Odroid U3 (old design!) the x86 feels slower running Ubuntu 16.xx

    I found out that the system is faster if I cool it with a fan, an option I initially did not order. I have ubuntu 16.10 installed on 32GB
    emmc card. I run Odroid also from its emmc card.

    I never tried Windows, but currently I am not impressed.

    The only advantage of the current setup is more programs work on x86 than on armhf. Honestly I run this as a summer PC as my i7 heats up the house too much. I did expect much more.

    There is however a full arduino compatible half of the board, this I am curious how it works.



    Now for my test results, Ubuntu 16.10, UDOO x86 Advanced, running from emmc disk:

    Code:
    sysbench 0.4.12:  multi-threaded system evaluation benchmark
    
    Running the test with following options:
    Number of threads: 4
    
    Doing CPU performance benchmark
    
    Threads started!
    Done.
    
    Maximum prime number checked in CPU test: 20000
    
    
    Test execution summary:
        total time:                          15.5837s
        total number of events:              10000
        total time taken by event execution: 62.3111
        per-request statistics:
             min:                                  5.77ms
             avg:                                  6.23ms
             max:                                 46.10ms
             approx.  95 percentile:               5.90ms
    
    Threads fairness:
        events (avg/stddev):           2500.0000/25.25
        execution time (avg/stddev):   15.5778/0.00


    I do not own an RPi 3.


    EDIT: Found a comparison between Odroid XU4 and RPi, doing the same benchmark: .

    We get much better times with the x86. Perhaps the Ubuntu running on armhf is simply more optimised?
     
    Last edited: Jun 29, 2017
  16. waltervl

    waltervl UDOOer

    Joined:
    Dec 12, 2015
    Messages:
    2,314
    Likes Received:
    580
    You could try lubuntu or xubuntu on the Udoo X86. These versions are more optimized for low spec cpu's which the Udoo X86 is in the X86 world.
     
  17. waltervl

    waltervl UDOOer

    Joined:
    Dec 12, 2015
    Messages:
    2,314
    Likes Received:
    580
    Indeed, In order to rule out individual situations it would be nice if more Udoo X86 Basic owners can run this test and share the results!
     
  18. Maurice

    Maurice Active Member

    Joined:
    Oct 13, 2015
    Messages:
    394
    Likes Received:
    87
  19. Benjamin

    Benjamin New Member

    Joined:
    Mar 29, 2017
    Messages:
    12
    Likes Received:
    7
    UDOO Advanced+, FreeBSD 12-CURRENT-8563ee90e56
    sysbench 0.4.12

    Code:
    sysbench --test=cpu --cpu-max-prime=20000 --num-threads=4 run
    Code:
    Maximum prime number checked in CPU test: 20000
    Test execution summary:
        total time:                          14.1995s
        total number of events:              10000
        total time taken by event execution: 56.7785
        per-request statistics:
             min:                                  5.49ms
             avg:                                  5.68ms
             max:                                150.09ms
             approx.  95 percentile:               5.94ms
    
    Threads fairness:
        events (avg/stddev):           2500.0000/8.86
        execution time (avg/stddev):   14.1946/0.00
    
    This box was also not completely isolated for the test, mythbackend/frontend were still running at the time of the test and it's a FreeBSD kernel with witness enabeld. Though it does show the Advanced plus is pretty consistent on each platform and it must be the CPU type impacting the basic.
     
    Ethin E. and Markus Laire like this.
  20. samholton

    samholton New Member

    Joined:
    May 22, 2017
    Messages:
    26
    Likes Received:
    6
    Udoo X86 Advanced (Debian Stretch)
    Code:
    sysbench 0.4.12:  multi-threaded system evaluation benchmark
    
    Running the test with following options:
    Number of threads: 4
    
    Doing CPU performance benchmark
    
    Threads started!
    Done.
    
    Maximum prime number checked in CPU test: 20000
    
    
    Test execution summary:
        total time:                          14.5241s
        total number of events:              10000
        total time taken by event execution: 58.0801
        per-request statistics:
             min:                                  5.70ms
             avg:                                  5.81ms
             max:                                 27.47ms
             approx.  95 percentile:               5.84ms
    
    Threads fairness:
        events (avg/stddev):           2500.0000/8.31
        execution time (avg/stddev):   14.5200/0.00
    
    Pi 3 Model B (Debian Jessie - RetroPie Image)
    Code:
    sysbench 0.4.12:  multi-threaded system evaluation benchmark
    
    Running the test with following options:
    Number of threads: 4
    
    Doing CPU performance benchmark
    
    Threads started!
    Done.
    
    Maximum prime number checked in CPU test: 20000
    
    
    Test execution summary:
        total time:                          124.1305s
        total number of events:              10000
        total time taken by event execution: 496.4051
        per-request statistics:
             min:                                 47.69ms
             avg:                                 49.64ms
             max:                                153.56ms
             approx.  95 percentile:              55.90ms
    
    Threads fairness:
        events (avg/stddev):           2500.0000/16.39
        execution time (avg/stddev):   124.1013/0.03
    
     

Share This Page