 Notice. New forum software under development. It's going to miss a few functions and look a bit ugly for a while, but I'm working on it full time now as the old forum was too unstable. Couple days, all good. If you notice any issues, please contact me. |
Forum Index : Microcontroller and PC projects : More sophisticated benchmark
| Author | Message | ||||
kiiid Guru  Joined: 11/05/2013 Location: United KingdomPosts: 671 |
I looked at the existing benchmarking code base for MM and found out it is far from great, so I spent some time these days developing a more serious and accurate benchmark which takes into account the actual performance in mixed type statements in a more sophisticated manner. The program appears deceivingly simple, but it works quite accurately. The result forms a slightly non-linear output even on the same processor running at different speeds, which corresponds with the reality when measuring the actual amount of performed calculations over time. The reference point here is a 5HMz MX470 running MMBasic 5.2 on ELLO 2M in standalone mode (no USB attached). That generates 100 grains performance. I would be grateful if the program is ran on various MMBasic systems ranging from the first Maximites through the most recent MZ ports (will publish my results when the 'official' release is ready). This way we can have a more reliable and unified reference about the performance of various systems at various speeds. 2016-11-26_195025_grainbench.zip http://rittle.org -------------- |
||||
Geoffg Guru Joined: 06/06/2011 Location: AustraliaPosts: 3165 |
It is hard with benchmarks, you try to have a representative mix... but representative of what? Yours is very orientated towards floating point so the MZ should do well. But not everyone uses so much floating point, especially in embedded controllers. To misquote Disraeli: "There are three kinds of lies: lies, damned lies, and benchmarks" Geoff Geoff Graham - http://geoffg.net |
||||
| kiiid Guru Joined: 11/05/2013 Location: United KingdomPosts: 671 |
Actually that's the thing, I have tried hard to make it balanced between integer calculations, fp, and string handling. In fact the MX series performs quite well. Here is what I have tested so far: Micromite, MX470/5MHz, MMBasic 5.2 100 grains Micromite, MX170/5MHz, MMBasic 4.7 136 grains Micromite, MX170/40MHz, MMBasic 4.7 1181 grains Micromite, MX170/48MHz, MMBasic 4.7 1417 grains Micromite, MX470/100MHz, MMBasic 5.2 2151 grains Micromite EXTREME, MZEFH/198MHz, MMbasic 5.3b3 4060 grains Micromite MZ, MZEFH/198MHz, MMBasic 5.3b5 4067 grains (unreleased version) One thing which catches immediate attention is that MMBasic 5.2 is about 36% slower than 4.7 under other equalised conditions, probably due to the massive number of additions since then. http://rittle.org -------------- |
||||
| WhiteWizzard Guru Joined: 05/04/2013 Location: United KingdomPosts: 2794 |
@kiiid, On your PCB (i.e. MZ 64pinner), running Extreme v5.3b3, I am getting 4242 consistently  No TFT, or Touch setup, just your onboard SD socket. Coding via console pins. Only other option set is BAUDRATE @ 115200 WW For everything Micromite visit micromite.org Direct Email: whitewizzard@micromite.o |
||||
| kiiid Guru Joined: 11/05/2013 Location: United KingdomPosts: 671 |
That's with no USB I assume. Good stuff, thanks WW http://rittle.org -------------- |
||||
| WhiteWizzard Guru Joined: 05/04/2013 Location: United KingdomPosts: 2794 |
Just have a WiFi bridge direct on the Tx/Rx pins. Power coming in on USB socket. For everything Micromite visit micromite.org Direct Email: whitewizzard@micromite.o |
||||
| kiiid Guru Joined: 11/05/2013 Location: United KingdomPosts: 671 |
The benchmark is comparative of the balanced performance when executing mixed code, which is a very real indication of the actual amount of work that the system can do over a specified amount of time. So if you are getting 4242 in your setup that means the USB is slowing the overall system down by about 4-5% which corresponds with the reality. My readings were all taken with USB console on (except for the 100 grains one). I am interested in the performance of the older Maximites and the ARM ports. Unfortunately having neither with me here... http://rittle.org -------------- |
||||
| WhiteWizzard Guru Joined: 05/04/2013 Location: United KingdomPosts: 2794 |
With OPTION CONSOLE OFF or ON (and a connection via USB D+/D-), it drops to 3942. When USB is not connected to a 'computer', (i.e. nothing connected to D+/D-) then is always 4242. For everything Micromite visit micromite.org Direct Email: whitewizzard@micromite.o |
||||
| matherp Guru Joined: 11/12/2012 Location: United KingdomPosts: 8592 |
It is worth noting that based on my very limited testing 5.3 seems to be about 7% slower than 5.2 with the original benchmark. I assume this is because a major part of the core Basic code was rewritten by Geoff to fix a horrible bug but I may be completely wrong (as often is the case) My best is 4292 on a MZ EXTREME 200MHz no USB connection |
||||
| WhiteWizzard Guru Joined: 05/04/2013 Location: United KingdomPosts: 2794 |
An immediate question would be 'why the different figures?'. Would temperature have an impact? Everything else should be the same; i.e. same PIC, same code, minimal circuit. Can only assume component tolerances and voltage are the other 'variables'. Interested to hear your thoughts . . . . For everything Micromite visit micromite.org Direct Email: whitewizzard@micromite.o |
||||
| kiiid Guru Joined: 11/05/2013 Location: United KingdomPosts: 671 |
What different figures? http://rittle.org -------------- |
||||
| WhiteWizzard Guru Joined: 05/04/2013 Location: United KingdomPosts: 2794 |
Peter=4292. WW=4242 Same hardware, same software For everything Micromite visit micromite.org Direct Email: whitewizzard@micromite.o |
||||
| matherp Guru Joined: 11/12/2012 Location: United KingdomPosts: 8592 |
Peter 200MHz 24MHz oscillator, WW 198MHz 12MHz oscillator |
||||
| kiiid Guru Joined: 11/05/2013 Location: United KingdomPosts: 671 |
Just for the fun I ran MMBasic on my i7-6700HQ/2.6GHz :) Had to modify a tiny bit the variable declaration statements (only!) in order to make the program compatible with the older syntax parsing. Here is what I got... 2016-12-05_131329_grain.zip  http://rittle.org -------------- |
||||
 Print this page |
