 Notice. New forum software under development. It's going to miss a few functions and look a bit ugly for a while, but I'm working on it full time now as the old forum was too unstable. Couple days, all good. If you notice any issues, please contact me. |
Forum Index : Microcontroller and PC projects : MMBasic 4.4 Beta 1
  Page 2 of 3   |
|||||
| Author | Message | ||||
Juri74 Senior Member  Joined: 06/02/2012 Location: ItalyPosts: 162 |
that's not fully exact.. you can access the program area with peek/poke with keyword PROGMEM and maybe it could be possible to modify the program on the fly.. however your idea of executing a program directly from A: drive, leaving all ram for variables, sprites, fonts etc.. could be a great idea.. if it could be done even with data files (example a font that is used directly when it is inside drive A:) could be even great idea... problem is that drive A: have a read/write cycle life too short to use with programs, however with fonts it could be still usable, i think, with a finished font, it can be placed in A: and used with an appropriate update to the FONT command (example FONT #nbr,"A:filename") like the MODPLAY command do.. Juri |
||||
MicroBlocks Guru  Joined: 12/05/2012 Location: ThailandPosts: 2209 |
The only 'problem' with executing a basic program that is stored in flash is that it would be slower. If i read the docs correctly every access to flash needs 7 wait states and from ram no wait states. If performance is not crucial it can be a good way to save some bytes. Also the erase/write cycles of flash will only allow for storage of more permanent programs, while development should still use RAM. At some time you need to test it while it is stored in flash. However the biggest drain on memory is of course VGA and strings. The current solution that lets you CHAIN programs is i think the best of both worlds. Your program can be as big as the storage of an sd card, while it still runs from ram. The improvement list for MMBasic will always be long and different for every person. My top wishes include 32 bit integer and better string storage. :) Microblocks. Build with logic. |
||||
| JohnS Guru Joined: 18/11/2011 Location: United KingdomPosts: 3678 |
Executing the BASIC from flash shouldn't slow things much as the real overhead is all the things the (MMBasic) interpreter does. If there was plenty of RAM it would make sense to copy MMBasic itself into the RAM! Compiling to a bytecode would add speed but is more trouble for Geoff. The upcoming PIC32 devices sound to be the real solution :) John |
||||
kiiid Guru Joined: 11/05/2013 Location: United KingdomPosts: 671 |
While the flash may be slightly slower than RAM, let's not forget that the MMBasic itself is executing from there and the interpretation of all the commands is code executing from the flash memory. Drive A: is memory no different than the one the very same interpreter is running from. I think there will be virtually no difference in the speed of the Basic programs whether they will be running from RAM or flash. But the benefits will be much more significant than the drawbacks. CHAIN is pretty much useless in many cases (that's why it has never been used much). Actually the only case when it can actually be useful is if you have one long program that could be broken into smaller parts. But that is a rare occasion. Speaking in that direction, I think GOSUS label,"filename" would be a valuable addition as it will allow the use of real external software libraries. That is actually on my roadmap for the MMBasic R3 for the 4105C module as well as the execution in place, which as I said may sound like a big deal, but is in fact a very small mod in the sources. But I think that should actually be in root version of MMBasic. Ok, maybe it should be called 5.0, instead of 4.4 in that case... Going even further, because there is no connection between when programs are running or being edited, the very same memory can be reused - when you edit a program, the editor loads the source in RAM. When you run it, it always runs from flash (just has to be saved) leaving the whole RAM for variables and other stuff. The flash memory of PIC32 has about 50000 guaranteed write cycles endurance. I don't believe you would be able to edit and save your program that many times (100 times a day for 500 days every day!). And let's not fool ourselves, in theory you can self-modify the Basic code during execution. But how many have actually done that and on how many occasions? And lastly, the new 32MZ family is not the silver bullet for everything. Yes, it will be much better, but that cannot be justification for not optimising the software or the whole party will head the PC way otherwise (crappy software hidden behind fast hardware)... 
http://rittle.org -------------- |
||||
| JohnS Guru Joined: 18/11/2011 Location: United KingdomPosts: 3678 |
I think you're way out on the guaranteed flash write cycles. It's far fewer at only 1000. John |
||||
| kiiid Guru Joined: 11/05/2013 Location: United KingdomPosts: 671 |
Correct. Minimum 1000... Anyway, my point stands unchanged http://rittle.org -------------- |
||||
| MicroBlocks Guru Joined: 12/05/2012 Location: ThailandPosts: 2209 |
If effort is made i think it should not be in the direction of running code from flash. String handling is currently the biggest waste of ram. It is also not the easiest part to change. If a one character string is using 256 bytes in memory you can imagine how much memory can be saved by better string handling. Imagine what amount of memory is used by a Dim a$(100). If optimizing is the goal, the biggest gain can be made solving this problem. Kiid, if you are good at programming why not give that a shot? Lots of people, including me would be very happy when strings are optimized. I think i read somewhere that Geoff is going to tackle the integer problem. Using integers instead of floats would not use less ram but it will speed up things tremendously (when used in calculations and loops). And to get back on topic. :) The random file access is working fine. I would change "APPEND" into "RANDOM" as it describes the use of the file better. The implementation is in my opinion good. Microblocks. Build with logic. |
||||
| kiiid Guru Joined: 11/05/2013 Location: United KingdomPosts: 671 |
Point taken. If Geoff is busy doing other stuff, I might give a shot at some better string handling... The only problem is that my revisions of MMBasic are specifically targeted to support the DTX2 modules hardware, and that might not be acceptable for some users  http://rittle.org -------------- |
||||
Geoffg Guru Joined: 06/06/2011 Location: AustraliaPosts: 3167 |
There is a whole pile of things in this thread and I thought that I would cover them in the one post. There is no EOF character. The EOF() function will return true when you have read the last byte from the file and if you try to read past that byte an error will be thrown. Neat idea but it is not as easy as it looks. This is because when a program is loaded into memory it is tokenised. This means command names, functions, etc are converted into single byte tokens - this both saves RAM and speeds up the interpreter. When a file is saved the tokens are converted back to the ASCII command names, etc so the user is unaware of the process. To make the interpreter work with a non tokenised program would require a huge rewrite that cannot be justified. I suppose that an alternative is that there could be a special command for saving the file in tokenised format and another command to run that format from flash or the SD card. It would be rather messy but is something to think about. It is 1000 cycles but the MMBasic filesystem on drive A: uses wear leveling so the number of file write/erases is far higher than 1000. This has been extensively covered in previous posts. I definitely agree with this. But, it will require a complex memory management system and will force the use of garbage collection which will slow down the interpreter considerably. At this time MMBasic uses a VERY simple memory system for strings and therefor is very fast. I shudder to think how slow it would run with something more complex. (garbage collection is code that continuously runs through the strings in RAM rearranging them to prevent fragmentation of the free memory space). Geoff Geoff Graham - http://geoffg.net |
||||
| kiiid Guru Joined: 11/05/2013 Location: United KingdomPosts: 671 |
Geoff, Yes, saving in tokenised format was basically what I had in mind. There is no need of a separate RUN command, though as it can be easily modified to recognise the file type. Loading of files of such type is also easy to be recognised by the LOAD command. And the saving could be just an extension to the existing SAVE, so no extra commands will be needed. So, another (let say for example .TOK) type of file, which would allow direct execution from the flash was the essence of the whole idea. What do you think about the other suggestion about GOSUB label,"filename"? It looks to me a bit more difficult for implementation, but would be really nice to have in programs something like: GOSUB MyFunc,"MyLib.bas" http://rittle.org -------------- |
||||
| MicroBlocks Guru Joined: 12/05/2012 Location: ThailandPosts: 2209 |
Just toying around with some ideas....... Garbage collection is always a pain. For a multi purpose computer it can still be ok, but in an embedded situation it is something to avoid. In the old TRS-80 level II there was a CLEAR command to set aside a specific amount of memory for string use. This gave the programmer some control about how much and often garbage collection took place. The worst use is often when swapping values like in a sort routine. Temp$=A$: B$ = A$: B$ = Temp$ it creates a maximum of 3 new strings and causes the old memory to become invalid. Lots of time wasted, but it can still be fast enough. If speed and especially repeatability (sp) of speed is required like in interrupt routines you would need a string that has its own predefined space. You don't want to have the chance that CG is started when in a time critical situation. (CG always off in case of an interrupt?, OR CG ON|OFF command?) If for instance you have an interrupt that requires to read a string from a peripheral in many cases it is just one or more bytes. In that case a string currently uses 256 bytes while it will never have more then for example 5. Maybe a way to define how much space a string needs would be a good compromise. As this string will never grow beyond its maximum size it will never have to be relocated. Maybe a STATIC keyword can be added to define such a string. the use would then be for normal strings: [code] CLEAR DYNAMIC 5000 'reserve 5000 bytes memory for dynamic strings DIM A$(10) '10 string pointers are created [/code] and for static strings (maybe also for other types): [code] CLEAR STATIC 5000 'reserve 5000 bytes for static variables. STATIC B$(10)[20] '1 (one!) string pointer to an array of 200 bytes in memory STATIC C$[100] 'one string pointer to 100 bytes in memory [/code] A pointer would only need to be 16 bits as it can be used as an offset to the start of the allocated area in memory. CLEAR DYNAMIC would be optional, maybe 1000 as a default. In the interpreter only that specific area has to be managed (GC). Ideally with the help of DMA strings can be moved around very quick and free up the processor. CLEAR STATIC also optional, 1000 as a default. Allocate a fixed location for every static defined variable. Added difficulty is that there can be situations where strings get bigger then defined. Those would have to trigger an error as that is a user programming error. Microblocks. Build with logic. |
||||
boss Senior Member Joined: 19/08/2011 Location: CanadaPosts: 268 |
Good morning, 1) Strings are most memory consuming variables in MMBasic, and the Garbage collection is a difficult task, and I don't think this is necessary in this case, why not to declare the strings the same way like we declare the ARRAYS? ie: Declare A$(4),B$(8) 'will save 500 bytes of scarce RAM. 2)upon my experience the interpreter is extremely comfortable during development and debugging phase of programming. Once this phase is over the program modifications are (usually) infrequent. Why not to have possibility to compile the .BAS program to .EXE (executable file even via cross compiler on PC)and RUN it from flash disc A:? Next embedded processor generation will contain more flash memory with no or very little additional cost (I expect 1-4MB) because flash memory occupies significantly smaller chip area than RAM. 3)double precision and integer32 math are highly desired (by me) |
||||
| shoebuckle Senior Member Joined: 21/01/2012 Location: AustraliaPosts: 189 |
I like the idea of defining the maximum length of a string variable, a$[5], to save RAM. At present it always takes a default of 256 characters. The length spec would need to be enclosed in other than () to avoid confusion with array definition. Geoff, would such an idea be easier to implement? This could be an optional addition to the DIM command. Cheers, Hugh |
||||
| Geoffg Guru Joined: 06/06/2011 Location: AustraliaPosts: 3167 |
It would certainly be easier than dynamic variable length allocation. At this time the memory management system built into MMBasic doles out memory in 256 byte chunks and everything is based around that. But, if it is changed to (say) 32 byte chunks then a string could be declared with a fixed length of a multiple of 32 bytes (eg, 96 bytes). The granularity could be more or less than 32 bytes and the optimum number would depend on tradeoffs between the size of the tables used to track these chunks and the amount of extra memory saved when strings were used. For example, a program that did not use strings at all would always have less RAM because the overhead associated with the 32 byte chunk size. On the other hand, a program that used strings a lot would be in front. And no, please don't somebody ask for the chunk size to be a configuration parameter... that way madness waits. The downside is that fixed length strings (even if optional) will add more non standard syntax to the language and another opportunity to confuse a beginner. I know that it is disappointing to see memory vanish into 256 byte strings but how many people have real world programs that have run out of memory because of the string size? It was my impression that the introduction of the more sophisticated memory management in ver 3.2 made so much more memory available that the issue went away. But I guess colour took some of that back. So, who has run out of memory because of strings? Secondly, would it be OK if fixed length strings were only implemented in arrays with different arrays allowed to have different sizes? And a third question, would an option at the beginning of the program to set the string size for all strings be better? Speak up! Geoff Geoff Graham - http://geoffg.net |
||||
| robert.rozee Guru Joined: 31/12/2012 Location: New ZealandPosts: 2294 |
i'd like to suggest the granularity be set at the size of the basic variable type, the numeric, which is 6 bytes long i believe? it would be a step forward to require all strings to be DIMensioned before use, and perhaps this could be extended to all variables. in pascal, all variables must be declared before use, as is the case in many other programming languages. the biggest advantage here is that someone can't accidentally misspell a variable name without an error being raised. declaring variables before use isn't a major burden, and i'd doubt this would confuse beginners. it would, indeed, make for much more readable code. the only down side - almost all existing programs would require a little modification (adding declarations), but then such changes are pretty straight forward. |
||||
| Geoffg Guru Joined: 06/06/2011 Location: AustraliaPosts: 3167 |
There have been a few suggestions like this in the past and this one has the advantage of allowing the memory to be automatically freed on RETURN from the GOSUB. But, its disadvantage would be speed as every GOSUB would require a load from permanent storage. Another suggestion was: MODULE LOAD "MyLib.bas" ... cal functions in the module ... MODULE UNLOAD "MyLib.bas" I am afraid that this is still on my todo list waiting for evidence of enough people running out of memory (says he donning a fireproof suit!  )
One of the issues with all these suggested new commands and functionality is that I have to be very careful when adding things to MMBasic. There are a number of examples (which I won't list here) where I have not put sufficient thought into some new addition and it turned out that the "feature" could have been done better or is not very useful or has some other issue. But, once something is in the language it can never be taken out because someone somewhere will have written a program that uses that feature. So, any bad decisions must stay there for ever. As a result I really must know that something is useful, will be used by lots of people, covers all possible uses, has a good syntax and fits with the BASIC language overall before I put it in. This also applies to something that you might develop - if it meets these standards I will gladly fold it into the standard version of MMBasic. All the above is made more complex because MMBasic and the Maximite is used for so many purposes from number crunching to games and embedded controllers. One person's feature may be another's baggage. Geoff Geoff Graham - http://geoffg.net |
||||
| MicroBlocks Guru Joined: 12/05/2012 Location: ThailandPosts: 2209 |
I think using a different keyword for declaring variables will keep it backward compatible. The STATIC keyword is a good candidate. If only this keyword allows string declaration of fixed length it would not conflict with the DIM statement. In functions and subroutines (especially interrupt routines) it can be of good use as it will keep using the same space and no constant deallocation and allocation is necessary. If STATIC is used in combination with a CLEAR you would be able to scan a program for the STATIC keyword (during the scan for functions and subroutines) and allocate the exact amount of space needed in the most optimized way. A scan will tell you how many variables are used and will help creating a lookup table of the exact size needed. A lookup table structure can be like this: [code] Datatype Dimensions Length Offset 3 bits 3 bit 10 bits 16 bits [/code] Datatype 0000 = Single, 0001 = string, 0010 = integer, etc Dimensions 0 is not an array otherwise the number of dimensions Length for a string datatype from 1 to 1024 Offset Beginning of storage area + Offset points to the area in memory where the data is stored. A preprocessing of variablenames can then be made to have quick variable name to stored data lookup. This will keep all existing programs running without modification and offers a faster and smaller string (and other variable types) handling for new made programs. Defining the fixed length of the string would need some extra syntax. I think the use of square brackets is the most terse: STATIC B$[100] 'one string with fixed length = 100 It is a bit strange ( I could get used to it :) ) when used in combination with an array: STATIC B$(100)[20] '100 string of fixed length 20 The normal strings and variable types that are used now can then stay as they are and can be optimized by a better memory management if that is still necessary, which will also stay backwards compatible. Microblocks. Build with logic. |
||||
| shoebuckle Senior Member Joined: 21/01/2012 Location: AustraliaPosts: 189 |
Hi Geoff, I have run out of memory on two occasions in developing Crunch, but this is the only program so far which has. The first was in sorting the list of reserved words and eliminating duplicates. The other is during processing source program lines to compress them. There is a lot of string manipulation and several passes required, all of which requires large arrays. Yes, being able to reduce the size of array variables would make it easier but that would take away the challenge of finding ways of doing the job within the limitations of MMBasic. Of course I would use it, but it isn't necessary IMHO. Setting the string size of all strings wouldn't be "better" but would almost certainly eliminate the need to divide the Crunch function into a sequence of programs, but don't do it for me, if I am the only one who has run out of memory because of string size. There is always a way around and it is fun to find a solution. Cheers, Hugh |
||||
| shoebuckle Senior Member Joined: 21/01/2012 Location: AustraliaPosts: 189 |
To be consistent with parameters on other commands, perhaps a comma would be better than square brackets. e.g. DIM a$(2,10),10 to define an array with elements 10 characters long. Making the length spec optional would make the command backward compatible. Cheers, Hugh |
||||
| kiiid Guru Joined: 11/05/2013 Location: United KingdomPosts: 671 |
+1 http://rittle.org -------------- |
||||
| Page 2 of 3 |
|||||
 Print this page |
