Home
JAQForum Ver 20.06
Log In or Join  
Active Topics
Local Time 22:43 29 Mar 2024 Privacy Policy
Jump to

Notice. New forum software under development. It's going to miss a few functions and look a bit ugly for a while, but I'm working on it full time now as the old forum was too unstable. Couple days, all good. If you notice any issues, please contact me.

Forum Index : Microcontroller and PC projects : To be or not to be: Tokenized BASIC

Author Message
RC_tech
Newbie

Joined: 05/07/2020
Location: Germany
Posts: 14
Posted: 07:58am 05 Jul 2020
Copy link to clipboard 
Print this post

I don't have my CMM2 yet, but as I checked that is has around 512K for BASIC, that was a bit disappointing. Today memory isn't the cost factor.

I noticed that the BASIC files are plain text. This allows it to be readable in a normal text editor, but is this neccessary? Wouldn't it be easier to get back to tokenized BASIC, like in every classic machine? It would be possible to be text editor compatible: Every command uses only two letters from A-Z. This would allow 676 BASIC commands. Two define a command, it begins and ends with an question mark. This method would also allow to enter programs faster. The commands can be typed normally, but in the editor the left Ctrl key is used as token qualifier. Holding left-Ctrl and two letters will result in a BASIC command, which is shown as clear text.

Examples:

Ctrl-pr = PRINT. Code file text: ?PR?
Ctrl-nx = NEXT. Code file text: ?NX?
Ctrl-go = GOTO. Code file text: ?GO?

The second charater may be the numbers 0-9 to combine similar functions, f.e. G0 = GOTO, G1 = GOSUB, then the number of possible commands would increase to 936.  

So, every command would use just four bytes in the command. This would save a lot of memory. And if you once have the shortcuts in mind, typing will become much faster.


Greets, RC.
 
matherp
Guru

Joined: 11/12/2012
Location: United Kingdom
Posts: 8517
Posted: 08:09am 05 Jul 2020
Copy link to clipboard 
Print this post

Basic is tokenised with in-built commands and functions represented by a single byte. Suggest you wait til you have a CMM2 before seeing if you can use all the memory. The 512K is just for the compressed code. Variables get another 5Mb

From the manual page  13
"All Programs Are Run From the SD Card
On the Colour Maximite 2 all programs reside on the SD card which acts as the "disk drive" for the computer.   As a result the SD card must be present for most operations.  This is different from the original Maximite where the SD card was not necessarily required.
When you edit a program you are editing the program on the SD card, when you run a program you will run it from the SD card, etc.  The reason for this arrangement is that when a program is loaded into memory for execution MMBasic will do a lot of pre-processing to speed up execution.  This includes inserting any include files specified in the source, stripping out all comments, removing unnecessary spaces and so on.  The resultant program is then saved in flash but it cannot be edited or listed because after pre-processing the executable program is not easily human readable.
The benefit of this arrangement is a marked improvement in the speed of execution and that much larger programs can fit into program memory and be run."
Edited 2020-07-05 18:10 by matherp
 
RC_tech
Newbie

Joined: 05/07/2020
Location: Germany
Posts: 14
Posted: 08:19am 05 Jul 2020
Copy link to clipboard 
Print this post

The BASIC uses just one byte for a token? Even the C128 used two bytes for a lot of commands. The original BASIC V2 commands used one, the newer ones got a qualifier byte, then the token byte.

512K are reminding me always on the Amiga 500. It worked for a lot of games, but later 1 MB were neccessary. How about video memory? As an Amiga user I know about the 'Chip Memory' problem. There was always not enough. Is the video memory grabbed from the variable memory, main memory, or has it is own memory?
 
panky

Guru

Joined: 02/10/2012
Location: Australia
Posts: 1094
Posted: 10:34am 05 Jul 2020
Copy link to clipboard 
Print this post

  RC_tech said  I don't have my CMM2 yet, but as I checked that is has around 512K for BASIC, that was a bit disappointing........

With great respect RC_Tech, you might benefit from reading through the release post on the CMM2 - Geoff and Peter, the two lead developers, detailed a lot of the thinking behind the development of the CMM2.

Additionally, it should be noted that the CMM2 is just one in a number of platforms that MMBasic runs on (the Microchip PIC32MX170/470, PIC32MZ2048 family, the Rasberry Pi, MS DOS, a range of STMicro chips including the ST32H743 that the CMM2 is based on). This presents great advantages of "horses for courses" but also imposes some limitations on what can be included in the language. It also imposes what can be achieved by what microcontrollers are available to run MMBasic on.

Lastly, and I think I am paraphrasing Geoff here, one of the original goals was to create a modern version of BASIC to run on current microcontrollers and to " as much as possible, implement a GWBASIC compatible language".

Anyway, welcome to the forum, it is a diverse, enthusiastic, knowledgeable and helpful group and I hope you enjoy creating wonderful things on your CMM2 when it arrives.

panky.
... almost all of the Maximites, the MicromMites, the MM Extremes, the ArmMites, the PicoMite and loving it!
 
robert.rozee
Guru

Joined: 31/12/2012
Location: New Zealand
Posts: 2282
Posted: 11:16am 05 Jul 2020
Copy link to clipboard 
Print this post

the idea of using 16-bit tokens or a mix of 8-bit and 16-bit has been suggested before. the problem is that this would require some major changes right at the heart of the micromite basic core code, and the chances of introducing instability would be high. to date, the safest approach has been to instead ration the scarce supply of existing 8-bit tokens and make use of alternative tricks to achieve similar results.

the micromite source can be obtained from geoff at no charge (for private use), and if anyone (or a group) wanted to carry out a 16-bit or mixed 8/16-bit conversion, i am sure geoff would be happy to integrate this back into micromite basic once you had ironed out all the wrinkles.


peter: i'm curious, what prevents tokenized code being executed from RAM? i do realize this would require all the abovementioned pre-processing to be carried out every time 'RUN' was typed, which would be inefficient.


512k of data is enormous - based upon the way micromite basic stores variables, this would be enough for:
- over 8000 individual float or integer varialbes, or,
- over 2000 individual strings, or,
- a 2-dimensional array of over 700x700 floats or integers, or,
- a 1-dimensional array of over 64000 floats or integers.


cheers,
rob   :-)
Edited 2020-07-05 21:17 by robert.rozee
 
matherp
Guru

Joined: 11/12/2012
Location: United Kingdom
Posts: 8517
Posted: 11:27am 05 Jul 2020
Copy link to clipboard 
Print this post

  Quote  peter: i'm curious, what prevents tokenized code being executed from RAM?

Nothing in principle but storing the tokenised version in flash means that ist is non-volatile. Reading from flash with the M7 processor is basically zero wait state due to caching so there is no advantage to RAM from a performance perspective

  Quote  512k of data is enormous

512K is just the tokenized program. There is 5Mb for variables + 3Mb for video memory
 
robert.rozee
Guru

Joined: 31/12/2012
Location: New Zealand
Posts: 2282
Posted: 01:20pm 05 Jul 2020
Copy link to clipboard 
Print this post

oops, you're right peter - i was writing the last post while on the run!

multiply the numbers by 10:
5mb of data is enormous - based upon the way micromite basic stores variables, this would be enough for:
- over 80,000 individual float or integer varialbes, or,
- over 20,000 individual strings, or,
- a 2-dimensional array of over 2200x2200 floats or integers, or,
- a 1-dimensional array of over 625,000 floats or integers.

as for the 512k of FLASH space allocated to the running program storage, that would be over 12,000 lines of BASIC code with an average line length (when tokenized) of 40 bytes.


cheers,
rob   :-)
 
Print this page


To reply to this topic, you need to log in.

© JAQ Software 2024