Home
JAQForum Ver 20.06
Log In or Join  
Active Topics
Local Time 11:30 26 Apr 2024 Privacy Policy
Jump to

Notice. New forum software under development. It's going to miss a few functions and look a bit ugly for a while, but I'm working on it full time now as the old forum was too unstable. Couple days, all good. If you notice any issues, please contact me.

Forum Index : Microcontroller and PC projects : MMBasic DATA statement and ASCII above 127

Author Message
vegipete

Guru

Joined: 29/01/2013
Location: Canada
Posts: 1082
Posted: 07:51pm 30 Dec 2020
Copy link to clipboard 
Print this post

It seems that DATA statements can hold strings with characters with ascii codes greater than 127, but the READ statement cannot read these characters. (CMM2, 5.06.00)

This program demonstrates this: datatest.zip
If you run the program, your get the basic characters and some minor weirdness.
But if you list or edit the program, you see a whole lot more.

Is this true of MMBasic in general or just the CMM2? Is this as intended?

Notes:
- editing the program off the CMM2 could very likely alter the ascii codes

- transferring the program to the CMM2 by serial could very likely alter the ascii codes. Direct SD card transfer may be required.

- the double quote character (") is deliberately omitted from the data statements
Visit Vegipete's *Mite Library for cool programs.
 
robert.rozee
Guru

Joined: 31/12/2012
Location: New Zealand
Posts: 2287
Posted: 11:53am 31 Dec 2020
Copy link to clipboard 
Print this post

  vegipete said  It seems that DATA statements can hold strings with characters with ascii codes greater than 127, but the READ statement cannot read these characters. (CMM2, 5.06.00)


the micromites (MX170, etc) do not allow entering characters with the MSB set when using the AUTOSAVE command - the MSB is automatically cleared. my assumption has always been that the same holds for the inbuilt editor, as well as when using XMODEM to upload. furthermore, some characters with the MSB set map to VT escape codes, hence could risk tripping up the inbuilt editor if manually entered.

hopefully geoff can confirm my above assumptions.

if the CMM2 does not filter out characters (1) in a loaded program with the MSB set, or (2) entered from a USB keyboard into the editor, then i'd be inclined to suggest this is something that needs fixing.


cheers,
rob   :-)
 
matherp
Guru

Joined: 11/12/2012
Location: United Kingdom
Posts: 8578
Posted: 01:09pm 31 Dec 2020
Copy link to clipboard 
Print this post

What you are trying to do doesn't make any sense. Strings can hold any number between 0 and 255 but if you want to construct a string with numbers outside of the normal range ASCII 32-127 then you need to use the chr$() function or poke.
Above 127 the character may or may not have a visual representation depending on the font and none of the CMM2 fonts include the characters you show.

Essentially you are trying to load binary data and the best way to do this is from file using the bin2str$ function.

Then if you want to display your characters you will need to create a font that includes them
 
vegipete

Guru

Joined: 29/01/2013
Location: Canada
Posts: 1082
Posted: 06:16pm 31 Dec 2020
Copy link to clipboard 
Print this post

  matherp said  Above 127 the character may or may not have a visual representation depending on the font and none of the CMM2 fonts include the characters you show.

Say what? May I point out page 30 of the manual?
  Quote  Font #1 (the default font) and font #4 have an extended
character set covering all characters from CHR$(32) to
CHR$(255) or 20 to FF (hex) as illustrated on the right.

All I want to do is put these characters directly in DATA statements precisely so I don't have to horse around with CHR$. My sample program demonstrates that I can get the characters into DATA statements - I just can't get them back out with READ.

LIST or EDIT the sample program I provided above and compare that to the output from RUN. The fault appears to lie in the READ statement - it doesn't read and return the exact string in quotes in the DATA statement but instead performs some processing.
Edited 2021-01-01 04:18 by vegipete
Visit Vegipete's *Mite Library for cool programs.
 
matherp
Guru

Joined: 11/12/2012
Location: United Kingdom
Posts: 8578
Posted: 06:37pm 31 Dec 2020
Copy link to clipboard 
Print this post

Standard across all versions of MMbasic. In the tokeniser:

   // first, make sure that only printable characters are in the line
   p = inpbuf;
   while(*p) {
       *p = *p & 0x7f;
       if(*p < ' ' || *p == 0x7f)  *p = ' ';
       p++;
   }
 
vegipete

Guru

Joined: 29/01/2013
Location: Canada
Posts: 1082
Posted: 06:40pm 31 Dec 2020
Copy link to clipboard 
Print this post

Here is what it looks like:

Yes, maybe I used some trickery to get the characters into the DATA statements in the first place, but that is my prerogative. Yes, screwy things could happen when doing this but that is for me as the programmer to take care of. I just want a READ statement that reads exactly what's there and lets me mess up on my own terms.  
Visit Vegipete's *Mite Library for cool programs.
 
matherp
Guru

Joined: 11/12/2012
Location: United Kingdom
Posts: 8578
Posted: 06:42pm 31 Dec 2020
Copy link to clipboard 
Print this post

Sorry - but as per my post this is standard across all versions of MMbasic in Geoff's code and not something I'm going to change
 
vegipete

Guru

Joined: 29/01/2013
Location: Canada
Posts: 1082
Posted: 06:47pm 31 Dec 2020
Copy link to clipboard 
Print this post

That tokeniser code doesn't explain what's happening.

The '& 0x7F' should strip the high bit, resulting in the normal characters appearing twice.
Visit Vegipete's *Mite Library for cool programs.
 
vegipete

Guru

Joined: 29/01/2013
Location: Canada
Posts: 1082
Posted: 06:48pm 31 Dec 2020
Copy link to clipboard 
Print this post

MMBasic standard. Fair enough, it is what it is.
Visit Vegipete's *Mite Library for cool programs.
 
TassyJim

Guru

Joined: 07/08/2011
Location: Australia
Posts: 5901
Posted: 09:19pm 31 Dec 2020
Copy link to clipboard 
Print this post

The simplest way to achieve what you want is to store the characters as HEX$ or c style escape sequences.

Jim
VK7JH
MMedit   MMBasic Help
 
Geoffg

Guru

Joined: 06/06/2011
Location: Australia
Posts: 3165
Posted: 06:15am 01 Jan 2021
Copy link to clipboard 
Print this post

You will never be able to embed characters with the high bit (&H80) set into the text of a program.  This is because the preprocessor in MMBasic converts keywords (PRINT, FOR, etc) into single byte tokens with the high bit set.  When MMBasic is running a program it can therefor quickly determine if a byte is a keyword or not by checking the high bit.

To prevent the interpreter being confused by someone using characters with the high bit set the preprocessor deliberately clears the high bit of the program text when it is loaded.  That way, when the program is saved to flash, only tokenised keywords can have this bit set.

Converting keywords into single byte tokens returns a significant performance benefit and therefore is not something that I would want to abandon.  So, if you want to use a character with the high bit set you will have to create it using CHR$().

Geoff
Geoff Graham - http://geoffg.net
 
Grogster

Admin Group

Joined: 31/12/2012
Location: New Zealand
Posts: 9061
Posted: 10:11am 02 Jan 2021
Copy link to clipboard 
Print this post

Don't try to create a font from data statements.
Use the built-in fonts, or define a new one with DefineFont #Nbr.

In the CMM2, fonts 1 and 4 have the extended character set you seem to be trying to create via data statements...

  Quote  All I want to do is put these characters directly in DATA statements precisely so I don't have to horse around with CHR$


Why?
CHR$ is there for precisely this reason.  I don't understand why you want to have the characters in DATA statements, when CHR$ is exceptionally easy to use - all you need is an ASCII table if you don't happen to have one handy and can't remember the numbers for the characters you want to use......

If your answer is along the lines of: 'I want to use the extended characters directly in my code', then that is simply not possible for the reasons stated by the members above.  You will HAVE to use CHR$ - no other way to do it.
Smoke makes things work. When the smoke gets out, it stops!
 
knivd

Regular Member

Joined: 07/09/2014
Location: United Kingdom
Posts: 71
Posted: 12:07pm 02 Jan 2021
Copy link to clipboard 
Print this post

  Grogster said  You will HAVE to use CHR$ - no other way to do it.


That is technically incorrect. See this (page 4).

This version was in fact even faster despite the 8-bit support. I have abandoned the work on it a long time ago due to various reasons, but the point is, it demonstrates that it is completely possible for such support to be done.
 
vegipete

Guru

Joined: 29/01/2013
Location: Canada
Posts: 1082
Posted: 09:53pm 02 Jan 2021
Copy link to clipboard 
Print this post

Note the following is only more explanation of what I am trying to do. High bit set for tokens explains why what I want isn't possible (although I have a thought on the matter at the end.)
========================

  Grogster said  Don't try to create a font from data statements.

I'm not sure where that came from. I specifically want to use the built-in fonts.

  Grogster said  
  Quote  All I want to do is put these characters directly in DATA statements precisely so I don't have to horse around with CHR$


Why?
Because WYSIWYG.

I'm adding a help page to a program nearing release. The following is a screen clip of a small portion of the help page:

My preference is to NOT require extra files. Distribution is in my opinion far simpler if a program is just a single, completely self contained file. So I would like to embed the information for the help page in the program.

A bunch of PRINT or TEXT commands could do this, but editing them is a royal pain. Adding a new line requires manually recalculating all the print offsets. Putting the text into data statements makes maintenance much easier.

But now consider how you would store the above help page clip.
data chr$(149)+chr$(146)+chr$(147)+chr$(148)+" to move."
sure doesn't work.

However, I have found a neat trick that does work.
Use the opening comments of the program as the help file!
The currently running program can open its own source file and read from it.

' Help File
'<^v> to move.  Hold Shift to select
'Even more help goes here.
'Press Ctrl-Q to quit
'END
'

open mm.info(current) for input as #1
do
 input #1, a$
 if a$ = "'END" then exit do
 print mid$(a$,2)  ' remove leading single quote character
loop
close #1
print
print "And there's your help file!"
end
Sorry, arrow characters get demolished by the forum software.

=============
My experimenting with MMBasic suggests that only EOL terminates a DATA statement. Once a DATA statement appears on a line, all following characters are data. So nothing following the DATA statement can be a token. This might allow the interpreter to accept high-bit set characters. Just a thought...
Visit Vegipete's *Mite Library for cool programs.
 
LeoNicolas

Guru

Joined: 07/10/2020
Location: Canada
Posts: 444
Posted: 11:28pm 02 Jan 2021
Copy link to clipboard 
Print this post

Vegipete
Is it not possible to load these data from a binary file?
 
TassyJim

Guru

Joined: 07/08/2011
Location: Australia
Posts: 5901
Posted: 01:18am 03 Jan 2021
Copy link to clipboard 
Print this post

  Quote  data chr$(149)+chr$(146)+chr$(147)+chr$(148)+" to move."

or
  Quote  data "\149\146\147\148 to move."


I haven't bothered to do the normal escape codes yet but they are easy enough to add to the function.

 READ hlp$
 hlp$ = e_str$(hlp$)
 CLS
 PRINT hlp$
 
DO:LOOP UNTIL INKEY$ <>""
END
 
FUNCTION e_str$(txt$)
 LOCAL ch$, n, k, s
 
 FOR n = 1 TO LEN(txt$)
   ch$ = MID$(txt$,n,1)
   SELECT CASE s
     CASE 0
       IF ch$ = "\" THEN
         s = 1
         k = 0
         c = 0
       ELSE
         e_str$ = e_str$ + ch$
       ENDIF
     CASE 1 ' we are in an escape sequence
       SELECT CASE ch$
         CASE "0","1","2","3","4","5","6","7","8","9"
           k = k*10 + VAL(ch$)
           c = c + 1
           IF c >=3 THEN
             e_str$ = e_str$ + CHR$(k)
             s = 0
           ENDIF
         CASE "\"
           e_str$ = e_str$ + ch$
           s = 0
       END SELECT
   END SELECT
 NEXT n
END FUNCTION
 
 
 DATA "\149\146\147\148 to move."


The code is very rough but hopefully you get the idea.
It makes the data statements readable without any nasty surprises.

Jim
Edited 2021-01-03 11:22 by TassyJim
VK7JH
MMedit   MMBasic Help
 
vegipete

Guru

Joined: 29/01/2013
Location: Canada
Posts: 1082
Posted: 02:27am 03 Jan 2021
Copy link to clipboard 
Print this post

  TassyJim said  The code is very rough but hopefully you get the idea.

Nice. Neat idea using the SELECT CASE to handle the state machine.

  TassyJim said  It makes the data statements readable without any nasty surprises.

Where's the fun in that?  
Visit Vegipete's *Mite Library for cool programs.
 
TassyJim

Guru

Joined: 07/08/2011
Location: Australia
Posts: 5901
Posted: 03:54am 03 Jan 2021
Copy link to clipboard 
Print this post

A few of the usual codes added and set to ignore ones it doesn't know about.
(Untested)

 READ hlp$
 hlp$ = e_str$(hlp$)
 CLS
 PRINT hlp$
 
DO:LOOP UNTIL INKEY$ <>""
END
 
FUNCTION e_str$(txt$)
 LOCAL ch$, n, k, s
 
 FOR n = 1 TO LEN(txt$)
   ch$ = MID$(txt$,n,1)
   SELECT CASE s
     CASE 0
       IF ch$ = "\" THEN
         s = 1
         k = 0
         c = 0
       ELSE
         e_str$ = e_str$ + ch$
       ENDIF
     CASE 1 ' we are in an escape sequence
       SELECT CASE ch$
         CASE "0","1","2","3","4","5","6","7","8","9" ' we expect a 3 digit decimal number
           k = k*10 + VAL(ch$)
           c = c + 1
           IF c >=3 THEN
             e_str$ = e_str$ + CHR$(k)
             s = 0
           ENDIF
         CASE "\"
           e_str$ = e_str$ + "\"      ' backslash
           s = 0
         case "b","B"
           e_str$ = e_str$ + chr$(8)  ' backspace
           s = 0
         case "t","T"
           e_str$ = e_str$ + chr$(9)  ' tab
           s = 0
         case "n","N"
           e_str$ = e_str$ + chr$(10) ' newline
           s = 0
         case "v","V"
           e_str$ = e_str$ + chr$(11) ' vertical tab
           s = 0
         case "f","F"
           e_str$ = e_str$ + chr$(12) ' formfeed
           s = 0
         case "r","R"
           e_str$ = e_str$ + chr$(13) ' return
           s = 0
         case "e","E"
           e_str$ = e_str$ + chr$(27) ' escape
           s = 0
         case "q","Q"
           e_str$ = e_str$ + chr$(34) ' double quote
           s = 0
         case else
           s = 0                      ' ignore unknown sequences
       END SELECT
   END SELECT
 NEXT n
END FUNCTION
 
 
 DATA "\149\146\147\148 to move."


Jim
VK7JH
MMedit   MMBasic Help
 
vegipete

Guru

Joined: 29/01/2013
Location: Canada
Posts: 1082
Posted: 04:32am 03 Jan 2021
Copy link to clipboard 
Print this post

This is working very well! (Also gives a hint of what is just about ready for release.)
 dim boxbeg(31)
 dim boxend(31)
 restore Help01
 y = 45
 font 4
 do
   read t$
   if t$ = "END" then
     exit do
   elseif t$ = "" then
     y = y + 8
   elseif t$ = "G" then  ' grayed out - not available
     read t$
     text 25,y,t$,,,1,&h303030
     y = y + 16
   else
     bs = 1
     bxn = 0
     escpos = instr(bs,t$,"\")    ' search for ESC sequences
     do while escpos
       select case mid$(t$,escpos+1,1)
         case "\"    ' double \\ means single one
           t$ = left$(t$,escpos-1) + mid$(t$,escpos+1)
           bs = escpos + 1
         case "0","1","2","3","4","5","6","7","8","9"
           ascval = min(255,max(32,val(mid$(t$,escpos+1,3))))
           t$ = left$(t$,escpos-1) + chr$(ascval) + mid$(t$,escpos+4)
         case "["    ' start box
           bxn = bxn + 1
           boxbeg(bxn) = (escpos-1)*mm.info(fontwidth) - 2
           t$ = left$(t$,escpos-1) + mid$(t$,escpos+2)
         case "]"    ' end box
           boxend(bxn) = (escpos)*mm.info(fontwidth) - boxbeg(bxn) - 9
           t$ = left$(t$,escpos-1) + mid$(t$,escpos+2)
         case "'"    ' change single to double quote
           t$ = left$(t$,escpos-1) + chr$(34) + mid$(t$,escpos+2)
       end select
       escpos = instr(bs,t$,"\")    ' search for more ESC sequences
     loop
     text 25,y,t$
     for i = 1 to bxn
       box 25 + boxbeg(i),y,boxend(i),mm.info(fontheight)
     next i
     y = y + 16
   endif
 loop
 end

Help01:
data "\[\149\] \[\146\] \[\147\] \[\148\] to move."
data "    Hold \[Shift\] to select, \[Ctrl\] to change cell size."
data "    Hold \[Alt\] to select rows or columns."
data "\[PgUp\] \[PgDn\] to move faster, hold \[Ctrl\] to move sideways faster."
data "\[Home\] to move to cell [A1]"
data ""
data "In general, \[Esc\] cancels the current operation."
data ""
data "Cells can hold 3 types of info: numbers, text or equations."
data "    To enter text, start with a quote character \[\'\]"
data "    To enter an equation, start with an equals sign \[=\]"
data "    A number starts with \[0\] \[1\] \[2\] \[3\] \[4\] \[5\] \[6\] \[7\] \[8\] \[9\] \[.\] \[+\] or \[-\]"
data "Press \[Alt\]-\[Enter\] to edit an existing cell"
data ""
data "When entering an equation, use \[\146\] or \[\147\] to switch to cell select mode."
data "    Choose cell with \[Enter\], \[ Space \] or )*+-/<=>\\^"
data "Cells are in [] brackets. Use $ for absolute reference."
data "In general, any MMBasic function can be used."
data "G","Sum(refcell to refcell) is also available."
data "\[Ctrl\]-\[K\] to recalculate entire spreadsheet."
data ""
data "\[Del\] and \[Backspace\] clear the highlighted cell or range."
data ""
data "For Cut, Copy and Paste, use \[Ctrl\]-\[X\], \[Ctrl\]-\[C\] and \[Ctrl\]-\[V\]."
data ""
data "Press \[Ctrl\]-\[Q\] to quit."
data ""
data "Press \[Ctrl\]-\[N\] to start a new file."
data "G","Press Ctrl-O to open a file."
data "G","Press Ctrl-S to save the current file."
data "END"

Visit Vegipete's *Mite Library for cool programs.
 
lizby
Guru

Joined: 17/05/2016
Location: United States
Posts: 3015
Posted: 02:11pm 03 Jan 2021
Copy link to clipboard 
Print this post

Well, a CMM2 spreadsheet. Too cool.

Glad you have worked out the IBMPC-type graphics. I tore my hair out working out a way to do that with DOS MMBasic & pi-cromite & H7/F4, including creating my own font (to work around the 8th bit clearing). I need to revisit now with these new ideas.

~
Edited 2021-01-04 00:31 by lizby
PicoMite, Armmite F4, SensorKits, MMBasic Hardware, Games, etc. on fruitoftheshed
 
Print this page


To reply to this topic, you need to log in.

© JAQ Software 2024