Home
JAQForum Ver 24.01
Log In or Join  
Active Topics
Local Time 23:42 23 Aug 2025 Privacy Policy
Jump to

Notice. New forum software under development. It's going to miss a few functions and look a bit ugly for a while, but I'm working on it full time now as the old forum was too unstable. Couple days, all good. If you notice any issues, please contact me.

Forum Index : Microcontroller and PC projects : Unexpected behaviour of =input$(1,#2)

Author Message
MustardMan

Senior Member

Joined: 30/08/2019
Location: Australia
Posts: 175
Posted: 11:32am 10 Nov 2020
Copy link to clipboard 
Print this post

Hi,

Hardware: Explore 100 (four hardware serial/UART ports).

I am having an issue with the INPUT function reading from a serial port. Baud rate of 38400, characters are 1mS apart.

I open the port with:
OPEN "COM2: 38400, 70, ComInt" AS #2

This should open port 2 (as #2 so it is easy to keep track of what is what) at 38400 with a buffer size of 70 bytes to generate an interrupt after a single character and vector to 'ComInt'.

The interrupt routine does the following:
s = INPUT$(1,#2)
Frame = Frame + s
IF s = CHR$(0) THEN Flag = 1

There is other (minimal) processing inside the interrupt to ensure the string does not get too long, etc.

Outside the interrupt, if Flag = 1 the string is processed and set back to zero size.

Looks good, but drops characters.

It looks as if things get busy and an interrupt occurs when there is more than one character in the buffer, only one character is read out and the rest are simply discarded!

For example, the string ABCDEFGHIJKLMNOPQRTSUV (22 characters) is sent. It is quiet and I get ABCDE, but then things get busy and characters start getting dropped, so I end up with a string ABCDEJKLMQRTUV (eg: 14 characters)

From my understanding this should not happen. The buffer may grow, but characters should never be dropped!

Say after "E" two characters come in, "FG". An interrupt is caused by "F", then the INPUT$(1,#2) should get "F" leaving "G" in the buffer, and the next interrupt (caused by "H") should grab "G"... but it does not appear to do so... it just drops "G" completely and returns "H" the next time an interrupt occurs.

Yes, I can see that such an occurrence may cause the buffer defined in the OPEN command to grow until it overflows, but I can can deal with that with LOC(#2).

According to the Micromite User Manual, INPUT$(nbr,[#]fnbr) "... will return as many characters as are waiting in the receive buffer up to 'nbr'". It says nothing about discarding characters if there are more than nbr.

Is this a bug?

Cheers,
 
Geoffg

Guru

Joined: 06/06/2011
Location: Australia
Posts: 3295
Posted: 02:01pm 10 Nov 2020
Copy link to clipboard 
Print this post

No the INPUT$() function does not discard characters and there is no bug.  This part of MMBasic has been very thoroughly tested and is used in many hundreds of applications by many people.

I don't know what your program is doing but interrupting on every character at 38400 baud means that your program will be continuously tied up in interrupts with little time to do anything else.  To make it worse the buffer is set to just 70 characters so it will be overrun almost immediately - and that is probably where characters are being lost.

At 38400 characters arrive very quickly and that needs well thought out code to handle that rate.
Geoff Graham - http://geoffg.net
 
MustardMan

Senior Member

Joined: 30/08/2019
Location: Australia
Posts: 175
Posted: 08:10pm 10 Nov 2020
Copy link to clipboard 
Print this post

Thanks for the reply Geoff.

I didn't think it would be a bug, as you say, it is something reasonably well used in embedded situations, and likely to have been tested to death.

I will look further into my problem. Though it'll probably be a week or two before I can report back at the rate I'm able to proceed!

Cheers.
 
lizby
Guru

Joined: 17/05/2016
Location: United States
Posts: 3396
Posted: 09:07pm 10 Nov 2020
Copy link to clipboard 
Print this post

A typical process is to use a big enough buffer to accept your input in the background, and then in the main loop use loc(f#) to determine if anything is waiting and to process it. Or interrupt only upon receipt of carriage return or other terminating character.
PicoMite, Armmite F4, SensorKits, MMBasic Hardware, Games, etc. on fruitoftheshed
 
MustardMan

Senior Member

Joined: 30/08/2019
Location: Australia
Posts: 175
Posted: 03:31am 14 Nov 2020
Copy link to clipboard 
Print this post

Hi,

To report back (in case anyone searches in the future and reads this)...

My problem turns out to be a problem that is related to my recent post regarding when an interrupt is serviced Line/statement execution and Interrupts

My serial port is configured to generate an interrupt every byte [OPEN "COM2: 38400, size, ComISR"].
I need to interrupt every byte to check the stream for special characters.

An update to MMbasic (v5.05.03) allows for interrupt on character, but only one specific character (it could not, for example, interrupt on all the control characters). It was close, but not the panacea I was hoping for.

What was happening is that the program would be processing a command, a byte would arrive, an interrupt would be queued for when the command finished, but another byte would come in during that time. The interrupt does not count how many interrupts are queued, only that there is an outstanding interrupt to be serviced.

As a result the internal buffer would slowly grow longer and longer as these 'extra' bytes arrived, eventually overflowing and dropping characters.

To deal with the fact that the ISR only gets called once even if there is a queue of non-serviced interrupts, I did the following (like lisby suggests, but inside the ISR):

Pseudocode
SUB ComISR
DO WHILE LOC(#2)   ' LOC returns the number of bytes waiting
 ch$ = INPUT$(1,#2)   ' Get single character
 [check for control characters, set flags as required]
 buf$ = buf$ + ch$
LOOP
END SUB

This also grabs characters that arrive while the ISR is executing.

Cheers,
Edited 2020-11-14 13:34 by MustardMan
 
Grogster

Admin Group

Joined: 31/12/2012
Location: New Zealand
Posts: 9629
Posted: 05:35am 14 Nov 2020
Copy link to clipboard 
Print this post

  lizby said  A typical process is to use a big enough buffer to accept your input in the background, and then in the main loop use loc(f#) to determine if anything is waiting and to process it. Or interrupt only upon receipt of carriage return or other terminating character.


Agreed.  I do it EXACTLY that way.  I don't bother with interrupts on the serial port buffer, in order to read the data.  You should not process ANYTHING inside an interrupt - just set flags, and exit to keep the interrupt as short as possible.  Processing of what is in the buffer, should be done in your main loop.

1) Open the COM port
2) DON'T use the interrupt - the buffer will AUTOMATICALLY save any bytes received in the background.
3) Use LOC in your main-loop, to find out if there is anything in the buffer to process.

An example:


OPEN "COM2:38400" as #2 'Open COM port, default buffer of 256 bytes

...

DO 'Primary loop - EVERYTHING is inside this one...
 DO 'Secondary loop - most actions happen inside this one....
   ...
   ...
   If LOC(#2) then 'There is something in the COM port buffer
     D$=Input$(LOC(#2),#2) 'Suck everything out of the buffer and store it in D$
     IF LEN(D$)<>25 then
       Print "Data length error!"
       Exit Do
     Endif
     CMD$=RIGHT$(D$,5) 'LAST five bytes of string are the command.
     DTA$=LEFT$(D$,20) 'FIRST twenty bytes are the data
   
   Endif
   ...
   ...
 LOOP
LOOP


You can check LOC anytime you like, and if it is NOT zero, then something has arrived while the main loop was doing other things, and so you can detect that there, and branch off and process the data.

Another example of how to do this, is kinda like in your pseudo code, where you read the buffer a byte at a time, looking for an EOM(end of message) marker byte, which has to be unique - I often use Escape, 27 decimal.

An example of the above loops with this method:


OPEN "COM2:38400" as #2 'Open COM port, default buffer of 256 bytes

...

DO 'Primary loop - EVERYTHING is inside this one...
 DO 'Secondary loop - most actions happen inside this one....
   ...
   ...
   If LOC(#2) then 'There is something in the COM port buffer
     DO
       X$=Input$(1,#2) 'Suck ONE byte out of the buffer and store it in X$
       IF X$=CHR$(27) then Exit Do 'EOM marker byte detected, so hop out of here...
       D$=D$+X$
     LOOP
     IF LEN(D$)<>25 then
       Print "Data length error!"
       Exit Do
     Endif
     CMD$=RIGHT$(D$,5) 'LAST five bytes of string are the command.
     DTA$=LEFT$(D$,20) 'FIRST twenty bytes are the data
   
   Endif
   ...
   ...
 LOOP
LOOP


This method can be extraordinarily helpful, as then the serial port buffer can actually become the message queue, storing multiple messages, and the main loop will read each message out of the buffer one at a time(hopping out when it detects an EOM byte), process that message, then loop around again and keep doing that till the buffer is empty.  The serial port buffer is a beautiful thing, once you get the hang of how it works and how it stores anything in the background, so your main program can do whatever, and just check the buffer as part of the program, safe in the knowledge that the buffer will AUTOMATICALLY save and queue up all the messages for you.

If you expect lots of messages, I would set the buffer to 512 bytes or even 1024 bytes perhaps - depends on your message packet length, and how often you are expecting them to arrive, vs how much time the main loop takes to complete.
Edited 2020-11-14 15:38 by Grogster
Smoke makes things work. When the smoke gets out, it stops!
 
JohnS
Guru

Joined: 18/11/2011
Location: United Kingdom
Posts: 4067
Posted: 09:46am 14 Nov 2020
Copy link to clipboard 
Print this post

Just to check: are both those examples real code because if they are they don't look to work as desired:

1st one: unlikely to be length 25 so will moan

2nd: the DO LOOP with check for 27 will wait until 27 arrives, blocking the main program

John
 
MustardMan

Senior Member

Joined: 30/08/2019
Location: Australia
Posts: 175
Posted: 10:29am 14 Nov 2020
Copy link to clipboard 
Print this post

@Grogster

Example number two is actually really close to what my code has to do, but your code does away with needing an interrupt entirely (and the complications that ensue).

In my code I have a soft-keypad that I have to scan and respond to as well, so I would have to incorporate the keypad scan inside the inner (tertiary) loop as it will block until the EOM marker is found, causing likely sluggish response. I have little doubt that is a modification that could be done with some careful thought.

I very much like your code, and it certainly spurs me on into ditching a COM interrupt altogether and just checking LOC in 'main'.

Cheers!
 
lizby
Guru

Joined: 17/05/2016
Location: United States
Posts: 3396
Posted: 01:17pm 14 Nov 2020
Copy link to clipboard 
Print this post

If you interrupt on EOM marker, there is no reason to block the main loop. The interrupt doesn't need to do anything more than increment a counter of the number of EOM markers. The main loop checks for counter > 0 and handles messages, decrementing the counter for each message processed.
PicoMite, Armmite F4, SensorKits, MMBasic Hardware, Games, etc. on fruitoftheshed
 
MustardMan

Senior Member

Joined: 30/08/2019
Location: Australia
Posts: 175
Posted: 07:58pm 14 Nov 2020
Copy link to clipboard 
Print this post

Went to bed last night thinking about it, woke up this-morning, and with virtually no changes to the code itself removed the need for the COM port interrupt entirely!

Wow! That is so neat... thanks for the tip Grogster!

I simply copied/pasted the code out of my ISR into the main loop body.

The only alterations I needed to make were changing some variable names as I was using STATICs in my ISR. DIMed them as globals, and it worked after two syntax errors (both not declared variables).

The difference with my code and the sample provided by Grogster is rather than
If LOC(#2) then 'There is something in the COM port buffer
  DO
     X$=Input$(1,#2) 'Suck ONE byte out of the buffer and store it in X$
     IF X$=CHR$(27) then Exit Do 'EOM marker byte detected, so hop out of here...
     D$=D$+X$
  LOOP
which blocks until eom is found

I do
DO WHILE LOC(#2) THEN 'There is something in the COM port buffer
  X$=INPUT$(1,#2) 'Suck ONE byte out of the buffer and store it in X$
  IF X$=CHR$(27) THEN eom = eom + 1
  D$=D$+X$
LOOP
which does not block and simply pushes an eom "flag" higher up the pole (incrementing a counter allows us to queue multiple messages)

And later
IF eom > 0 THEN ...process... : eom = eom - 1


@Lizby
I thought about using the built-in OPEN interrupt facility to increment eom, but because interrupts are non-syncronous (by definition), there is a high likelyhood that eom would get incremented before the code in the main loop had sucked all the preceding bytes from the buffer.
Depends on the intent of the program... do we need to do something the instant eom arrives?
Or do we need our message up to eom to be complete and intact?

Cheers,
Edited 2020-11-15 06:09 by MustardMan
 
Grogster

Admin Group

Joined: 31/12/2012
Location: New Zealand
Posts: 9629
Posted: 11:13pm 15 Nov 2020
Copy link to clipboard 
Print this post

  JohnS said  Just to check: are both those examples real code because if they are they don't look to work as desired:

1st one: unlikely to be length 25 so will moan

2nd: the DO LOOP with check for 27 will wait until 27 arrives, blocking the main program

John


Of course.  They are only EXAMPLES.  I have made no allowance for data-length errors other then 25, or the ESC byte not being present etc - you are correct.  They are NOT running excerpts, just examples written off the top of my head, but it shows the basic idea behind processing the serial port buffer inside the main loop.  Extra data filtering and allowance for strings growing to overflow size if no ESC character received etc will be needed, but as I said - these are just examples to give the OP an example of how you do it like this.
Edited 2020-11-16 09:18 by Grogster
Smoke makes things work. When the smoke gets out, it stops!
 
Grogster

Admin Group

Joined: 31/12/2012
Location: New Zealand
Posts: 9629
Posted: 11:46pm 15 Nov 2020
Copy link to clipboard 
Print this post

@ MustardMan: Some extra strength can be built into the example routines I posted, to keep JohnS happy.

(no offense John, your points are valid - i'm just ribbing ya!)


OPEN "COM2:38400" as #2 'Open COM port, default buffer of 256 bytes

...

DO 'Primary loop - EVERYTHING is inside this one...
DO 'Secondary loop - most actions happen inside this one....
  ...
  ...
  If LOC(#2) then 'There is something in the COM port buffer
    TIMER=0 'Reset the system timer
    D$="" 'Clear D$ for this time through the loop
    DO
      X$=Input$(1,#2) 'Suck ONE byte out of the buffer and store it in X$
      IF X$=CHR$(27) then Exit Do 'EOM marker byte detected, so hop out of here...
      D$=D$+X$
    LOOP UNTIL TIMER>4000 'Exit if more then 4s has elapsed(no EOM detected)
    IF LEN(D$)<>25 then 'This will catch any short or long message
      Print "Data length error!"
      Exit Do
    Endif
    CMD$=RIGHT$(D$,5) 'LAST five bytes of string are the command.
    DTA$=LEFT$(D$,20) 'FIRST twenty bytes are the data
 
  Endif
  ...
  ...
LOOP
LOOP


Now the code will exit if the EOM byte is detected, but it will also exit if the system timer exceeds four seconds if no EOM byte is detected.

The reason I am using the whole IF X$=CHR$(27) then Exit Do rather then LOOP UNTIL X$=CHR$(27), is so that when an EOM byte is detected, it is NOT added to D$ as it is not technically part of the message, it is just the marker byte the code uses to separate out data packets for processing.

You COULD do it that way though, and then use LEFT$ to remove the last byte from the message(which would be the EOM byte).  In other words:


TIMER=0
DO
 X$=Input$(1,#2) 'Suck ONE byte out of the buffer and store it in X$
 D$=D$+X$
LOOP UNTIL X$=CHR$(27) or TIMER>4000 'Exit if we have an EOM byte, or more then 4s has elapsed
DTA$=LEFT$(D$,(LEN(D$)-1)) 'Copy message into DTA$, dropping the EOM byte from the end


You then process what you have, and decide if you have a valid packet or not.
This might involve several tests on the data in D$ to validate it, or discard it as a rouge packet and get the next one etc.

In my example, I am simply using a 25-byte packet, with the first 20 bytes being data, and the last five being a command, and these are stripped out as part of the data filtering, into their own strings for processing further down the code(not shown here).  But they can be anything you like, I am just showing that as an example of how you can pull apart a valid data packet, to get the bytes you want into the strings you want.  There is also MID$ which is very useful, but I have not used it in this example - see the manual under FUNCTIONS.  Change the four-second timeout in the data sucking loop, to suit your needs.  Might need to be longer, might need to be shorter - again just an example JohnS!    
Edited 2020-11-16 09:58 by Grogster
Smoke makes things work. When the smoke gets out, it stops!
 
JohnS
Guru

Joined: 18/11/2011
Location: United Kingdom
Posts: 4067
Posted: 02:55pm 16 Nov 2020
Copy link to clipboard 
Print this post

No worries - keep the lessons coming lol

John
 
romba6

Newbie

Joined: 04/07/2020
Location: United Kingdom
Posts: 37
Posted: 05:47pm 16 Nov 2020
Copy link to clipboard 
Print this post

  JohnS said  No worries - keep the lessons coming lol

John

Thank you all for the help in explaining this. I am in the process of trying to do a VT100 terminal emulation - connected to my Z180 SC126 kit @ 9600 which is running CP/M.
For me, (a hardware guy) this is quite a challenge but the info here very helpful and I'm learning a lot!!! Why try this? - just because !!!
 
JohnS
Guru

Joined: 18/11/2011
Location: United Kingdom
Posts: 4067
Posted: 08:17pm 16 Nov 2020
Copy link to clipboard 
Print this post

  romba6 said  Why try this? - just because !!!

Excellent. That's as good a reason as any!

John
 
MustardMan

Senior Member

Joined: 30/08/2019
Location: Australia
Posts: 175
Posted: 07:37am 17 Nov 2020
Copy link to clipboard 
Print this post

A cut-down version of my code...
OPEN "COM2: 38400" AS #2                      ' RS485 Channel 2 (receive)

DO
 DO WHILE LOC(#2)                            ' If there are any characters in the UART buffer, loop to get them all out
   s = INPUT$(1,#2)                          '  (including any that come in while we are looping)
   IF s = Delimiter THEN                     ' Incoming data stream delimiter
     EOM = EOM + 1                           ' Counter of non-processed lines waiting for processing
   ENDIF
   PIN(TP35) = HIGH                          ' Debug: Pulse for every character
   IF LEN(Buffer) < MaxBuffer THEN           ' Discard data that would cause a buffer overflow
     Buffer = Buffer + s
   ELSE
     Overflow = Overflow + 1                 ' How many characters got lost (running total)
   ENDIF
   PIN(TP35) = LOW
 LOOP

 IF EOM > 0 THEN
   n = INSTR(Buffer, Delimiter)                ' Search for delimiter (0=not found, otherwise returns position starting at 1)
   IF n = 0 THEN
     IF (Debug AND &b00000010) THEN            ' Debug: buffer has garbage, discard it
       PRINT "Discarding <";Buffer;">"
     ENDIF
     Buffer = ""                               ' Delimiter not found, zero string and counter
     EOM = 0
   ELSE
     s = LEFT$(Buffer, n - 1)                  ' Get everything up to, but not including, the delimiter
     Buffer = MID$(Buffer, n + 1)              ' Shorten buffer. If the length is not specified, MID$ returns everything from 'n' onwards
     EOM = EOM - 1                             '  (n+1 dumps the delimiter)
     IF (Debug AND &b00000011) THEN            ' Debug: print recovered line
       PRINT " [";s;"]";
     ENDIF
   ENDIF
 ENDIF

 IF TOUCH(DOWN) OR (b > 0) THEN                      ' Something pressed (or something previously pressed)?
   ........
 ENDIF
LOOP

I like it because it does not "block" at all - it does not even wait for a pre-set timeout. Hence it responds to touch events really snappily, as well as responding snappily when a complete string arrives (determined by EOM > 0, when 'Delimiter' is found in the input stream).

Waiting for a timout is perfectly valid, but it will 'block' until you reach that timeout. You have to think about the requirements of your scenario, and build your code to suit.

Cheers,
 
Grogster

Admin Group

Joined: 31/12/2012
Location: New Zealand
Posts: 9629
Posted: 08:37am 17 Nov 2020
Copy link to clipboard 
Print this post

  MustardMan said  You have to think about the requirements of your scenario, and build your code to suit.

Cheers,


Could not agree more.
Well done, nice looking code.
Smoke makes things work. When the smoke gets out, it stops!
 
JohnS
Guru

Joined: 18/11/2011
Location: United Kingdom
Posts: 4067
Posted: 10:30am 17 Nov 2020
Copy link to clipboard 
Print this post

I haven't run the code but it looks appropriate and so much easier to debug :)

John
 
Print this page


To reply to this topic, you need to log in.

The Back Shed's forum code is written, and hosted, in Australia.
© JAQ Software 2025