![]() |
Forum Index : Microcontroller and PC projects : Unexpected behaviour of =input$(1,#2)
Author | Message | ||||
MustardMan![]() Senior Member ![]() Joined: 30/08/2019 Location: AustraliaPosts: 175 |
Hi, Hardware: Explore 100 (four hardware serial/UART ports). I am having an issue with the INPUT function reading from a serial port. Baud rate of 38400, characters are 1mS apart. I open the port with: OPEN "COM2: 38400, 70, ComInt" AS #2 This should open port 2 (as #2 so it is easy to keep track of what is what) at 38400 with a buffer size of 70 bytes to generate an interrupt after a single character and vector to 'ComInt'. The interrupt routine does the following: s = INPUT$(1,#2) Frame = Frame + s IF s = CHR$(0) THEN Flag = 1 There is other (minimal) processing inside the interrupt to ensure the string does not get too long, etc. Outside the interrupt, if Flag = 1 the string is processed and set back to zero size. Looks good, but drops characters. It looks as if things get busy and an interrupt occurs when there is more than one character in the buffer, only one character is read out and the rest are simply discarded! For example, the string ABCDEFGHIJKLMNOPQRTSUV (22 characters) is sent. It is quiet and I get ABCDE, but then things get busy and characters start getting dropped, so I end up with a string ABCDEJKLMQRTUV (eg: 14 characters) From my understanding this should not happen. The buffer may grow, but characters should never be dropped! Say after "E" two characters come in, "FG". An interrupt is caused by "F", then the INPUT$(1,#2) should get "F" leaving "G" in the buffer, and the next interrupt (caused by "H") should grab "G"... but it does not appear to do so... it just drops "G" completely and returns "H" the next time an interrupt occurs. Yes, I can see that such an occurrence may cause the buffer defined in the OPEN command to grow until it overflows, but I can can deal with that with LOC(#2). According to the Micromite User Manual, INPUT$(nbr,[#]fnbr) "... will return as many characters as are waiting in the receive buffer up to 'nbr'". It says nothing about discarding characters if there are more than nbr. Is this a bug? Cheers, |
||||
Geoffg![]() Guru ![]() Joined: 06/06/2011 Location: AustraliaPosts: 3295 |
No the INPUT$() function does not discard characters and there is no bug. This part of MMBasic has been very thoroughly tested and is used in many hundreds of applications by many people. I don't know what your program is doing but interrupting on every character at 38400 baud means that your program will be continuously tied up in interrupts with little time to do anything else. To make it worse the buffer is set to just 70 characters so it will be overrun almost immediately - and that is probably where characters are being lost. At 38400 characters arrive very quickly and that needs well thought out code to handle that rate. Geoff Graham - http://geoffg.net |
||||
MustardMan![]() Senior Member ![]() Joined: 30/08/2019 Location: AustraliaPosts: 175 |
Thanks for the reply Geoff. I didn't think it would be a bug, as you say, it is something reasonably well used in embedded situations, and likely to have been tested to death. I will look further into my problem. Though it'll probably be a week or two before I can report back at the rate I'm able to proceed! Cheers. |
||||
lizby Guru ![]() Joined: 17/05/2016 Location: United StatesPosts: 3396 |
A typical process is to use a big enough buffer to accept your input in the background, and then in the main loop use loc(f#) to determine if anything is waiting and to process it. Or interrupt only upon receipt of carriage return or other terminating character. PicoMite, Armmite F4, SensorKits, MMBasic Hardware, Games, etc. on fruitoftheshed |
||||
MustardMan![]() Senior Member ![]() Joined: 30/08/2019 Location: AustraliaPosts: 175 |
Hi, To report back (in case anyone searches in the future and reads this)... My problem turns out to be a problem that is related to my recent post regarding when an interrupt is serviced Line/statement execution and Interrupts My serial port is configured to generate an interrupt every byte [OPEN "COM2: 38400, size, ComISR"]. I need to interrupt every byte to check the stream for special characters. An update to MMbasic (v5.05.03) allows for interrupt on character, but only one specific character (it could not, for example, interrupt on all the control characters). It was close, but not the panacea I was hoping for. What was happening is that the program would be processing a command, a byte would arrive, an interrupt would be queued for when the command finished, but another byte would come in during that time. The interrupt does not count how many interrupts are queued, only that there is an outstanding interrupt to be serviced. As a result the internal buffer would slowly grow longer and longer as these 'extra' bytes arrived, eventually overflowing and dropping characters. To deal with the fact that the ISR only gets called once even if there is a queue of non-serviced interrupts, I did the following (like lisby suggests, but inside the ISR): Pseudocode SUB ComISR DO WHILE LOC(#2) ' LOC returns the number of bytes waiting ch$ = INPUT$(1,#2) ' Get single character [check for control characters, set flags as required] buf$ = buf$ + ch$ LOOP END SUB This also grabs characters that arrive while the ISR is executing. Cheers, Edited 2020-11-14 13:34 by MustardMan |
||||
Grogster![]() Admin Group ![]() Joined: 31/12/2012 Location: New ZealandPosts: 9629 |
Agreed. I do it EXACTLY that way. I don't bother with interrupts on the serial port buffer, in order to read the data. You should not process ANYTHING inside an interrupt - just set flags, and exit to keep the interrupt as short as possible. Processing of what is in the buffer, should be done in your main loop. 1) Open the COM port 2) DON'T use the interrupt - the buffer will AUTOMATICALLY save any bytes received in the background. 3) Use LOC in your main-loop, to find out if there is anything in the buffer to process. An example: OPEN "COM2:38400" as #2 'Open COM port, default buffer of 256 bytes ... DO 'Primary loop - EVERYTHING is inside this one... DO 'Secondary loop - most actions happen inside this one.... ... ... If LOC(#2) then 'There is something in the COM port buffer D$=Input$(LOC(#2),#2) 'Suck everything out of the buffer and store it in D$ IF LEN(D$)<>25 then Print "Data length error!" Exit Do Endif CMD$=RIGHT$(D$,5) 'LAST five bytes of string are the command. DTA$=LEFT$(D$,20) 'FIRST twenty bytes are the data Endif ... ... LOOP LOOP You can check LOC anytime you like, and if it is NOT zero, then something has arrived while the main loop was doing other things, and so you can detect that there, and branch off and process the data. Another example of how to do this, is kinda like in your pseudo code, where you read the buffer a byte at a time, looking for an EOM(end of message) marker byte, which has to be unique - I often use Escape, 27 decimal. An example of the above loops with this method: OPEN "COM2:38400" as #2 'Open COM port, default buffer of 256 bytes ... DO 'Primary loop - EVERYTHING is inside this one... DO 'Secondary loop - most actions happen inside this one.... ... ... If LOC(#2) then 'There is something in the COM port buffer DO X$=Input$(1,#2) 'Suck ONE byte out of the buffer and store it in X$ IF X$=CHR$(27) then Exit Do 'EOM marker byte detected, so hop out of here... D$=D$+X$ LOOP IF LEN(D$)<>25 then Print "Data length error!" Exit Do Endif CMD$=RIGHT$(D$,5) 'LAST five bytes of string are the command. DTA$=LEFT$(D$,20) 'FIRST twenty bytes are the data Endif ... ... LOOP LOOP This method can be extraordinarily helpful, as then the serial port buffer can actually become the message queue, storing multiple messages, and the main loop will read each message out of the buffer one at a time(hopping out when it detects an EOM byte), process that message, then loop around again and keep doing that till the buffer is empty. The serial port buffer is a beautiful thing, once you get the hang of how it works and how it stores anything in the background, so your main program can do whatever, and just check the buffer as part of the program, safe in the knowledge that the buffer will AUTOMATICALLY save and queue up all the messages for you. ![]() If you expect lots of messages, I would set the buffer to 512 bytes or even 1024 bytes perhaps - depends on your message packet length, and how often you are expecting them to arrive, vs how much time the main loop takes to complete. Edited 2020-11-14 15:38 by Grogster Smoke makes things work. When the smoke gets out, it stops! |
||||
JohnS Guru ![]() Joined: 18/11/2011 Location: United KingdomPosts: 4067 |
Just to check: are both those examples real code because if they are they don't look to work as desired: 1st one: unlikely to be length 25 so will moan 2nd: the DO LOOP with check for 27 will wait until 27 arrives, blocking the main program John |
||||
MustardMan![]() Senior Member ![]() Joined: 30/08/2019 Location: AustraliaPosts: 175 |
@Grogster Example number two is actually really close to what my code has to do, but your code does away with needing an interrupt entirely (and the complications that ensue). In my code I have a soft-keypad that I have to scan and respond to as well, so I would have to incorporate the keypad scan inside the inner (tertiary) loop as it will block until the EOM marker is found, causing likely sluggish response. I have little doubt that is a modification that could be done with some careful thought. I very much like your code, and it certainly spurs me on into ditching a COM interrupt altogether and just checking LOC in 'main'. Cheers! |
||||
lizby Guru ![]() Joined: 17/05/2016 Location: United StatesPosts: 3396 |
If you interrupt on EOM marker, there is no reason to block the main loop. The interrupt doesn't need to do anything more than increment a counter of the number of EOM markers. The main loop checks for counter > 0 and handles messages, decrementing the counter for each message processed. PicoMite, Armmite F4, SensorKits, MMBasic Hardware, Games, etc. on fruitoftheshed |
||||
MustardMan![]() Senior Member ![]() Joined: 30/08/2019 Location: AustraliaPosts: 175 |
Went to bed last night thinking about it, woke up this-morning, and with virtually no changes to the code itself removed the need for the COM port interrupt entirely! Wow! That is so neat... thanks for the tip Grogster! I simply copied/pasted the code out of my ISR into the main loop body. The only alterations I needed to make were changing some variable names as I was using STATICs in my ISR. DIMed them as globals, and it worked after two syntax errors (both not declared variables). The difference with my code and the sample provided by Grogster is rather than If LOC(#2) then 'There is something in the COM port buffer which blocks until eom is foundDO X$=Input$(1,#2) 'Suck ONE byte out of the buffer and store it in X$ IF X$=CHR$(27) then Exit Do 'EOM marker byte detected, so hop out of here... D$=D$+X$ LOOP I do DO WHILE LOC(#2) THEN 'There is something in the COM port buffer which does not block and simply pushes an eom "flag" higher up the pole (incrementing a counter allows us to queue multiple messages)X$=INPUT$(1,#2) 'Suck ONE byte out of the buffer and store it in X$ IF X$=CHR$(27) THEN eom = eom + 1 D$=D$+X$ LOOP And later IF eom > 0 THEN ...process... : eom = eom - 1 @Lizby I thought about using the built-in OPEN interrupt facility to increment eom, but because interrupts are non-syncronous (by definition), there is a high likelyhood that eom would get incremented before the code in the main loop had sucked all the preceding bytes from the buffer. Depends on the intent of the program... do we need to do something the instant eom arrives? Or do we need our message up to eom to be complete and intact? Cheers, Edited 2020-11-15 06:09 by MustardMan |
||||
Grogster![]() Admin Group ![]() Joined: 31/12/2012 Location: New ZealandPosts: 9629 |
Of course. They are only EXAMPLES. I have made no allowance for data-length errors other then 25, or the ESC byte not being present etc - you are correct. They are NOT running excerpts, just examples written off the top of my head, but it shows the basic idea behind processing the serial port buffer inside the main loop. Extra data filtering and allowance for strings growing to overflow size if no ESC character received etc will be needed, but as I said - these are just examples to give the OP an example of how you do it like this. ![]() Edited 2020-11-16 09:18 by Grogster Smoke makes things work. When the smoke gets out, it stops! |
||||
Grogster![]() Admin Group ![]() Joined: 31/12/2012 Location: New ZealandPosts: 9629 |
@ MustardMan: Some extra strength can be built into the example routines I posted, to keep JohnS happy. ![]() (no offense John, your points are valid - i'm just ribbing ya!) ![]() OPEN "COM2:38400" as #2 'Open COM port, default buffer of 256 bytes ... DO 'Primary loop - EVERYTHING is inside this one... DO 'Secondary loop - most actions happen inside this one.... ... ... If LOC(#2) then 'There is something in the COM port buffer TIMER=0 'Reset the system timer D$="" 'Clear D$ for this time through the loop DO X$=Input$(1,#2) 'Suck ONE byte out of the buffer and store it in X$ IF X$=CHR$(27) then Exit Do 'EOM marker byte detected, so hop out of here... D$=D$+X$ LOOP UNTIL TIMER>4000 'Exit if more then 4s has elapsed(no EOM detected) IF LEN(D$)<>25 then 'This will catch any short or long message Print "Data length error!" Exit Do Endif CMD$=RIGHT$(D$,5) 'LAST five bytes of string are the command. DTA$=LEFT$(D$,20) 'FIRST twenty bytes are the data Endif ... ... LOOP LOOP Now the code will exit if the EOM byte is detected, but it will also exit if the system timer exceeds four seconds if no EOM byte is detected. The reason I am using the whole IF X$=CHR$(27) then Exit Do rather then LOOP UNTIL X$=CHR$(27), is so that when an EOM byte is detected, it is NOT added to D$ as it is not technically part of the message, it is just the marker byte the code uses to separate out data packets for processing. You COULD do it that way though, and then use LEFT$ to remove the last byte from the message(which would be the EOM byte). In other words: TIMER=0 DO X$=Input$(1,#2) 'Suck ONE byte out of the buffer and store it in X$ D$=D$+X$ LOOP UNTIL X$=CHR$(27) or TIMER>4000 'Exit if we have an EOM byte, or more then 4s has elapsed DTA$=LEFT$(D$,(LEN(D$)-1)) 'Copy message into DTA$, dropping the EOM byte from the end You then process what you have, and decide if you have a valid packet or not. This might involve several tests on the data in D$ to validate it, or discard it as a rouge packet and get the next one etc. In my example, I am simply using a 25-byte packet, with the first 20 bytes being data, and the last five being a command, and these are stripped out as part of the data filtering, into their own strings for processing further down the code(not shown here). But they can be anything you like, I am just showing that as an example of how you can pull apart a valid data packet, to get the bytes you want into the strings you want. There is also MID$ which is very useful, but I have not used it in this example - see the manual under FUNCTIONS. Change the four-second timeout in the data sucking loop, to suit your needs. Might need to be longer, might need to be shorter - again just an example JohnS! ![]() ![]() Edited 2020-11-16 09:58 by Grogster Smoke makes things work. When the smoke gets out, it stops! |
||||
JohnS Guru ![]() Joined: 18/11/2011 Location: United KingdomPosts: 4067 |
No worries - keep the lessons coming lol John |
||||
romba6![]() Newbie ![]() Joined: 04/07/2020 Location: United KingdomPosts: 37 |
Thank you all for the help in explaining this. I am in the process of trying to do a VT100 terminal emulation - connected to my Z180 SC126 kit @ 9600 which is running CP/M. For me, (a hardware guy) this is quite a challenge but the info here very helpful and I'm learning a lot!!! Why try this? - just because !!! |
||||
JohnS Guru ![]() Joined: 18/11/2011 Location: United KingdomPosts: 4067 |
Excellent. That's as good a reason as any! John |
||||
MustardMan![]() Senior Member ![]() Joined: 30/08/2019 Location: AustraliaPosts: 175 |
A cut-down version of my code... OPEN "COM2: 38400" AS #2 ' RS485 Channel 2 (receive) DO DO WHILE LOC(#2) ' If there are any characters in the UART buffer, loop to get them all out s = INPUT$(1,#2) ' (including any that come in while we are looping) IF s = Delimiter THEN ' Incoming data stream delimiter EOM = EOM + 1 ' Counter of non-processed lines waiting for processing ENDIF PIN(TP35) = HIGH ' Debug: Pulse for every character IF LEN(Buffer) < MaxBuffer THEN ' Discard data that would cause a buffer overflow Buffer = Buffer + s ELSE Overflow = Overflow + 1 ' How many characters got lost (running total) ENDIF PIN(TP35) = LOW LOOP IF EOM > 0 THEN n = INSTR(Buffer, Delimiter) ' Search for delimiter (0=not found, otherwise returns position starting at 1) IF n = 0 THEN IF (Debug AND &b00000010) THEN ' Debug: buffer has garbage, discard it PRINT "Discarding <";Buffer;">" ENDIF Buffer = "" ' Delimiter not found, zero string and counter EOM = 0 ELSE s = LEFT$(Buffer, n - 1) ' Get everything up to, but not including, the delimiter Buffer = MID$(Buffer, n + 1) ' Shorten buffer. If the length is not specified, MID$ returns everything from 'n' onwards EOM = EOM - 1 ' (n+1 dumps the delimiter) IF (Debug AND &b00000011) THEN ' Debug: print recovered line PRINT " [";s;"]"; ENDIF ENDIF ENDIF IF TOUCH(DOWN) OR (b > 0) THEN ' Something pressed (or something previously pressed)? ........ ENDIF LOOP I like it because it does not "block" at all - it does not even wait for a pre-set timeout. Hence it responds to touch events really snappily, as well as responding snappily when a complete string arrives (determined by EOM > 0, when 'Delimiter' is found in the input stream). Waiting for a timout is perfectly valid, but it will 'block' until you reach that timeout. You have to think about the requirements of your scenario, and build your code to suit. Cheers, |
||||
Grogster![]() Admin Group ![]() Joined: 31/12/2012 Location: New ZealandPosts: 9629 |
Could not agree more. ![]() Well done, nice looking code. Smoke makes things work. When the smoke gets out, it stops! |
||||
JohnS Guru ![]() Joined: 18/11/2011 Location: United KingdomPosts: 4067 |
I haven't run the code but it looks appropriate and so much easier to debug :) John |
||||
![]() |
![]() |
The Back Shed's forum code is written, and hosted, in Australia. | © JAQ Software 2025 |