 Notice. New forum software under development. It's going to miss a few functions and look a bit ugly for a while, but I'm working on it full time now as the old forum was too unstable. Couple days, all good. If you notice any issues, please contact me. |
Forum Index : Microcontroller and PC projects : MMBasic Serial Coms enhancement suggest
| Author | Message | ||||
| erbp Senior Member  Joined: 03/05/2016 Location: AustraliaPosts: 195 |
I guess this is one for Geoff: Would it be possible to enhance the MMBasic Serial Communication handling to provide as an alternative to the current ability to specify a number of characters that must be received before an interrupt is triggered, that a specific "End of Message Sentinel" character could be specified and the interrupt would then be triggered when that character is transferred by the firmware from the UART Hardware Rx buffer to the Firmware Rx buffer. This would make handling variable length data messages much easier and more efficient than at present. As an aid to managing the integrity of received messages, if I have control over the transmitting system, I always "wrap" each message in specific "Start of Message" and "End of Message" sentinel characters. My preference is the STX (Start Of Text) character - CHR$(02) for the Start of Message sentinel and the ETX (End Of Text) character - CHR$(03) for the End of Message sentinel, but any suitable control characters can be used. For the above mentioned enhancement, only the End of Message sentinel is required. If my reading of the UART specs for the PIC32's is correct, the UART hardware receive buffer is quite small, so the firmware must be frequently handling transfer of the received characters out of the UART buffer into the firmware Rx buffer. If it could check the received characters on each such transfer looking for a match on the specified EOM sentinel, then raising the interrupt without that adding too much overhead to the firmware interrupt handler that would be super. To keep overhead to a minimal I think that it should be limited to specifying a SINGLE EOM character, not a string of characters that either need to be matched in full or matching any one of the characters in the string. If this can be implemented, I believe it could utilize the existing format of the 'comspec$' parameter of the Serial Port OPEN command. If the 'intlevel' parameter is upgraded to be 'intlevel/EOMchar', then if the value of this parameter is numeric it would select the existing option - interrupt raised when the specified number of characters have been received, otherwise if not numeric it could be interpreted as the EOM sentinel character to match on prior to raising the interrupt. Personally I would like to see the EOM character limited to a character from column 0 or 1 of the ASCII chart - i.e. a Control Character with hex value 00 thru 1F, but if it were determined to be more useful to allow any non-numeric character, then that would be OK too. Using the 'intlevel' parameter in this way would mean that compatibility with all existing programs would be maintained. I would be very interested to hear if this functionality could be implemented. It would certainly make life easier for me in a project I am currently working on and I think it would be useful in the wider context as well. Cheers, Phil. |
||||
Geoffg Guru  Joined: 06/06/2011 Location: AustraliaPosts: 3308 |
Wow, I just love it when someone comes up with a brilliant idea then, even better, goes on to figure out how the syntax could work. I was going to leave it for a month or two before I started on the next beta but this is a great idea and worth a beta to implement it. Give me a few days and I should have something. Geoff Geoff Graham - http://geoffg.net |
||||
goc30 Guru Joined: 12/04/2017 Location: FrancePosts: 435 |
hi erbp your idea is interresting but it may complicate the driver when using the COM port for pre-programmed external connections such as GPS, or Modbus in binary mode. And you do not take enough into account the sending info and its own problems In your case, I think the simplest way would be to make a basic driver on the model of what was done before. Your driver must just receive each byte via COM interrupt, and store it in a exchange table, then it update the status word of the exchange (beginning of reception, reception in progress, end of reception, error, etc.) Before, when we made drivers for COM port, we used a complete exchange table which made it possible to specify if you work in account of word or in stop code. it is this table that would be managed by your driver. For transmit problem, I have asked to have an interrupt on "buffer transmit empty". In this case, i can transmit each byte one by one with interrupt mode. |
||||
| JohnS Guru Joined: 18/11/2011 Location: United KingdomPosts: 4133 |
The syntax with a / feels less than obvious. A colon (:) instead maybe? One thing that strikes me is that someone using this may see the EOM and then do lots of work in the interrupt routine - exactly the worst sort of thing to encourage. Maybe at least a note in the manual not to do that? (I can't actually see this as a worthwhile feature at all, i.e. I'm with goc30, but that's up to Geoff of course.) John |
||||
| PicFan Senior Member Joined: 18/03/2014 Location: AustriaPosts: 133 |
Maybe this is a solution for you, I use it with some "wireless" applications and also with the console without any timing problems. option explicit option default integer dim inB$ = "" dim tmp$ = "" dim rec$ = "" Open "COM1:9600,64,rxINT(),1" As #1 on key conINT() do loop sub rxINT() inB$ = input$(1, #1) if (asc(inB$) = &H03) then rec$ = tmp$ tmp$ = "" print "END" print rec$ else tmp$ = tmp$ + inB$ endif end sub REM In MM_Chat VT100 Mode ! sub conINT() inb$ = inkey$ if (asc(inB$) = &H0D) then rec$ = tmp$ tmp$ = "" print "END" print rec$ else tmp$ = tmp$ + inB$ print tmp$ endif end sub greetings wolfgang |
||||
| Andrew_G Guru Joined: 18/10/2016 Location: AustraliaPosts: 872 |
Hi all, Once again Geoff's immediate response is refreshing - that is one thing that is so good about the MM and this community! I'm again reluctant to step foot into the "land of giants" as my serial comms is limited to NMEA at 4800 but does this provide an option? For example, it uses <CR><LF> after the "$hh" to mark the end of each sentence (I test for this in my code). Does this achieve what the OP is after? Cheers, Andrew |
||||
| Geoffg Guru Joined: 06/06/2011 Location: AustraliaPosts: 3308 |
That was exactly my thinking Andrew. The reason why I thought it was such a good idea was because the other day I was writing an interrupt driven routine to get data from a GPS module. Because GPS lines are variable length it was necessary to trigger an interrupt on every character received which was wasteful of the CPU. With this feature you could trigger the interrupt when the carriage return was received ie, once a line. Serial data is nearly always variable length. Another example is getting data from the ESP8266 WiFi Module. Geoff Graham - http://geoffg.net |
||||
CaptainBoing Guru Joined: 07/09/2016 Location: United KingdomPosts: 2171 |
I am definitely in the camp of "the language should provide a set of really good primitives, bells and whistles should be how the app uses them". I use a tightly controlled buffer with frame markers just as you do in all of my serial comms input handling, but to implement the above, you would also need a timeout and other means of bailing as your program would lose control while the firmware was spinning receiving characters (or not) waiting for your ETX char. This could flood buffers, or anything all the while you can't do anything about it. I think it is asking for the input routine to become too specialized and no longer primitive. Is the expense in Flash worth it? I do it quite cheaply (with the comments removed) and simply with MMBasic primitives: If Loc(#2)<>0 Then ' something has arrived from the bluetooth txcvr Pause 20' give generous time for a complete command string to arrive - 16mS ish b$=b$+Input$(Min(16,Loc(#2)),#2) ' grab the whole buffer or the bottom 16 chars which ever is smaller and add it to the rx buffer string If Len(b$)>=16 Then ' if the buffer looks like it could be a command, (they look like $D1505054337041!) ' check for the STX($) and ETX(!) chars and optimistically, parse for a command if Left$(b$,1)<>"$" Or Mid$(b$,16,1)<>"!" Then FlushIO:Goto BadFrame ... This achieves what you have in your example, yes it complicates the procedure a tiny bit but I never lose control to the firmware. I never use [LINE] INPUT because if for some reason the terminating <CR> gets lost, my program effectively hangs. By receiving the chars at my leisure I maintain a bullet-proof control over comms - which experience has taught me is not to be trusted in industrial environments. So from my own selfish PoV, I probably wouldn't use any such improvements - I am such a control grouch!  Soz |
||||
| JohnS Guru Joined: 18/11/2011 Location: United KingdomPosts: 4133 |
I've not noticed a shortage of CPU such that an interrupt per character is a problem. However, a time out in case the terminator is corrupted/dropped is almost always needed if you want a robust solution (as I do). So much as this looks interesting it also looks like it can easily lead to flaky code. John |
||||
| erbp Senior Member Joined: 03/05/2016 Location: AustraliaPosts: 195 |
Hi, I wanted to take to opportunity to respond the some of the many comments received to my suggestion, many if which I think are based on a misunderstanding of my request. Unfortunately after spending 45 minutes typing up my reply I managed to lose the post  I don't have time to re-do it all. So a brief summary will have to suffice.1. It is only intended as an ALTERNATIVE OPTION to the existing "get interrupted whenever <n> characters are in the Rx buffer". If you don't like the idea, don't use it, or don't use any interrupt at all. 2. It doesn't "lose control to the firmware". That depends on which command you use to do the actual receive of the data into your program. I use the INPUT$(nbr, [#]fnbr) function (as recommended in Appendix A of the Micromite Manual) - this function is non-blocking and is unaffected by my suggestion. 3. It frees the program of having to handle each character one at a time so that you can compare to see if you have reached the end of a variable length message (however you recognize that condition), so is actually more efficient. 4. If you don't already incorporate timeouts into your handling of exchanging serial data between devices then you are already asking for trouble. I don't see that my suggestion changes this - either for better or worse. 5. Yes, it doesn't address all possible situations with regards to receiving variable length data, particularly in situations where you cannot control the format of the data being received. It doesn't help if the data doesn't contain some form of EOM or frame marker and you have no way to include such. In those situations continue to use whatever you do today. As I said this is an OPTION - you are not forced to use it if it doesn't meet your needs. 6. My use of the "/" in referring to intlevel/EOMchar wasn't meant to imply using a slash in the actual command syntax. I was merely trying to explain that the current 'intlevel' optional parameter would have a dual interpretation - either as a number of characters or as specifying the desired EOM sentinel character. I will leave the actual syntax to Geoff, should he decide to proceed with the suggestion. I am happy to abide by whatever decision Geoff makes on this - it was only a suggestion after all. BTW Geoff, if you do decide to proceed don't rush on my behalf, I won't be in a position to test anything for at least another week. I'd better hit the post button now before I manage to lose this reply too. Phil. |
||||
Turbo46 Guru Joined: 24/12/2017 Location: AustraliaPosts: 1646 |
@erbp, Your idea is a good one - well explained and thought out. Especially for the application that Geoffg has in mind. But if I am in control of what is being sent and received I prefer the use of a CRC and use timing to determine the end of the message. @Geoffg, Some people seem reluctant to use a CRC because of the perceived complexity. With that in mind and because MMBasic uses Xmodem protocol internally, how difficult would it be to add an Xmodem CRC string function to MMBasic? ie: XCRC$( string$ ) Adding a CRC to a message would be a simple as: TxMsg$ = message$ + XCRC(message$) Check a received message by: IF XCRC(RxMsg) = CHR$(0)+CHR$(0) THEN ' CRC is correct Bill Keep safe. Live long and prosper. |
||||
| erbp Senior Member Joined: 03/05/2016 Location: AustraliaPosts: 195 |
@Turbo46, I wasn't advocating against using a CRC to determine message integrity, I just didn't mention it because it can be used with or without SOM/EOM markers and regardless of how you determine when you have received a 'complete' message. I see use of CRC as a separate topic not specifically related to my suggestion. Phil. |
||||
| Turbo46 Guru Joined: 24/12/2017 Location: AustraliaPosts: 1646 |
Sorry erbp, I wasn't trying to hijack your post - just offering an alternative because in these days messages can contain non-printing ascii characters and so any SOM/EOM character could appear at any point in a message. Bill Keep safe. Live long and prosper. |
||||
Grogster Admin Group  Joined: 31/12/2012 Location: New ZealandPosts: 9753 |
I am probably missing something obvious here, but how come you can't just check for the end-of-message marker byte in the code, and act on that: The buffer holds on to all messages one after the other, and you just read them out of the buffer one at a time. I get the feeling I am not quite following what you are trying to do, otherwise I expect you probably would just do what I have done above.  Smoke makes things work. When the smoke gets out, it stops! |
||||
| Turbo46 Guru Joined: 24/12/2017 Location: AustraliaPosts: 1646 |
@Grogster, I think the point is that you only attend to the message when you know you have a complete one. Do other things until an interrupt occurs. Bill Keep safe. Live long and prosper. |
||||
panky Guru Joined: 02/10/2012 Location: AustraliaPosts: 1116 |
+1 for Phil's initial suggestion and +1 for Bill's CRC suggestion. panky ... almost all of the Maximites, the MicromMites, the MM Extremes, the ArmMites, the PicoMite and loving it! |
||||
| JohnS Guru Joined: 18/11/2011 Location: United KingdomPosts: 4133 |
Well, that's stuck in a loop but of course the equivalent via interrupt per character is trivial and that's what people do instead. You're not really missing anything (other than that obvious difference). This would be an enhanced feature that isn't in any sense needed but may be desired. Because the existing way is small, simple, robust and easily copes with things such as timeouts and/or CRCs I can't see the new feature as useful but others take a very different view as you can see. I suppose if not used then all it'd be doing would be taking up flash memory space. On CRCs, I think I'd go for a much more robust one than the one with XModem but it's a different topic than this thread really. John |
||||
 Print this page |
 |
The Back Shed's forum code is written, and hosted, in Australia. | © JAQ Software 2025 |
