 Notice. New forum software under development. It's going to miss a few functions and look a bit ugly for a while, but I'm working on it full time now as the old forum was too unstable. Couple days, all good. If you notice any issues, please contact me. |
Forum Index : Microcontroller and PC projects : Xmodem Receive fn$ bug on MMBasic (ARMmite F4) using MMedit File Mgr
| Author | Message | ||||
| erbp Senior Member  Joined: 03/05/2016 Location: AustraliaPosts: 195 |
Hi, I used MMEDIT File Manager to copy a data file from PC to SDCard on an ARMmite F4. When processing that data file my program did strange stuff which I have eventually tracked down to what I believe is a bug in the XMODEM RECEIVE [filename$] command in the MM firmware. This is demonstrated in the following. I created a test data file and a test program on my PC and put these two files on a SDCard that was physically plugged in to a Card Reader on the PC. The test data file (MRC_Test.txt) reports being 378 bytes long. I then took this SDCard to my ARMmite F4, loaded and ran my test program (PhilTest.bas). Following is the results as displayed via the console: > run Enter name of test file: ? A:\MRC\MRC_Test.txt File A:\MRC\MRC_Test.txt - Length: 378 1: 0 , 1 0 0 , 7 , 1 , 2 , 2 , 0 30 2C 31 30 30 2C 20 20 37 2C 20 20 31 2C 20 20 32 2C 20 20 32 2C 20 20 30 2: 1 , 0 , 0 , 1 5 , 0 , 2 , 31 2C 20 20 30 2C 20 20 30 2C 20 31 35 2C 20 20 30 2C 20 20 32 2C 20 20 20 3: 1 , 9 , 2 , 1 1 , 2 , 1 , 31 2C 20 20 39 2C 20 20 32 2C 20 31 31 2C 20 20 32 2C 20 20 31 2C 20 20 20 4: 1 , 1 0 , 3 , 1 2 , 3 , 1 , 31 2C 20 31 30 2C 20 20 33 2C 20 31 32 2C 20 20 33 2C 20 20 31 2C 20 20 20 5: 1 , 1 1 , 4 , 1 3 , 4 , 1 , 31 2C 20 31 31 2C 20 20 34 2C 20 31 33 2C 20 20 34 2C 20 20 31 2C 20 20 20 6: 1 , 1 2 , 5 , 1 4 , 5 , 1 , 31 2C 20 31 32 2C 20 20 35 2C 20 31 34 2C 20 20 35 2C 20 20 31 2C 20 20 20 7: 1 , 1 3 , 6 , 1 5 , 6 , 1 , 31 2C 20 31 33 2C 20 20 36 2C 20 31 35 2C 20 20 36 2C 20 20 31 2C 20 20 20 8: 1 , 1 4 , 7 , 1 5 , 7 , 1 , 31 2C 20 31 34 2C 20 20 37 2C 20 31 35 2C 20 20 37 2C 20 20 31 2C 20 20 20 9: 2 , 1 5 , 8 , , , , 32 2C 20 31 35 2C 20 20 38 2C 20 20 20 2C 20 20 20 2C 20 20 20 2C 20 20 20 10: 3 , 5 , - 6 , 4 , , , 33 2C 20 20 35 2C 20 2D 36 2C 20 20 34 2C 20 20 20 2C 20 20 20 2C 20 20 20 11: 3 , 5 , 7 , 4 , , , 33 2C 20 20 35 2C 20 20 37 2C 20 20 34 2C 20 20 20 2C 20 20 20 2C 20 20 20 12: 4 , 0 , - 1 1 , 1 0 , - 1 , , 34 2C 20 20 30 2C 2D 31 31 2C 20 31 30 2C 20 2D 31 2C 20 20 20 2C 20 20 20 13: 4 , 0 , 2 , 1 0 , 1 2 , , 34 2C 20 20 30 2C 20 20 32 2C 20 31 30 2C 20 31 32 2C 20 20 20 2C 20 20 20 14: 6 , - 1 2 , - 8 , - 1 2 , 5 , 1 2 , - 8 36 2C 2D 31 32 2C 20 2D 38 2C 2D 31 32 2C 20 20 35 2C 20 31 32 2C 20 2D 38 PhilTest Terminated. > 14 records of clean data - exactly as it should be. I then used MMEdit File Manager to transfer a second copy of the test data file to the SDCard. The below screen shot is prior to the transfer - note the size of the MRC_Test.txt file is showing the same 378 bytes in both the PC and MM panels.  I then uploaded the file from PC to SDCard as MRC_Test_2.txt - following screen shot shows that the MRC_Test_2.txt file size is 384 bytes on the SDCard!!  Running my test program against the uploaded test file show the following results: > run Enter name of test file: ? A:\MRC\MRC_Test_2.txt File A:\MRC\MRC_Test_2.txt - Length: 384 1: 0 , 1 0 0 , 7 , 1 , 2 , 2 , 0 30 2C 31 30 30 2C 20 20 37 2C 20 20 31 2C 20 20 32 2C 20 20 32 2C 20 20 30 2: 1 , 0 , 0 , 1 5 , 0 , 2 , 31 2C 20 20 30 2C 20 20 30 2C 20 31 35 2C 20 20 30 2C 20 20 32 2C 20 20 20 3: 1 , 9 , 2 , 1 1 , 2 , 1 , 31 2C 20 20 39 2C 20 20 32 2C 20 31 31 2C 20 20 32 2C 20 20 31 2C 20 20 20 4: 1 , 1 0 , 3 , 1 2 , 3 , 1 , 31 2C 20 31 30 2C 20 20 33 2C 20 31 32 2C 20 20 33 2C 20 20 31 2C 20 20 20 5: 1 , 1 1 , 4 , 1 3 , 4 , 1 , 31 2C 20 31 31 2C 20 20 34 2C 20 31 33 2C 20 20 34 2C 20 20 31 2C 20 20 20 6: 1 , 1 2 , 5 , 1 4 , 5 , 1 , 31 2C 20 31 32 2C 20 20 35 2C 20 31 34 2C 20 20 35 2C 20 20 31 2C 20 20 20 7: 1 , 1 3 , 6 , 1 5 , 6 , 1 , 31 2C 20 31 33 2C 20 20 36 2C 20 31 35 2C 20 20 36 2C 20 20 31 2C 20 20 20 8: 1 , 1 4 , 7 , 1 5 , 7 , 1 , 31 2C 20 31 34 2C 20 20 37 2C 20 31 35 2C 20 20 37 2C 20 20 31 2C 20 20 20 9: 2 , 1 5 , 8 , , , , 32 2C 20 31 35 2C 20 20 38 2C 20 20 20 2C 20 20 20 2C 20 20 20 2C 20 20 20 10: 3 , 5 , - 6 , 4 , , , 33 2C 20 20 35 2C 20 2D 36 2C 20 20 34 2C 20 20 20 2C 20 20 20 2C 20 20 20 11: 3 , 5 , 7 , 4 , , , 33 2C 20 20 35 2C 20 20 37 2C 20 20 34 2C 20 20 20 2C 20 20 20 2C 20 20 20 12: 4 , 0 , - 1 1 , 1 0 , - 1 , , 34 2C 20 20 30 2C 2D 31 31 2C 20 31 30 2C 20 2D 31 2C 20 20 20 2C 20 20 20 13: 4 , 0 , 2 , 1 0 , 1 2 , , 34 2C 20 20 30 2C 20 20 32 2C 20 31 30 2C 20 31 32 2C 20 20 20 2C 20 20 20 14: 6 , - 1 2 , - 8 , - 1 2 , 5 , 1 2 , - 8 36 2C 2D 31 32 2C 20 2D 38 2C 2D 31 32 2C 20 20 35 2C 20 31 32 2C 20 2D 38 15: 1A 1A 1A 1A 1A PhilTest Terminated. > Note the 15th record has appeared containing 5 0x1A bytes. This would appear to be the XMODEM "filler" bytes inserted in the final block of the transfer to make the block up to the 128 byte size used by XMODEM. Note that the new file size of 384 bytes = 3 x 128. While this problem is actually using the ARMmite F4 firmware, as XMODEM RECEIVE is part of the core firmware I suspect it is not a problem specific to the F4 and probably impacts all firmwares except the MM2, so I guess is probably one for Geoff's bug list. Attached is a zip file with a copy of the PhilTest.bas program and the (good) MRC_Test.txt data file if this helps investigate the problem. PhilTest.zip Cheers, Phil. |
||||
TassyJim Guru  Joined: 07/08/2011 Location: AustraliaPosts: 6283 |
It is not an error. XMODEM works in 128 byte blocks. There is also a 1K version but MMBasic sticks to the original 128 byte block size. For the last packet, the file has to be padded to a 128 byte boundary. The standard allows for crtl-Z (which is the old CPM end of file character) or zero value bytes. MMBasic reports the file size, so in MMEdit, if you are transferring from the 'mite to the PC, there is an option to truncate the file back to the original size. There is no way to do that when transferring from PC to 'mite so the padding characters remain. If your data contains 1A, you will need to have some way of determining file size to cater for the end of file characters. Jim VK7JH MMedit |
||||
| TassyJim Guru Joined: 07/08/2011 Location: AustraliaPosts: 6283 |
In the data file you provided, each valid record ends in CRLF If you read using LINE INPUT as in the test program, the last record would be all 1A of any length up to 128 bytes. There should be only one record (line) with 1A bytes That will be easy to test for. The extra padding is not a problem when transferring program files because MMBasic looks for the end-of-file character and truncates. Likewise, MMEdit will strip and trailing NULLs or 1As Jim VK7JH MMedit |
||||
| erbp Senior Member Joined: 03/05/2016 Location: AustraliaPosts: 195 |
Hi Jim, I agree that for the test file it would be easy to trap the padding characters, unfortunately not so in my real scenario. There I am using the following approach: DIM STRING D$(1500,6) LENGTH 3 <other code> INPUT #1, D$(0), D$(1), D$(2), D$(3), D$(4), D$(5), D$(6) In this case the program will crash when it tries to read the padding characters (if there are 4 or more - which is highly likely) as the D$(0) string length will be exceeded. I know I can change this to use a LINE INPUT and then parse out the 7 columns of data separately in code, but the above is very convenient. I guess I could also check for and trap the string length exceeded error, but it is all getting rather messy. At least there are options.  For the moment I will just resort to moving the SDCard to the Card Reader attached to my PC when I want to copy this file (which should eventually be rarely). @Geoff - it might be an idea to add a note in the MMBasic manual for the XMODEM RECEIVE [filename$] command to warn that the resulting file MAY have some extraneous characters added to the end. Regards, Phil. |
||||
| TassyJim Guru Joined: 07/08/2011 Location: AustraliaPosts: 6283 |
If removing the SD card is inconvenient, you could write a stand-alone program to scan the file line at a time and save the valid data to a temp file. This temp file will have all the lines except the last line with the troublesome 1As (or NULLs). You will know when you transfer the file so you will know when it has to be run. If your program crashes with this wrong data, it will also crash with any file fault so depending on importance, some error trapping is a good idea. Reading files is like human input, all possibilities need to be considered. Jim VK7JH MMedit |
||||
| erbp Senior Member Joined: 03/05/2016 Location: AustraliaPosts: 195 |
Hi Jim, Thanks, I can manage with moving the SD card so will just use that method. I agree about the importance of trapping errors, and normally do this. In the case of this program, if it can't successfully read the file data then it can't perform its intended function, so even trapping the error would only allow me to display a message and close the program - gracefully rather than a crash - but the end result is the same so I opted not to include the error handling stuff. Either way I would need to investigate & fix the cause of the error before I can use the program successfully again. Anyway thanks for your input on this. Regards, Phil. |
||||
 Print this page |
 |
The Back Shed's forum code is written, and hosted, in Australia. | © JAQ Software 2025 |
