 Notice. New forum software under development. It's going to miss a few functions and look a bit ugly for a while, but I'm working on it full time now as the old forum was too unstable. Couple days, all good. If you notice any issues, please contact me. |
Forum Index : Microcontroller and PC projects : Armmite F4: File Access Issue
| Author | Message | ||||
| erbp Senior Member  Joined: 03/05/2016 Location: AustraliaPosts: 195 |
@Matherp - I'm afraid I have another issue with the Armmite F4 firmware. The LOC(#n) function in the 5.05.08 MMBasic (13-August-2019 version) returns the incorrect value for the current file pointer position. I have a program that builds a table containing the file position value for the start of each record in the file, which is subsequently used to allow any record to be accessed based on record number. The table is built when the program is started by opening the file FOR INPUT and using the LINE INPUT... command to read record at a time thru the entire file. After each LINE INPUT command, the LOC(#n) function should return the position pointer to the start of the NEXT record in the file, and this value is stored in the table. After the table is built the file is closed and re-opened FOR RANDOM and then any required record can be accessed using the table to provide the relevant position value for the SEEK #n,<pos> command. This works very nicely on a MM+ (E100) running Ver 5.05.02 Beta 1, but on the F4 the pointer values in the table are completely wrong so it does not work at all. This zip file contains a simple test program and data file. Place the data file on an SDCard as A:/TEST/Test.txt, load the program and run it to see the results. Below are the results from my tests using this test program and data file:- Correct results from MM+: Micromite Plus MMBasic Ver 5.05.02 Beta 1 Copyright 2011-2018 Geoff Graham > run Starting SDCard File Access Test 8 records read: 1: 1 2: 32 3: 71 4: 96 5: 132 6: 160 7: 196 8: 244 End of SDCard File Access Test > Incorrect results from Armmite F4: ARMmite MMBasic Version 5.05.08 Copyright 2011-2019 Geoff Graham Copyright 2016-2019 Peter Mather > run Starting SDCard File Access Test 8 records read: 1: 1 2: 283 3: 283 4: 283 5: 283 6: 283 7: 283 8: 283 End of SDCard File Access Test Also while I am on the subject of SDCard File access on the F4 - is there a reason why the file path as displayed via FILES or returned by CWD$ does not include the Logical Drive Letter? e.g. CWD$ will return :/ instead of A:/ as it would on a MM+ device. Regards, Phil. |
||||
TassyJim Guru  Joined: 07/08/2011 Location: AustraliaPosts: 6283 |
You are using LOC() which is supposed to only work for files opened as RANDOM Unfortunately, LINE INPUT doesn't seem to work for RANDOM. It is a lucky quirk that allows your program to work on the MM+ From the MM+ manual: It would be nice if it did work on the Armites. Jim VK7JH MMedit |
||||
| TassyJim Guru Joined: 07/08/2011 Location: AustraliaPosts: 6283 |
Definitely something strange going on. If you open a file with RANDOM, you can SEEK and LOC() and they agree but I can't get INPUT$() to start reading at that location. It seems to keep it's own internal LOC() Jim VK7JH MMedit |
||||
| erbp Senior Member Joined: 03/05/2016 Location: AustraliaPosts: 195 |
Jim, thanks. I had missed that LOC() was only for RANDOM, but it seems to be quite happy working with the file opened for INPUT. I just tested now and on the MM+ it seems to work OK with the file opened for RANDOM as well. I presume your comment about LINE INPUT relates to the F4, as I have LINE INPUT working on the MM+ for both INPUT and RANDOM files. For RANDOM you have to use SEEK after opening as at file open the file position pointer is set to End Of File, and so there is nothing to read from that position. Cheers, Phil. |
||||
| matherp Guru Joined: 11/12/2012 Location: United KingdomPosts: 10315 |
Some time ago I implemented buffered reads for the SDcard to speed up input. It now reads 512bytes at a time into a buffer and the buffer serves the individual character requests from MMBasic (all MMBasic file I/O is one character at a time). I hadn't properly updated the LOC or SEEK code to work with the buffer. This should now be corrected and I'll post the bug fix on the main Armmite thread Edited 2019-09-24 01:56 by matherp |
||||
| TassyJim Guru Joined: 07/08/2011 Location: AustraliaPosts: 6283 |
The latest firmware is an improvement. SEEK works but LOC() still doesn't. Using the original test.txt and this code: DOS gives: F4 gives: I made it work by calculating the file position rather than using LOC() The second column is the LOC() after the read. To prove it's usability, I then reopened the file and retrieved record No 5. Jim VK7JH MMedit |
||||
| erbp Senior Member Joined: 03/05/2016 Location: AustraliaPosts: 195 |
Jim beat me to it - I was about to post some test results but they confirm what Jim has found so I won't bother to post them. I have found one other difference in the way the LINE INPUT command works on the F4 compared to the MM+. I suspect this was there all along but I never noticed it because without SEEK working it would be very difficult to set up the conditions to test. I believe this only affects RANDOM access. On the MM+ LINE INPUT returns data from the current file position pointer (presumably set by a previous SEEK) to the next Carriage Return but not including the CR. At this time LOC will be pointing to the Line Feed (LF) character that normally follows a CR. Another LINE INPUT without using SEEK to alter the pointer then returns the next record up to the next CR. It silently ignores the LF character. In this way two successive LINE INPUT commands will return consistent results whether the file uses CR+LF or CR only as the end of line terminator. Same applies if you use SEEK to specifically set the pointer to a LF character following a CR. The next LINE INPUT will ignore the LF and return the record following. On the F4, LINE INPUT does NOT ignore the LF following a CR - so using the above scenario, the first LINE INPUT will return the requested record data (not including the CR) - same as the MM+. But the next LINE INPUT without altering the pointer will return an empty string, as it has NOT ignored the LF and furthermore it has treated it as a line terminator too. The third LINE INPUT will then return the data for the next record. I believe the MM+ implementation to be the desirable one that should prevail. Cheers, Phil P.S. From a brief test the BLIT READ / BLIT WRITE / BLIT CLOSE commands look to be working as they should - thanks for that. |
||||
| matherp Guru Joined: 11/12/2012 Location: United KingdomPosts: 10315 |
I hate boundary conditions  My code works perfectly EXCEPT for the last block in the file and if the file is only one block long then........ Now just need to work out how to fix it.  PS the MM+ gives a slightly different answer to DOS. The numbers are always 1 greater in the first part of Jim's test. i.e.they match the second part 1,33,72 etc. Edited 2019-09-24 19:44 by matherp |
||||
| matherp Guru Joined: 11/12/2012 Location: United KingdomPosts: 10315 |
That took 6 hours Another version posted |
||||
| TassyJim Guru Joined: 07/08/2011 Location: AustraliaPosts: 6283 |
The latest firmware looks good to me. Thanks Peter. I tried combinations of RANDOM and INPUT with LOC() and SEEK and all gave the same correct results. The difference to DOS is interesting and I will look further when time permits. Jim VK7JH MMedit |
||||
| erbp Senior Member Joined: 03/05/2016 Location: AustraliaPosts: 195 |
I have tested the latest firmware and while LOC and SEEK are working, There are discrepancies in the way LOC and LINE INPUT work. This is partly explained in my previous post on this thread (yesterday). This also is the cause of the differences in numbers for the LOC values referred to by both Peter and Jim. I don't have the DOS version on MMBASIC installed so can't verify but from the details above I believe DOS and MM+ work the same way and the F4 is slightly different. I will refer specifically to MM+ since that is what I tested on. The first MM+ LINE INPUT statement on a file opened FOR INPUT returns the record data up to the CR line terminator, but does not return the CR. At this time a LOC() will return the pointer to the character following the CR. Since most files (on Windows anyway) use a CR+LF as line terminator this will be the LF character following the CR. The next LINE INPUT statement will actually find the LF character and skip it, returning data from the first character in the next record thru to the next CR. On the F4 the first LINE INPUT statement on a file opened FOR INPUT returns the record data up to the CR+LF line terminator, but does not return either the CR or LF. At this time a LOC() will return the pointer to the character following the LF - i.e the first data character of the next record. This is why the LOC values for the same file are 1 greater on the F4 compared to the MM+ (and I presume DOS). It is not really a LOC problem, just a by-product of where the LINE INPUT has left the pointer - after the CR or after the LF. Now on a file using CR+LF as line terminators, the difference is totally academic - each LINE INPUT statement returns the next successive data record. RANDOM access based on using saved LOC values for the start of each record also works. What won't work is if you try to read a file that uses ONLY a CR character as line terminator (I think UNIX and maybe LINUX do this). In this case the MM+ still works fine as it works off the CR only and silently skips the LF following, so if there is no LF it just doesn't have to skip anything. HOWEVER the F4 will return the entire file content as a single record on the first LINE INPUT statement, as it appears that it looks for CR+LF as the terminator, which of course it doesn't find. RANDOM ACCESS LINE INPUT doesn't work either, even if you know the correct offsets to the "start" of each record - it will return the data from the specified start (SEEK) value to end of file. In actual fact unless the file is very short the system will crash as the string returned will exceed the 255 character limit. Now for my purposes I don't actually care about files that only use CR as the terminator, so the firmware version released today will solve my problems, but I do wonder whether the LINE INPUT / LOC functions shouldn't work the same way on all the platforms they are implemented on.  Cheers, Phil. |
||||
| matherp Guru Joined: 11/12/2012 Location: United KingdomPosts: 10315 |
The MM+ and the Armmite give the same output on Jim's test program. MMBasic for DOS is different by one character. I can't understand the issue you are seeing on CR/LF as the F4 uses exactly the same code as the MM+ I've just checked again. |
||||
| matherp Guru Joined: 11/12/2012 Location: United KingdomPosts: 10315 |
OK now I understand. Geoff has changed the LINE INPUT code sometime between version 5.4 and 5.5.02 on the MM+. 5.4.10 (for example) interprets a single CR as an end-of-line. 5.05.02 and the Armmite interprets a single LF as an end of line. Both are happy with CRLF. The 5.05.02 version makes the input compatible with both LINUX formatted files and DOS ones I don't know who if anyone uses a single CR. I assume Geoff will stick with the new version going forward and therefore you should assume that lines in files terminated with a single LF are OK for line input. Lines in files terminated with a single CR are not. 5.4.10 if(c == '\n') { // what to do with a newline if(nbrchars == 0) continue; // skip a LF that is at the beginning (probably followed a CR) else c = '\r'; // if it was not preceeded with a CR convert it to a CR } if(c == '\r') { if(filenbr == 0 && EchoOption) MMPrintString("\r\n"); break; // end of the line - stop collecting } 5.05.02 and Armmite F4 if(c == '\n') { // what to do with a newline break; // a newline terminates a line (for a file) } if(c == '\r') { if(filenbr == 0 && EchoOption) { MMPrintString("\r\n"); break; // on the console this means the end of the line - stop collecting } else continue ; // for files loop around looking for the following newline } Edited 2019-09-26 04:08 by matherp |
||||
| TassyJim Guru Joined: 07/08/2011 Location: AustraliaPosts: 6283 |
Those strange MAC people like to be different. Also, when you are talking to external devices, the EOL can be CRLF, LFCR, CR or LF If you have saved incoming data to file, you may end up with any of the above. In those cases, the programmer will have to get creative. Jim VK7JH MMedit |
||||
panky Guru Joined: 02/10/2012 Location: AustraliaPosts: 1115 |
Is this something where an OPTION statement may be worth considering? ie. OPTION TERMINATION xx where xx can be CR LF CRLF LFCR or in fact any character sequence. This could also cater for CSV style files where the seperator can be COMMA, TAB etc. ... or is this just adding complexity that is only to address fairly specific circumstances? panky ... almost all of the Maximites, the MicromMites, the MM Extremes, the ArmMites, the PicoMite and loving it! |
||||
| erbp Senior Member Joined: 03/05/2016 Location: AustraliaPosts: 195 |
Peter / Jim / Panky, Thanks for the feedback and comments. As I said I am happy with the way it now works as I intend to use files with CRLF terminators. I was mainly trying to point out that there was a difference in behaviour between the platforms and that under some circumstances that might cause problems (or at least some confusion). While Panky's suggestion would be a neat and flexible way to potentially address the problem, personally I think the added complexity is not warranted - just my opinion. If others feel it worthwhile they can argue a case for it to be included. Peter - thanks for your efforts in getting the original SEEK / LOC issues resolved - THEY definitely were impacting my code - should be all good now. Cheers, Phil. |
||||
 Print this page |
 |
The Back Shed's forum code is written, and hosted, in Australia. | © JAQ Software 2025 |
