 Notice. New forum software under development. It's going to miss a few functions and look a bit ugly for a while, but I'm working on it full time now as the old forum was too unstable. Couple days, all good. If you notice any issues, please contact me. |
Forum Index : Microcontroller and PC projects : MMBasic 4.4 Beta 1
Page 1 of 3   |
|||||
| Author | Message | ||||
Geoffg Guru  Joined: 06/06/2011 Location: AustraliaPosts: 3165 |
I have a test version of MMBasic 4.4 with random file I/O and I would be grateful if anyone who was waiting for random file I/O (the SEEK command) would give it a spin and let me know if it meets expectations. You can download it from: http://geoffg.net/Downloads/Maximite/MMBasic_4.4_Beta_1.zip The only difference between this and 4.3A is the random file I/O. Also in the zip is the DOS version. I had to make some changes to the handling of CR/LF to implement the SEEK command so, if you use the DOS version to read/write files, please give it a test also. Thanks Geoff Geoff Graham - http://geoffg.net |
||||
marcwolf Senior Member  Joined: 08/06/2009 Location: AustraliaPosts: 119 |
Hi Geoff I am trying it on the DOS Maximate as I don't have a hardware maximite on me. It looks very good - little different from the TRS-80 disk basic but certainly usable. Normally one would define the Rec Len in the Open command and then use LTRIM and RTRIM to leftpad or rightpad the field owever this works very well too (just a different learning curve) However it be very easy to make some functions to do that for you :) Many many thanks for all of your great work in getting this out so quickly.. Dave Coding Coding Coding.. Keep those keyboards coding.. RAW CODE!!!!! |
||||
kiiid Guru Joined: 11/05/2013 Location: United KingdomPosts: 671 |
Hi Geoff, Based on the description and the demo code it looks to me like the SEEK command actually places the file pointer at some specified byte position considering the file as flat binary format. Is that right? Kon http://rittle.org -------------- |
||||
| Geoffg Guru Joined: 06/06/2011 Location: AustraliaPosts: 3165 |
That is correct. Geoff Graham - http://geoffg.net |
||||
| ajkw Senior Member Joined: 29/06/2011 Location: AustraliaPosts: 290 |
Geoff, It would appear to be working as described/expected on my original MM a couple of things however data$ = input$(64, #1) returns a zero length string it seems as data$ is considered a keyword as it becomes capitalised dat$ = input$(64, #1) works fine and (this is also off this topic. refer previous conversations) 4.4 is rock solid in regards video jitters unlike 4.3a Lastly, the word APPEND seems to suggest that new data written will be inserted rather than overwriting existing data. I am not saying overwriting is wrong but more giving a general heads up about what happens when you write or print to a file. Rgd's Anthony. |
||||
| Geoffg Guru Joined: 06/06/2011 Location: AustraliaPosts: 3165 |
Thanks for that Anthony. I am still thinking of adding a new mode for the OPEN command called RANDOM. That would be less confusing than using APPEND. I still have not got to the bottom of the video jitters. This test version is thoroughly debugged so if you want to you can keep using it instead of 4.3A. Geoff Geoff Graham - http://geoffg.net |
||||
| marcwolf Senior Member Joined: 08/06/2009 Location: AustraliaPosts: 119 |
If you were to differentiate between APPEND and RANDOM then you could add the record length to the end of the open command open "aaaa" for random(64) as #1 Then all inputs$ will already have the record size, and you can also use the record length in actual calculation re go to record rather than byte position. Plus you can also handle for the user the padding details. Dave Coding Coding Coding.. Keep those keyboards coding.. RAW CODE!!!!! |
||||
BobDevries Senior Member Joined: 08/06/2011 Location: AustraliaPosts: 266 |
On the Tandy Colour Computer, the commands for direct(random) access filing go like this: OPEN "D", #1, "OFFICE/DAT", 8 The "D" specifies Direct Access, and the 8 shows the record size. As well, the command: FIELD #1, 3 AS A$, 5 AS B$ allows you to split up the record for use with user selected strings Various other commands are used: LSET, RSET to left or right-justify the string. MKN$(), CVN() to convert numeric variables to/from string variables. I'm not necessarily advocating that these should be used on the MM, but just showing another way that has been used successfully. Regards, Bob Devries Dalby, QLD, Australia |
||||
| marcwolf Senior Member Joined: 08/06/2009 Location: AustraliaPosts: 119 |
There are several ways the RANDOM statement has been set up but in all of them the record length has been part of it. The field command would be nice but would require quite a bit of code to implement (from my observation of the MMBasic code) and can easily be handled via the MID$ Dave Coding Coding Coding.. Keep those keyboards coding.. RAW CODE!!!!! |
||||
MicroBlocks Guru Joined: 12/05/2012 Location: ThailandPosts: 2209 |
Don't forget about variable length records! Defining a record length at opening time can prevent them from being used. Microblocks. Build with logic. |
||||
| marcwolf Senior Member Joined: 08/06/2009 Location: AustraliaPosts: 119 |
In defence of Geoff's skills as a programmer - variable length records are VERY difficult to implement and few systems have them Firstly every record needs a header with the number of bytes in that record. Then to move to X record you would need to do this 1. Read first record header and get number of bytes 2. Skip numbers of bytes in record and read next record header. 3. Repeat process 2 until record count is met. Any GET to a record would mean that every record up to and including that record would have to be read to implement it. Replacing a record is even more difficult as you have to write the whole file in and out again. Alternatively you can use an two file system with an Index file containing the start position in bytes for each record and the number of bytes in that record, and then go to the appropriate byte position in the Master file. However again housekeeping re updating, deleting, and inserting new records will take up more processing time. Again it depends on your usage for the Maximite - simple but powerful controller or full scale computer with a DOS and Database capability. Marc Coding Coding Coding.. Keep those keyboards coding.. RAW CODE!!!!! |
||||
| MicroBlocks Guru Joined: 12/05/2012 Location: ThailandPosts: 2209 |
Variable length is actually pretty simple. Just a little bookkeeping. It most certainly must not be made part of MMBasic. As long as it is possible to SEEK to a certain location in a file and use a GET or READ to get a certain amount of bytes into a string it would be up to the programmer using the available basic commands to implement a variable length records field structure. To add to your description an update of a record can be done on the same location in the file as long as it is the same length or shorter. If it is longer the simplest way is to append it to the file. Many VLR implementations have some configurable reserved space allocated so that edits done that make the data a few bytes bigger would not result in lost space and out of sequence data storage. Another approach is a linked list where every record is preceded by two file position pointers, one to the previous and one to the next record. Followed by its length and data. A linked list allows the file to be sorted by manipulating the previous and next pointers and leave the data in place. Sorting a 20 megabyte file having such a structure can be done using very little memory. At some moment you would get fragmentation in the file when many edits are done. A simple reading of records following the pointers and writing this data sequentially to a new file will compact and reorganize the file in one go. If that sequence is actually a sort of the data you end up with a compact field that is sorted and can be read sequentially. If you need the same data but sorted differently you can then make for each field you need a file with only the previous and next pointers. You can then do very fast binary searches and it speed is not getting much slower even with huge amounts of data. Very powerful data manipulation and retrieving can be done with only some bookkeeping when files are opened for RANDOM and allows positioning on a byte level with a SEEK and data retrieval/storage with GET/PUT or READ/WRITE. A fixed record system would only need a constant with which you can multiply the record number to get a file position. Something that is trivial in any programming language. Microblocks. Build with logic. |
||||
| marcwolf Senior Member Joined: 08/06/2009 Location: AustraliaPosts: 119 |
Well - I've been programming since 1976 and have used all sorts of methods including Linked Lists (same as an Index file) Before the days of relational databased you had to use all sorts of tips and tricks to get the code working. Atleast nowaday we do not have to worry about what file is saved firsted on a SD card :), where as on early days if hard drives lots of work was done to ensure that indexes were access fasted, folllowed my data files, and finally program files. My first forey into networking was in 1984 and lots of fun trying to make record lockin systems and playing with buffer sizes. It worked too!! Coding Coding Coding.. Keep those keyboards coding.. RAW CODE!!!!! |
||||
| Geoffg Guru Joined: 06/06/2011 Location: AustraliaPosts: 3165 |
Interesting discussion. At the start I considered using the Microsoft syntax for accessing random files (I try to stay compatible) but, in my opinion, their way of doing it was overly complicated and added too many new commands/functions for such a simple job - not Bill's finest hour. The important thing here is that the MM is generally NOT used as a business computer and random files will be used for more lightweight tasks rather than implementing a database or accounting system. As a result I decided to keep it simple, even to the point of reusing the open mode of APPEND (in the final version I will change it to open for RANDOM access). Implementing variable length records with linked lists would be fun and, for one mad moment, I was tempted but it would add a lot of complexity for a feature that would be rarely used. Geoff Geoff Graham - http://geoffg.net |
||||
| kiiid Guru Joined: 11/05/2013 Location: United KingdomPosts: 671 |
I agree with Geoff. Everything above byte level access depends on the exact application. RANDOM is more suitable, that's true. Speaking about files, I think it would be great to have OPTION for disabling the write protection check. http://rittle.org -------------- |
||||
| marcwolf Senior Member Joined: 08/06/2009 Location: AustraliaPosts: 119 |
Although I would ask that that the record length be set in the OPEN command so that the LOC and LOF relates to the Record pointer rather than the Byte pointer. That way the INPUT$ command will get a record without needing the record length embedded in it Dave Coding Coding Coding.. Keep those keyboards coding.. RAW CODE!!!!! |
||||
James_From_Canb Senior Member Joined: 19/06/2011 Location: AustraliaPosts: 265 |
Will an EOF character read from the file be interpreted as an EOF, or as a binary character? In other words, can we specify that a file is binary? James My mind is aglow with whirling, transient nodes of thought careening through a cosmic vapor of invention. Hedley Lamarr, Blazing Saddles (1974) |
||||
| marcwolf Senior Member Joined: 08/06/2009 Location: AustraliaPosts: 119 |
I remember that in most RANDOM files the data was treated as binary by default. Dave Coding Coding Coding.. Keep those keyboards coding.. RAW CODE!!!!! |
||||
| MicroBlocks Guru Joined: 12/05/2012 Location: ThailandPosts: 2209 |
Implementing it in the MMBasic firmware would indeed be a waste of time as it is extremely complicated to make a generic variable record length system. However for a specific application it is rather simple to add by just making some subroutines or functions. As long as the seek command seeks on a byte level and a get/read/input is able to read a specific number of bytes (binary) it can be implemented. Another usage is when using text files. Imagine having a datalogger that writes a line to a file every minute. You have a routine that listens to commands on a serial port. One of those commands can be for transmitting the log file contents to the serial port. Maybe you have a command that only wants the last 10000 bytes of a log file. In that case the basic program opens the file, gets its length, subtracts 10000 from that, use the seek command to go to that location in the file and then uses the standard input to read the log line by line until EOF. Sure the first line would not always start at the beginning of that line, but that is not important because the incredible gain you get is that you do not have to read the whole previous part of the log. I think it all can be done if the seek command just does one simple thing and that is put the location where the next read or write starts. Whether it is a standard INPUT# or PRINT# or some other statement for reading/writing a certain amount of bytes. This leads to my conclusion that the way Geoff made it (see first post zipfile) is just how it should be in my opinion. :) A flat binary file that is seekable. Adding record length at opening time will complicate things as it is not only a simple thing to add. You need other statements like GET and PUT that use that record length. You would need FIELD to setup a record, you need MKI$ CVI$ etc to be able to read/write integers, floats etc. Microblocks. Build with logic. |
||||
| kiiid Guru Joined: 11/05/2013 Location: United KingdomPosts: 671 |
I have a proposal to Geoff for something else being part of the 4.4 release. Self-modifying code is not possible to be written in BASIC anyway, so why should the program reside in the RAM thus occupying valuable memory, which could be used otherwise? While during the process of writing and editing obviously it must be in RAM, for execution, and especially for execution from A:, there is absolutely no need. So, I suggest programs from A: to be executed in place directly reading from the flash memory. The same could also be implemented for programs executed from B:, but probably as an option only since the process there is a bit slower and more complex. Of course I can do XIP in my custom MMBasic revision for the 4105C module only as the whole modification is not too great, but I believe it would be much better if it was an integral part of the universal MMBasic release thus making it available to all users. What do other users think about that? http://rittle.org -------------- |
||||
| Page 1 of 3 |
|||||
 Print this page |
