 Notice. New forum software under development. It's going to miss a few functions and look a bit ugly for a while, but I'm working on it full time now as the old forum was too unstable. Couple days, all good. If you notice any issues, please contact me. |
Forum Index : Microcontroller and PC projects : CMM2: Archiving dirs for easy transfer over serial
| Author | Message | ||||
epsilon Senior Member  Joined: 30/07/2020 Location: BelgiumPosts: 255 |
Hi all, I made a pair of tools, mac.bas and max.bas, that allow you to roll up (and extract) a directory tree into a single file archive. The archive can then easily transferred over serial using XMODEM. I don't like the idea of swapping SD cards in and out all the time. The motivation for archiving is to allow easy transfer of software packages over serial instead. mac.bas and max.bas work on Color Maximite2 and on PC. Details below. Comments are very welcome. Maybe there are already better tools available that allow you to do something like this? This is my first MMBasic program so feedback on the source code is welcome too. Cheers, Epsilon. mar.zip mac/max Color Maximite 2 Archiving Tool by Epsilon -------------------------------------------------- Version 0.1 Mac.bas allows you to roll up a directory tree into a single file archive. The created archive has the same name as the given directory, with a .mar extension. Max.bas allows you to unpack a .mar archive, creating a directory tree with the same name as the given archive, without the .mar extension. Mac.bas and Max.bas work both on PC and on Color Maximite 2, so you can download a package on your PC (e.g. Mauro's psgmini_cmm2 demo), archive it on PC using mac.bas, transfer it over to your CMM2 using XMODEM, and unpack the archive on your CMM2 using max.bas. Example 1: Creating a .mar archive on CMM2 ------------------------------------------ > list files A:/ <DIR> bak <DIR> psgmini_cmm2 10:47 18-09-2020 7847 mac.bas 09:23 18-09-2020 9216 max.bas ... 3 directories, 13 files > > *mac psgmini_cmm2 Create Maximite Archive 0.1 by Epsilon Processing dir psgmini_cmm2 Processing file .gitattributes Processing file psgdemo.bas Processing file psgmini.inc Processing file README.md Processing dir ASSETS Processing file ALESTE.VGM ... Done. > list files A:/ <DIR> bak <DIR> psgmini_cmm2 10:47 18-09-2020 7847 mac.bas 09:23 18-09-2020 9216 max.bas 11:08 18-09-2020 878068 psgmini_cmm2.mar ... Example 2:Extracting a .mar archive on CMM2 ------------------------------------------- > list files A:/ <DIR> bak 10:47 18-09-2020 7847 mac.bas 09:23 18-09-2020 9216 max.bas 11:08 18-09-2020 878068 psgmini_cmm2.mar ... > *max psgmini_cmm2.mar Extract Maximite Archive 0.1 by Epsilon mkdir psgmini_cmm2 Processing file .gitattributes Processing file psgdemo.bas Processing file psgmini.inc Processing file README.md mkdir ASSETS Processing file ALESTE.VGM Processing file ALEXKIDD.VGM Processing file AWESOME.vgm Processing file BATTLETOADS2.VGM Processing file BEAST1.vgm ... End of archive reached. Done. > list files A:/ <DIR> bak <DIR> psgmini_cmm2 10:47 18-09-2020 7847 mac.bas 09:23 18-09-2020 9216 max.bas 11:08 18-09-2020 878068 psgmini_cmm2.mar ... Example 3: Creating a .mar archive on PC ---------------------------------------- c:\cmm2\mar>dir Volume in drive C is OS Volume Serial Number is 8497-A711 Directory of c:\cmm2\mar 09/18/2020 12:04 PM <DIR> . 09/18/2020 12:04 PM <DIR> .. 09/18/2020 10:48 AM 7,845 mac.bas 09/18/2020 12:02 PM 2,365 mac_max.readme 09/17/2020 10:11 PM 9,178 max.bas 09/18/2020 12:04 PM 0 ntsh.temp 09/18/2020 12:04 PM <DIR> psgmini_cmm2 5 File(s) 897,456 bytes 3 Dir(s) 101,267,017,728 bytes free c:\cmm2\mar>..\DOS_MMBasic\MMBasic.exe mac.bas psgmini_cmm2 ... c:\cmm2\mar>dir Volume in drive C is OS Volume Serial Number is 8497-A711 Directory of c:\cmm2\mar 09/18/2020 12:04 PM <DIR> . 09/18/2020 12:04 PM <DIR> .. 09/18/2020 10:48 AM 7,845 mac.bas 09/18/2020 12:02 PM 2,365 mac_max.readme 09/17/2020 10:11 PM 9,178 max.bas 09/18/2020 12:04 PM 0 ntsh.temp 09/18/2020 12:04 PM <DIR> psgmini_cmm2 09/18/2020 12:03 PM 878,068 psgmini_cmm2.mar 5 File(s) 897,456 bytes 3 Dir(s) 101,267,017,728 bytes free Example 4: Extracting a .mar archive on PC ------------------------------------------ c:\cmm2\mar>dir Volume in drive C is OS Volume Serial Number is 8497-A711 Directory of c:\cmm2\mar 09/18/2020 12:04 PM <DIR> . 09/18/2020 12:04 PM <DIR> .. 09/18/2020 10:48 AM 7,845 mac.bas 09/18/2020 12:02 PM 2,365 mac_max.readme 09/17/2020 10:11 PM 9,178 max.bas 09/18/2020 12:04 PM 0 ntsh.temp 09/18/2020 12:03 PM 878,068 psgmini_cmm2.mar 5 File(s) 897,456 bytes 3 Dir(s) 101,267,017,728 bytes free c:\cmm2\mar>..\DOS_MMBasic\MMBasic.exe max.bas psgmini_cmm2.mar c:\cmm2\mar>dir Volume in drive C is OS Volume Serial Number is 8497-A711 Directory of c:\cmm2\mar 09/18/2020 12:04 PM <DIR> . 09/18/2020 12:04 PM <DIR> .. 09/18/2020 10:48 AM 7,845 mac.bas 09/18/2020 12:02 PM 2,365 mac_max.readme 09/17/2020 10:11 PM 9,178 max.bas 09/18/2020 12:04 PM 0 ntsh.temp 09/18/2020 12:04 PM <DIR> psgmini_cmm2 09/18/2020 12:03 PM 878,068 psgmini_cmm2.mar 5 File(s) 897,456 bytes 3 Dir(s) 101,267,017,728 bytes free Epsilon CMM2 projects |
||||
| twofingers Guru  Joined: 02/06/2014 Location: GermanyPosts: 1133 |
Hi and welcome epsilon, I did not yet try, but I like the idea and your code seems clear, straight forward and readable. Very good job (so far). Thanks!  Kind regards Michael |
||||
| epsilon Senior Member Joined: 30/07/2020 Location: BelgiumPosts: 255 |
Thanks twofingers. I fixed some issues on the DOS side of things. See attachment. I also created a GitHub repository, for future revisions: https://github.com/epsilon537/mar_cmm2 /epsilon. mar.zip Epsilon CMM2 projects |
||||
| twofingers Guru Joined: 02/06/2014 Location: GermanyPosts: 1133 |
Hi epsilon, please check your function dirExists%(dirToProcess_l$) I get an error if the folder exist (from MKDIR @line290/v0.2). @line 37 is wrong dirExists% = (DIR$(dirName$, FILE) <> "") please use dirExists% = (DIR$(dirName$, DIR) <> "") Some suggestions:: I would like to ask for a new folder name if the folder exist. and/or If a folder exist then should we have the option to include the files. ... just some ideas ...  Michael Edited 2020-09-20 07:08 by twofingers |
||||
| epsilon Senior Member Joined: 30/07/2020 Location: BelgiumPosts: 255 |
Hi Michael, Now I'm confused. The 0.2 version of max.bas dirExists% looks like this: 'This function checks if given directory name already exists or not FUNCTION dirExists%(dirName$) IF MM.DEVICE$ = "DOS" THEN ... ELSE dirExists% = (DIR$(dirName$, DIR) <> "") ENDIF END FUNCTION i.e. it does use the DIR qualifier, not the FILE qualifier. If there would have been a FILE qualifier there instead, I can understand that you'd get an error on line 290 because the dirExists% check would fail. Re your suggestion, I don't really like the idea of renaming, because then what you've unpacked doesn't truly reflect what's inside the archive anymore. Current behavior is to abort when it runs into an existing file or dir. Maybe that's too abrupt. I checked how unzip behaves on Windows in this case. It prompts you to skip all, to replace all, or to skip or replace on a file by file basis. Maybe I should do something like that. /Epsilon. Epsilon CMM2 projects |
||||
| twofingers Guru Joined: 02/06/2014 Location: GermanyPosts: 1133 |
 Hi Epsilon, you are too fast. Sorry, the issue was in the v0.1. I didn't expect you fixed it so fast. Giving the new folder a new name if the old folder exists is just an idea. I think I will realize that for myself. Regards Michael |
||||
| twofingers Guru Joined: 02/06/2014 Location: GermanyPosts: 1133 |
Hi epsilon, Some feedback ... I have to report a little issue regarding "mac.bas" v0.2 for DOS:  Your code does not tolerate path names with spaces (e.g. "D: mar folder /") if I start MMBASIC from there. eg: D:mar folder/ (includes MMBASIC.exe and "mar.bas") D:mar folder/testfolder (is the folder to archivate) When I start from "D:/marfolder/", I don't get an error message. ----------- I would suggest deleting the temporary file ("ntsh.temp") after the program has run. ----------- A friendly question: May I distribute your MAR.ZIP with my contributions if I use it? And again: Thanks! Michael |
||||
| epsilon Senior Member Joined: 30/07/2020 Location: BelgiumPosts: 255 |
Hi Michael, Thank you for the good feedback! I fixed the issues you reported in V0.3, now on GitHub: https://github.com/epsilon537/mar_cmm2 Feel free to distribute mar.zip as much as you want. However, if you're going to make changes to the tool, I would prefer if you rename it so people can distinguish between your version and mine. Cheers, Epsilon. Epsilon CMM2 projects |
||||
| twofingers Guru Joined: 02/06/2014 Location: GermanyPosts: 1133 |
Hi epsilon, the extraction program is now working as expected, perfect!  I think the MAR program is a candidate for the "Welcome Tape". MAR stands for Maximite ARchiver? Thanks a lot. Kind regards Michael |
||||
thwill Guru Joined: 16/09/2019 Location: United KingdomPosts: 3830 |
My vision of the "Welcome Tape" is that it will only contain programs that can sensibly launched from the menu, thus excluding command-line tools no matter how useful. I think there is definitely room to compile a distribution of "Tools" as well, it's just not something that I am willing (or have the time) to take on. In my opinion as a community we have a growing "problem" in that a lot of great content is being created, but it gets posted to the forum and then vanishes from sight after a week or two unless the author is inclined to keep "banging the drum". For a while "capsikin" tried to keep this list on Fruit of the Shed up-to-date, but it's a largely thankless task. I don't know what the solution is, I suspect it requires two or more dedicated archivists with time on their hands ... unfortunately that isn't me. In the meantime could I suggest that if you've created something great then you consider putting a link to it in your signature. Best wishes, Tom Edited 2020-09-24 22:28 by thwill Game*Mite, CMM2 Welcome Tape, Creaky old text adventures |
||||
| twofingers Guru Joined: 02/06/2014 Location: GermanyPosts: 1133 |
Thanks Tom! I understand!  The point is that the program is extremely helpful when having a Program consists of several parts. (e.g. Lunar Lander) IMHO. I also don't know whether epsilon would even like that ...  Kind regards Michael |
||||
| epsilon Senior Member Joined: 30/07/2020 Location: BelgiumPosts: 255 |
Sure I'd like that, you know, for bragging rights I appreciate the suggestion Michael! But like Tom said, CMM2 needs a software repository, and that's not Welcome Tape. I wouldn't mind using fruitoftheshed.com as the CMM2 repository. It's a wiki. Anybody can create an account and add content. I just added a link to the MAR github page in the utilities section, next to FileBox: http://fruitoftheshed.com/Colour%20MaxiMite%202%20(CMM2).Example-games-and-other-programs.ashx?NoRedirect=1#Utilities_8&NS=Colour%20MaxiMite%202%20(CMM2) If a lot of people start adding content it might get messy quickly, but I think that's a good problem to have Edited 2020-09-25 00:29 by epsilon Epsilon CMM2 projects |
||||
| twofingers Guru Joined: 02/06/2014 Location: GermanyPosts: 1133 |
Okay! I will not continue to insist. Michael |
||||
| epsilon Senior Member Joined: 30/07/2020 Location: BelgiumPosts: 255 |
Hi all, Here is a new version of MAR: Color Maximite 2 Archiving Tool ------------------------------- 0.4: - Combined mac.bas and max.bas into a single program, mar.bas. - Added option to create and extract LZ1 compressed archives. - Created an equivalent Python version, mar.py, intended for running on a host machine (Windows, MacOSX, Linux). - DOS MMBasic version of mac.bas is no longer supported. Please use Python version instead. ... Description ----------- On CMM2, mar.bas allows you to roll up and optionally compress a directory tree into a single file archive, and vice versa. mar.py is an equivalent Python implementation intended for running on Windows, MacOSX or Linux. This allows you to create your archive on a host machine, transfer it over to CMM2 and unpack it there, and vice versa. Usage ----- On CMM2: *mar c <dir> : archive directory <dir> into file <dir>.mar *mar cz <dir> : archive and lz1 compress directory <dir> into file <dir>.mz1 *mar x <archive>.mar : extract <archive>.mar archive *mar xz <archive>.mz1 : extract <archive>.mz1 compressed archive On Windows/MacOSX/Linux: python mar c <dir> : archive directory <dir> into file <dir>.mar python mar cz <dir> : archive and lz1 compress directory <dir> into file <dir>.mz1 python mar x <archive>.mar : extract <archive>.mar archive python mar xz <archive>.mz1 : extract <archive>.mz1 compressed archive Required CMM2 firmware version ------------------------------ V5.06.00 Required Python version ----------------------- 3.x GitHub ------ https://github.com/epsilon537/mar_cmm2 mar.zip Epsilon CMM2 projects |
||||
panky Guru Joined: 02/10/2012 Location: AustraliaPosts: 1094 |
Disappointed to see that the MM DOS version no longer supported - have no desire to learn another language (Python) if I want to learn how your program works of play with any changes - any chance you might re-consider supporting the MM DOS Windows version? Thanks for sharing a great program - I really like the idea behind it. Doug. ... almost all of the Maximites, the MicromMites, the MM Extremes, the ArmMites, the PicoMite and loving it! |
||||
| epsilon Senior Member Joined: 30/07/2020 Location: BelgiumPosts: 255 |
Sorry to disappoint you Doug, but that's not going to happen. A DOS MMBasic version of mar.bas would only run on a Windows machine, so I would still have to create a Python version for MacOSX and Linux. I would end up with three different code bases to maintain. I initially supported DOS MMBasic because I thought code re-use would make it easy, but it didn't. The platform differences between DOS and CMM2 MMBasic are significant and the DOS version is not as far along as the CMM2 version. If you look at the old mac.bas, you'll see lots of IF MM.DEVICE$ = "DOS" conditionals, introducing complexity and a source for bugs in an otherwise simple program. Epsilon CMM2 projects |
||||
Turbo46 Guru Joined: 24/12/2017 Location: AustraliaPosts: 1590 |
I'm sorry too. Bill Keep safe. Live long and prosper. |
||||
| epsilon Senior Member Joined: 30/07/2020 Location: BelgiumPosts: 255 |
Hi all, I added the option to create self-extracting archives to mar.bas: *mar s <dir> : archive directory <dir> into self-extracting file <dir>.mar.bas *mar sz <dir> : archive and lz1 compress directory <dir> into self-extracting file <dir>.mz1.bas When you execute the generated <dir>.mar.bas or <dir>.mz1.bas file, it will create directory <dir> and any subdirs and unpack itself into that directory tree. The idea is to make it easy to publish applications consisting of multiple files (e.g. games). As an example, I attached a self-extracting archive of the SaucerShooter demo game. Please note that firmware version V5.07.00b23 or later is required. The code relies on the new #COMMENT START/END feature. Cheers, Ruben/Epsilon. GitHub: https://github.com/epsilon537/mar_cmm2 mar.zip SaucerShooter.mar.bas.zip Epsilon CMM2 projects |
||||
 Print this page |
