Home
JAQForum Ver 24.01
Log In or Join  
Active Topics
Local Time 05:41 02 Aug 2025 Privacy Policy
Jump to

Notice. New forum software under development. It's going to miss a few functions and look a bit ugly for a while, but I'm working on it full time now as the old forum was too unstable. Couple days, all good. If you notice any issues, please contact me.

Forum Index : Microcontroller and PC projects : Trimming XMODEM padding charactres

Author Message
TassyJim

Guru

Joined: 07/08/2011
Location: Australia
Posts: 6283
Posted: 04:27am 23 Feb 2023
Copy link to clipboard 
Print this post

XMODEM has served us well for 40 years but it does have one undesired effect.



XMODEM sends data in 128 byte packets and the final packet gets padded out to 128 bytes with either chr$(0) or chr$(26). Either is valid and it will depend on the sending program so you have to be prepared to handle both padding characters.

This padding is not normally a problem. MMBasic doesn’t care and I have yet to see an image format that gets confused.

If you are using a file for data storage and APPEND to the file, and the file has been padded, you are likely to have problems reading the data.

This Basic program strips the offending zero or ^Z characters form any file with an extension in the safefiles list. I am not willing to trim graphics or data (bin etc) files but if you have one that always ends with a standard ASCII character such as <CR>, you can add the extension type to the list.

The program checks the extension and file size. The assumption is that if the filesize is not a multiple of 128 bytes, it has not been transferred by XMODEM.
Only the last 128 bytes are checked to save time and XMODEM will not add any more than 128 characters so that is a valid decision.

It runs on all files on the current directory

 ' XMODEM trim by TassyJim 23 Feb 23
 OPTION EXPLICIT
 OPTION DEFAULT INTEGER
 DIM f$, tfile$ = "tempfile.txt"
 DIM safefile$= " bas py html htm txt csv log "
 DIM filetype$, packet$
 DIM INTEGER n, k, fsize, blocks
 
 f$ = DIR$("*",FILE)
 DO WHILE f$ <> ""
   'print f$
   filetype$ = FIELD$(f$,2,".")
   'print f$, filetype$
   IF INSTR(safefile$,filetype$) THEN
     fsize = MM.INFO(FILESIZE f$)
     IF fsize MOD 128 = 0 THEN
       blocks = fsize/128
       OPEN tfile$ FOR output AS #2
       OPEN f$ FOR INPUT AS #3
       FOR n = 1 TO blocks-1
         packet$ = INPUT$(128,#3)
         PRINT #2, packet$;
       NEXT n
       packet$ = INPUT$(128,#3)
       CLOSE #3
       FOR k = LEN(packet$) TO 1 STEP -1
         IF MID$(packet$,k,1) <> CHR$(0) AND MID$(packet$,k,1) <> CHR$(26) THEN
           EXIT FOR
         ENDIF
       NEXT k
       packet$ = LEFT$(packet$, k)
       PRINT #2, packet$;
       CLOSE #2
       KILL f$
       RENAME tfile$ AS f$
       PRINT f$, fsize, MM.INFO(filesize f$)
     ENDIF
   ENDIF
   f$ = DIR$()
 LOOP


Jim
Edited 2023-02-23 14:29 by TassyJim
VK7JH
MMedit
 
Volhout
Guru

Joined: 05/03/2018
Location: Netherlands
Posts: 5090
Posted: 12:54pm 23 Feb 2023
Copy link to clipboard 
Print this post

Hi Jim,

I know the problems, since I edit programs in the build in editor to remove the last bugs (main development is in MMEdit) and then transfer back to the PC environment. I normally use a "notepad" alike editor to remove the trailing 0x00's and save the file again with same name.

Thanks for this tool. It may come in handy.
Does MMCC file manager fix this also ? Trim the extra 0's ?

Regards,

Volhout
Edited 2023-02-23 22:54 by Volhout
PicomiteVGA PETSCII ROBOTS
 
TassyJim

Guru

Joined: 07/08/2011
Location: Australia
Posts: 6283
Posted: 08:02pm 23 Feb 2023
Copy link to clipboard 
Print this post

  Volhout said  
Does MMCC file manager fix this also ? Trim the extra 0's ?


Not yet.
The previous versions (pre 5.0) did have the option to trim when you transfer files to the PC. I will probably add the option to the current versions.

The method used was to retrieve the file size before transfer and trim all files to that size.
The problem arises if you hadn't trimmed the file previously and the file size includes the padding.
I will have to get smarter this time.

When you open a bas file in MMEdit, the padding does get trimmed.

Jim
Edited 2023-02-24 06:04 by TassyJim
VK7JH
MMedit
 
disco4now

Guru

Joined: 18/12/2014
Location: Australia
Posts: 1003
Posted: 11:32pm 23 Feb 2023
Copy link to clipboard 
Print this post

Hi Jim,
I discovered this when trying some of Toms (@twill's) unit tests. The md5 encrypt tests were failing because of the padding on the test files. Looking forward to it being an option in MMCC.
Gerry
Latest F4 Latest H7 FotS
 
thwill

Guru

Joined: 16/09/2019
Location: United Kingdom
Posts: 4311
Posted: 11:52pm 23 Feb 2023
Copy link to clipboard 
Print this post

  disco4now said  I discovered this when trying some of Toms (@twill's) unit tests.


Blimey, I'm going to have to lie down to recover from the shock that anyone other than myself has run them ... fortunately it's bed-time.

Best wishes,

Tom
MMBasic for Linux, Game*Mite, CMM2 Welcome Tape, Creaky old text adventures
 
Print this page


To reply to this topic, you need to log in.

The Back Shed's forum code is written, and hosted, in Australia.
© JAQ Software 2025