Home
JAQForum Ver 24.01
Log In or Join  
Active Topics
Local Time 05:13 02 Aug 2025 Privacy Policy
Jump to

Notice. New forum software under development. It's going to miss a few functions and look a bit ugly for a while, but I'm working on it full time now as the old forum was too unstable. Couple days, all good. If you notice any issues, please contact me.

Forum Index : Microcontroller and PC projects : Hexedit binary file editor

     Page 2 of 2    
Author Message
matherp
Guru

Joined: 11/12/2012
Location: United Kingdom
Posts: 10315
Posted: 05:38pm 01 Oct 2020
Copy link to clipboard 
Print this post

  Quote  Using integer array as well as framebuffer to fix the memory issue.


The framebuffer takes memory from the same area as arrays so using it has no advantage over accessing an integer array using peek and poke at the byte level
 
vegipete

Guru

Joined: 29/01/2013
Location: Canada
Posts: 1132
Posted: 06:36pm 01 Oct 2020
Copy link to clipboard 
Print this post

  matherp said  The framebuffer takes memory from the same area as arrays so using it has no advantage over accessing an integer array using peek and poke at the byte level

"MATH SET nbr, array()" beats "PAGEWRITE FRAMEBUFFER : CLS" for more flexible filling of byte values.

However, "FRAMEBUFFER BACKUP" and "FRAMEBUFFER RESTORE" need loops with arrays. Creative use of blit can quickly copy large (and small) chunks of data. There's no telling what creative and confusing actions could be accomplished with sprites. And I still think viewing the file on screen could on occasion be useful.

=====
Aside: What is the difference between "FRAMEBUFFER WRITE" and "PAGEWRITE FRAMEBUFFER"?
Visit Vegipete's *Mite Library for cool programs.
 
jirsoft

Guru

Joined: 18/09/2020
Location: Czech Republic
Posts: 533
Posted: 07:55pm 01 Oct 2020
Copy link to clipboard 
Print this post

  epsilon said  
  jirsoft said  Hi epsilon,
For the memory problem I would solve with stack of changes:

pointer% = -1
DIM changes%(10000) '10k of changes per hand looks enough

Every time you changes a byte, you simply put into changes% new value, old value and offset

pointer% = pointer% + 1
changes%(pointer%) = (offset_in_file% << 16) + (old_value% AND 255) << 8 + (new_value% AND 255)


You can then check if some change isn't already in stack and on write you just save the stack...

JirSoft


Yes, I considered that option. This would also allow for an easy undo implementation.
The reason I decided against it is that worst case, e.g. with a big ctrlF fill operation, it eats up a ton of memory just the same.
I started toying with the idea of making a text editor. There I would definitely use this changelist/stack concept.

As said, the modified$ array can be made 16x more space efficient, and the fileBuf$ array can made be 2x more space efficient. If I combine that with paging for really large files (just having a section of the file in memory instead of the whole thing), I think we're in good shape.

  jirsoft said  
But just an idea, I'm still waiting for my CMM2, so I can't try it  
JirSoft


The wait can be so long... but it's worth it! Hang in there!


It's also possible to use just small buffer in RAM (what you can see on screen) and the rest have in "secret" file on SD. So you are not limited by RAM..
1. Undo stack on SD (limitless)
or
2. Mirror file, where you can do all the changes and on end just rename original to .OLD and mirror to original

JirSoft
Jiri
Napoleon Commander and SimplEd for CMM2 (GitHub),  CMM2.fun
 
epsilon

Senior Member

Joined: 30/07/2020
Location: Belgium
Posts: 255
Posted: 07:49am 02 Oct 2020
Copy link to clipboard 
Print this post

  twofingers said  Hi Ruben,

I think a useful feature would be to save parts of a file to repair corrupt files (CMM2 bug) without using a PC.
The file size is not that important (<100KB usually).

Regards
Michael


Only the modified parts of the file that's being edited are saved back.
I don't understand the nature of that corrupt file bug well enough to see how hexedit can help. In my case I just lose the bottom 10 lines of the file. I assumed the file just got truncated, in which case hexedit won't be very helpful. However, if all/most of the contents of the file are still there and are just not being displayed properly because a 0 byte or something like that snuck in, then you can use hexedit to fix that.
Next time I run into that corruption issue, I'll take a look with hexedit.
Epsilon CMM2 projects
 
twofingers

Guru

Joined: 02/06/2014
Location: Germany
Posts: 1593
Posted: 12:45pm 02 Oct 2020
Copy link to clipboard 
Print this post

  Quote  I don't understand the nature of that corrupt file bug well enough to see how hexedit can help.

Hi Ruben,
in my case the bug changes the file size eg from 25000 to 45000 bytes and fills it up with garbage (s. screenshot). The editor can no longer load the file because the line length is too long. I usually delete the garbage with Notepad ++ on Windows. However, some program lines at the end are lost.

Having a Hexedit that can copy bytes 1 to &Hxxxxxxxxx to a new file could have helped.

But this is no problem. I can just write a little CMM2 tool that will copy a file while the line length is less than 240 bytes.

EDIT: I did it. I think something like this can do the job.
'***************************************************
' This tool may help for corrupted ".BAS"-files
' with a LINE IS TOO LONG issue (no warranty)
'****************************************************
OPTION BASE 0
OPTION EXPLICIT
MODE 1,8

do
 CLS
 input "Corrupted file:";FN$

 if FN$="" then
   PRINT "Bye!"
   END
 endif

 FileExist=MM.INFO(FILESIZE FN$)<>-1

 if Not FileExist then
    PRINT "Error: File '"+FN$+"' not found!"
    PAUSE 1000
 endif
loop until FileExist
open FN$ for input as #1

open "repfile.bas" for output as #2

do
 a$=input$(1,#1)
 b$=b$+a$
 i=i+1
 if a$=chr$(13) then
   a$=input$(1,#1)
   b$=b$+a$
   i=i+1
   if a$=chr$(10) then
     PRINT b$;
     PRINT #2,b$;
     b$=""
     i=0
   end if
 end if
loop until EOF(#1) or i >= 240

close #1,#2
PRINT "done!"


Thank you for your attention.

Michael

EDIT: Notepad++ screenshot corrupted file:



Edited 2020-10-03 05:47 by twofingers
causality ≠ correlation ≠ coincidence
 
epsilon

Senior Member

Joined: 30/07/2020
Location: Belgium
Posts: 255
Posted: 09:47am 04 Oct 2020
Copy link to clipboard 
Print this post

Happy Sunday all,

I uploaded hexedit v0.3 to GitHub:

https://github.com/epsilon537/hexedit_cmm

ChangeLog
---------
0.3:
- Fixed cosmetic bug where first character sometimes remains inverted right after loading a file.
- Fixed bug with goto when in ASCI column.
- Fixed ctrlF fill bug.
- New feature: Ctrl-T toggles between different word sizes: 8-bit, 16-bit, 32-bit and 64-bit.
- New feature: Ctrl-E exports part of file as text or as binary.
- Reduced redraw artifacts when editing.
- Added file size to footer.

0.2:
- Documentation updates.
- Fixed error 'Function name + variable name must be less than 33 characters'.
- Using "." instead of " " in ASCII block locations past file size.
- Using "?" instead of non-printable characters in ASCII block.
- Fixed program crash when insert is pressed.
- Added Loading..., Inserting..., Deleting..., Filling... indications.
- Fixed heap memory issues.
- Increase File Size Limit to 1.6MB


Michael, the export binary feature should do what you want.

Cheers,
Epsilon.
Epsilon CMM2 projects
 
twofingers

Guru

Joined: 02/06/2014
Location: Germany
Posts: 1593
Posted: 04:47pm 04 Oct 2020
Copy link to clipboard 
Print this post

Hi Ruben,

the export function works for my corrupt file as expected. Thanks!

One idea: I would suggest setting the default values for the export function: startaddr = 0, endaddr = current cursorpos.

I changed the SUB printHeader in my copy of your Hexedit:
SUB printHeader
 LOCAL header$ = "Hexedit V"+VERSION$+" by Epsilon.";
 LOCAL byteNr$ = "             00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F  0123456789ABCDEF"

 'Print inverted
 PRINT @(0,0,2) header$ + SPACE$(MM.HRES/MM.INFO(FONTWIDTH) - LEN(header$))
 'Print CYAN)
 color rgb(CYAN)
 PRINT @(0,MM.INFO(FONTHEIGHT),0) byteNr$
 color rgb(WHITE)
END SUB

Two observations:
1. The modified indicator works not correct if I delete/insert a Byte.
2. Using a basic function as a variable is not a good idea (loc% - loc()).  

Your code is just inspiring! Thanks!

Kind regards
Michael
causality ≠ correlation ≠ coincidence
 
epsilon

Senior Member

Joined: 30/07/2020
Location: Belgium
Posts: 255
Posted: 01:15pm 05 Oct 2020
Copy link to clipboard 
Print this post

  twofingers said  
One idea: I would suggest setting the default values for the export function: startaddr = 0, endaddr = current cursorpos.


I'll add that one to the feature request heap.
At some point I'm going to have to write a smart INPUT function that supports default arguments, Esc key to abort etc. Right now I'm still just using MMBasic's INPUT.

  twofingers said  
I changed the SUB printHeader in my copy of your Hexedit:
SUB printHeader
 LOCAL header$ = "Hexedit V"+VERSION$+" by Epsilon.";
 LOCAL byteNr$ = "             00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F  0123456789ABCDEF"

 'Print inverted
 PRINT @(0,0,2) header$ + SPACE$(MM.HRES/MM.INFO(FONTWIDTH) - LEN(header$))
 'Print CYAN)
 color rgb(CYAN)
 PRINT @(0,MM.INFO(FONTHEIGHT),0) byteNr$
 color rgb(WHITE)
END SUB



Great idea! I plugged that right in. Thanks!

  twofingers said  
Two observations:
1. The modified indicator works not correct if I delete/insert a Byte.


In the sense that everything after the deleted/inserted byte shows up as modified?
Maybe I broke the principle of least surprise there. When a byte is deleted, all bytes behind it move up one position, so are 'modified' compared to the original. It sounds reasonable to me, but I can imagine opinions on that may vary.
One annoying consequence is that insert followed by delete, or vice versa, does not 'unmodify' the bytes after it again. Same things as overwriting a byte sequence, then going back and overwriting it again with the original byte values. Once a location is marked as modified it stays modified until the file is saved.

  twofingers said  
2. Using a basic function as a variable is not a good idea (loc% - loc()).  


Oops! I'll switch to pos then.    

In the meantime I uploaded v0.4.

https://github.com/epsilon537/hexedit_cmm

ChangeLog
---------
0.4:
- New feature: Ctrl-S find byte sequence.
- Added column header, contributed by Michael "twofingers".
- Modified some key bindings (Sorry!). Load and Save moved to F3 and F2. Ctrl-S is now bound to the search function.
- Increased file size limit to 3MB.

Apologies for changing the keybindings. I figured out could still get away with it less than a week after the initial release. I just had to bind the search function to either Ctrl-S or Ctrl-F, so Load and Save moved to F3 and F2.

Cheers,
Ruben.
Edited 2020-10-06 17:06 by epsilon
Epsilon CMM2 projects
 
twofingers

Guru

Joined: 02/06/2014
Location: Germany
Posts: 1593
Posted: 03:35pm 06 Oct 2020
Copy link to clipboard 
Print this post

Hi Ruben,
  epsilon said  
At some point I'm going to have to write a smart INPUT function that supports default arguments, Esc key to abort etc. Right now I'm still just using MMBasic's INPUT.

A smart INPUT function is a very good idea! I made one years ago for my "File Selector" for Maximite.
A updated version is here (INPUT function).

  epsilon said  
  twofingers said  
Two observations:
1. The modified indicator works not correct if I delete/insert a Byte.


In the sense that everything after the deleted/inserted byte shows up as modified?

No, just the indicator in the footer.



Kind regards  
Michael
Edited 2020-10-07 19:19 by twofingers
causality ≠ correlation ≠ coincidence
 
epsilon

Senior Member

Joined: 30/07/2020
Location: Belgium
Posts: 255
Posted: 12:53pm 07 Oct 2020
Copy link to clipboard 
Print this post

Hi Michael,

I'll take a look at that smart input function and file selector. Thank you!

  Quote  No, just the indicator in the footer.


Got it (finally!). I uploaded a fix.

https://github.com/epsilon537/hexedit_cmm
Epsilon CMM2 projects
 
twofingers

Guru

Joined: 02/06/2014
Location: Germany
Posts: 1593
Posted: 03:25pm 07 Oct 2020
Copy link to clipboard 
Print this post

Hi Ruben,
  epsilon said  ... I'll take a look at [...] file selector.
The Maximite File Selector is just an example  of the use of a (more or less) smart input function (MaskEdit). It doesn't work for CMM2!

Thanks for the updated Hexedit!

Kind regards
Michael
causality ≠ correlation ≠ coincidence
 
     Page 2 of 2    
Print this page


To reply to this topic, you need to log in.

The Back Shed's forum code is written, and hosted, in Australia.
© JAQ Software 2025