 Notice. New forum software under development. It's going to miss a few functions and look a bit ugly for a while, but I'm working on it full time now as the old forum was too unstable. Couple days, all good. If you notice any issues, please contact me. |
Forum Index : Microcontroller and PC projects : Hexedit binary file editor
Page 1 of 2   |
|||||
| Author | Message | ||||
epsilon Senior Member  Joined: 30/07/2020 Location: BelgiumPosts: 255 |
Hi all, I made a hex editor for binary files. See description of the tool below. I hope that some of you will find it useful. I might derive a memory editor (a peek/poke front-end) from this once the dust has settled. Feedback is welcome of course, in particular if you find that some key bindings aren't working correctly. I have a Belgian keyboard, which is not supported. I have it currently configured as a french keyboard, which is not a perfect match, some I'm not 100% sure I got all key combos correctly. Let me know please. Any other feedback is welcome too of course, including coding practices. GitHub: https://github.com/epsilon537/hexedit_cmm  hexedit.zip Cheers, Epsilon. Description ----------- HexEdit allows you to edit binary files, i.e. files that you can't edit with a regular text editor such as the built-in editor. The file's contents are shown as hexadecimal byte values and as ASCII code in two side-by-side blocks. Changes can be made both in the hex table section and in the ASCII section. The usual file navigation with cursors keys, page up&down, home, end, etc. should work as expected. Hexedit is a full-screen console-only editor. I does not work over serial, sorry. Press F1 to get help on key bindings. Current limitations ------------------- File Size Limit: 1MB Byte mode only. No support (yet?) for 16-bit and 32-bit modes. Epsilon CMM2 projects |
||||
datawiz Newbie  Joined: 10/08/2020 Location: United StatesPosts: 26 |
This is great! I'll have to take a look at it after work. Out of curiosity, what was the reason for not working over serial? Was it because of screen positioning, the use of the modifier keys (alt, control, shift, win/gui), or something else? Thanks, Rich/dw |
||||
Sasquatch Guru  Joined: 08/05/2020 Location: United StatesPosts: 377 |
This is great! I was just thinking about something like this earlier today. The ability to poke (or at least peek) around in memory would also be interesting. Keep up the good work! -Carl |
||||
| epsilon Senior Member Joined: 30/07/2020 Location: BelgiumPosts: 255 |
Thank you! The reason for not working over serial is laziness mostly  I'm using some graphics functions: a line across the screen, the help pop-up is a sprite... The footer line is also continuously being updated to reflect current status, which doesn't work well over serial, unless you make it a bit smarter probably. Full-screen updates such as page scroll are also a bit slow over serial depending on your baud rate settings. Anyway, nothing fundamental. I'm pretty sure the tool can be made to work over serial. Modifier keys appear to work over serial. Cheers, Epsilon. Epsilon CMM2 projects |
||||
elk1984 Senior Member Joined: 11/07/2020 Location: United KingdomPosts: 228 |
Yes, thank you. Something that keeps me on the MM rahter than using a PC to fill in gaps helps keep the retro dream of MM only and self contained boot to BASIC vision alive. We all know a PC can do X, but back in the 8 bit days, there was no move it to a PC to do X available.... Sentimental, yes. But that's why I'm coding on the CMM rather than learning something different. |
||||
| Tinine Guru Joined: 30/03/2016 Location: United KingdomPosts: 1646 |
So HexEdit runs on the CMM2? It didn't work for me on MMBASIC-DOS. |
||||
| epsilon Senior Member Joined: 30/07/2020 Location: BelgiumPosts: 255 |
Yes, this is CMM2 only. My bad, I should have mentioned that. Sometimes I forget there are other platforms  I'll update this is in the readme as well. Epsilon CMM2 projects |
||||
| zeitfest Guru Joined: 31/07/2019 Location: AustraliaPosts: 581 |
there is a hexeditor for windows etc Edited 2020-09-30 19:53 by zeitfest |
||||
| epsilon Senior Member Joined: 30/07/2020 Location: BelgiumPosts: 255 |
Hi elk1984. I'm glad you wrote this. I was a bit worried that nobody would get it. You can of course edit your binary files on PC, with hex editors that are probably much fancier than mine, but where's the fun in that. Epsilon CMM2 projects |
||||
twofingers Guru Joined: 02/06/2014 Location: GermanyPosts: 1593 |
Hi epsilon, as always: good job! A useful tool. The code is crystal clear. Thanks!  Some comments: I think the memory is being used too much. I cannot start the program with "Option RAM" (which I usually work with). With "Option Flash" I cannot open the help popup (F1). Some ideas (only if desired): 1. You could reduce BUF_SIZE% (e.g. to 70000-75000). 2. You might add a decimal mode. 3. You could check if "Long Strings" are an improvement. You are giving away half of the memory space with "fileBuf$ () LENGTH 1". Kind regards Michael PS I like the idea sprites to use as a popup. Edited 2020-09-30 21:26 by twofingers causality ≠correlation ≠coincidence |
||||
| epsilon Senior Member Joined: 30/07/2020 Location: BelgiumPosts: 255 |
Thanks Michael. Good feedback! I'll make sure the tool works with those options. 1. I intend to add support for arbitrary long files through some kind of paging mechanism, so I can set BUF_SIZE smaller without limiting functionality too much. 2. I was saving that for decedit. Just kidding. 3. Yes, I think I may switch to long strings, at least for the 'modified' array. It's really just a boolean array, so having a string length[1] array for that purpose is 16x more wasteful than it has to be! Epsilon CMM2 projects |
||||
jirsoft Guru Joined: 18/09/2020 Location: Czech RepublicPosts: 533 |
Hi epsilon, I took just short look on your Hexedit and looks really good! For the memory problem I would solve with stack of changes: pointer% = -1 DIM changes%(10000) '10k of changes per hand looks enough Every time you changes a byte, you simply put into changes% new value, old value and offset pointer% = pointer% + 1 changes%(pointer%) = (offset_in_file% << 16) + (old_value% AND 255) << 8 + (new_value% AND 255) You can then check if some change isn't already in stack and on write you just save the stack... But just an idea, I'm still waiting for my CMM2, so I can't try it  JirSoft Jiri Napoleon Commander and SimplEd for CMM2 (GitHub), Â CMM2.fun |
||||
vegipete Guru Joined: 29/01/2013 Location: CanadaPosts: 1132 |
Very useful program. I'm having a few problems though: Error in line 1015: Function name + variable name must be less than 33 characters for STATIC variables I don't understand why this occurs. Changing the function name to "checkKey" fixed it.Error in line 486: Not enough heap memory I also saw a similar error after using CTL-C to quit. I needed to type "NEW" at the command prompt to get things to behave again.Some thoughts for memory use: When you use DIM fileBuf$(BUF_SIZE%) LENGTH 1 each character ends up taking two bytes.Perhaps instead, you could use screen pages for your data buffers. Each MODE 1,8 page is 480,000 bytes long. You could PEEK and POKE directly into that to read and store values. For example: MODE 1,8 PAGE WRITE 6 ' use page 6 as data buffer CLS ' clear buffer to 0 PAGE WRITE 0 ' back to visible screen MemAddr% = MM.INFO(PAGE ADDRESS 6) ' get address of first byte in data buffer value = PEEK(BYTE MemAddr%) ' read first byte from the buffer value2 = PEEK(BYTE MemAddr% + 1000) ' read thousandth byte POKE BYTE MemAddr% + 1000, value2 + 7 ' store value in thousandth position As an alternate to PEEK and POKE, the PIXEL function and command would also allow directly reading and writing bytes to the buffer, although you would have to watch your active page and calculate x and y coordinates. The PAGE COPY commands could be useful too. Viewing the entire buffer might even show interesting things about the binary file. Finally, the FRAMEBUFFER commands are worth a look, because they could allow different buffer sizes. For example, "FRAMEBUFFER CREATE 1000,1000" should create a 1 megabyte buffer. ========= Could non-printing characters on the ASCII side be displayed as "." instead of " "? They would be easier to see on screen. ========= Very nice work! You source code is nice and easy to read and understand. This will become a very nice utility program. Visit Vegipete's *Mite Library for cool programs. |
||||
| epsilon Senior Member Joined: 30/07/2020 Location: BelgiumPosts: 255 |
Yes, I considered that option. This would also allow for an easy undo implementation. The reason I decided against it is that worst case, e.g. with a big ctrlF fill operation, it eats up a ton of memory just the same. I started toying with the idea of making a text editor. There I would definitely use this changelist/stack concept. As said, the modified$ array can be made 16x more space efficient, and the fileBuf$ array can made be 2x more space efficient. If I combine that with paging for really large files (just having a section of the file in memory instead of the whole thing), I think we're in good shape. The wait can be so long... but it's worth it! Hang in there! Epsilon CMM2 projects |
||||
| epsilon Senior Member Joined: 30/07/2020 Location: BelgiumPosts: 255 |
SUB checkKeyPress STATIC nConsecHomePresses% = 0, nConsecEndPresses% = 0 Strange. The length of the static and the sub combined does not exceed 32 characters. It is exactly 32 characters in fact. I'll make it shorter, but I don't understand why I'm not hitting that same error. Different FW version maybe? I'm at 5.05.05. Very interesting idea to use FRAMEBUFFER as a kind of memory allocator! I might do that. At 8 bits, the limit is at 1600x1200 = 1.875MB. That's good enough for my purposes here. Will do. Thanks for the great feedback! Epsilon CMM2 projects |
||||
thwill Guru Joined: 16/09/2019 Location: United KingdomPosts: 4311 |
Hi "epsilon", Nice work, it's good to see some tools being produced as well as games. I'm not sure if it is applicable/useful but as part of writing Z-MIM I wrote a file-backed virtual memory implementation for the original Colour Maximite which you might be able to repurpose: https://github.com/thwill1000/zmim/blob/master/src/mem_cmm1.inc Best wishes, Tom MMBasic for Linux, Game*Mite, CMM2 Welcome Tape, Creaky old text adventures |
||||
| twofingers Guru Joined: 02/06/2014 Location: GermanyPosts: 1593 |
Hi Ruben, I think a useful feature would be to save parts of a file to repair corrupt files (CMM2 bug) without using a PC. The file size is not that important (<100KB usually). Regards Michael causality ≠correlation ≠coincidence |
||||
| matherp Guru Joined: 11/12/2012 Location: United KingdomPosts: 10310 |
Earlier versions didn't trap the error (same in all versions of MMBasic) which could cause major runtime errors. The latest version of the CMM2 S/W does trap the error but you have found a very strange bug. Try: SUB checkKeyPress STATIC nConsecHomePresses%= 0, nConsecEndPresses%= 0 For some reason the space before the = is included in the variable name length test. I'll fix this in the next beta. Re the memory issue. Use an integer array and access individual bytes with peek and poke - check out PEEK(VARADDR arrayname%()) which gives you the address of the beginning of the array in memory |
||||
| vegipete Guru Joined: 29/01/2013 Location: CanadaPosts: 1132 |
Ignore my suggestion to use the PIXEL function and command to read and write the data buffer stored in video pages. These don't give byte access. Instead they work with colour maps which would just confuse things for this application. I've used matherp's suggestion of using a large integer array with peek and poke. It works well. But video pages or the framebuffer might be even better because of the copy, cls and blit functions. I like jirsoft's idea of a stack of changes, especially because of the undo possibilities. And while we are loading you down with all kinds of extras for you program, how about a file selector dialog? ;-) Maybe I'll work on something. Your awesome 'help screen as sprite' idea has given me an idea... Visit Vegipete's *Mite Library for cool programs. |
||||
| epsilon Senior Member Joined: 30/07/2020 Location: BelgiumPosts: 255 |
Hi all, I uploaded V0.2 to GitHub: https://github.com/epsilon537/hexedit_cmm I also put a link of Fruit of the Shed. These are the changes: - Documentation updates. - Fixed error 'Function name + variable name must be less than 33 characters'. - Using "." instead of " " in ASCII block locations past file size. - Using "?" instead of non-printable characters in ASCII block. - Fixed program crash when insert is pressed. - Added Loading..., Inserting..., Deleting..., Filling... indications. - Fixed heap memory issues. - Increased File Size Limit to 1.6MB Using integer array as well as framebuffer to fix the memory issue. Thanks again for the suggestion. Epsilon CMM2 projects |
||||
| Page 1 of 2 |
|||||
 Print this page |
 |
The Back Shed's forum code is written, and hosted, in Australia. | © JAQ Software 2025 |
