TheBackShed.com

Forum Index : Microcontroller and PC projects : Automatic backup software?

Page 1 of 2

Author

Message

Grogster

Admin Group

Joined: 31/12/2012
Location: New Zealand
Posts: 9593

Posted: 06:44am 04 Sep 2024

Copy link to clipboard

Print this post

Hello all.

I currently use iDrive, and it works really well, and only ever backs up files that have changed or been created since the last sync.

Is there a software that anyone can reccomend, that I can run on the local machine, that will backup to a permanently connected USB flash or SSD drive?

It's getting hard to remember exactly what files I opened, changed or created on any one day....

Smoke makes things work. When the smoke gets out, it stops!

JohnS
Guru

Joined: 18/11/2011
Location: United Kingdom
Posts: 4038

Posted: 08:05am 04 Sep 2024

Copy link to clipboard

Print this post

Grogster said that I can run on the local machine, that will backup to a permanently connected USB flash or SSD drive?

Which OS is on the local machine?

John

TassyJim

Guru

Joined: 07/08/2011
Location: Australia
Posts: 6270

Posted: 08:16am 04 Sep 2024

Copy link to clipboard

Print this post

I'm a cheapscape.
For Windows, XCOPY or ROBOCOPY and a schedule

There are equivalent options for linux.

Jim

VK7JH
MMedit

twofingers

Guru

Joined: 02/06/2014
Location: Germany
Posts: 1577

Posted: 08:22am 04 Sep 2024

Copy link to clipboard

Print this post

Hi Grogster,
this is what ChatGPT:says:

Quote Du:
Is there a Windows software , that I can run on the local machine, that will backup only ever backs up files that have changed or been created since the last sync. to a permanently connected USB flash or SSD drive?
ChatGPT:

Yes, there are several Windows software options that can back up only files that have changed or been created since the last sync, a process commonly known as "incremental backup." Here are some popular options:

1. File History (Built-in Windows Tool)
Description: Windows has a built-in tool called File History that automatically backs up files in your libraries, desktop, contacts, and favorites to a connected drive.
How It Works: File History only backs up files that have been changed or created since the last backup.
How to Use: You can access it through the Control Panel or Settings > Update & Security > Backup.
2. FreeFileSync
Description: FreeFileSync is a free and open-source software that lets you compare and synchronize files and folders.
Incremental Backup: You can set it up to perform incremental backups by syncing only the files that have changed.
Additional Features: It supports mirror sync, versioning, and can handle large file structures efficiently.
Website: FreeFileSync
3. SyncBack
Description: SyncBack is a versatile backup and synchronization software with a free version available.
Incremental Backup: It allows you to back up only changed or new files, saving time and storage space.
Additional Features: Offers advanced options like file versioning, encryption, and scheduling.
Website: SyncBack
4. AOMEI Backupper
Description: AOMEI Backupper is a comprehensive backup solution that supports incremental and differential backups.
Incremental Backup: You can configure it to back up only the files that have changed since the last backup.
Additional Features: Includes options for full system backups, disk cloning, and scheduling.
Website: AOMEI Backupper
5. Cobian Backup
Description: Cobian Backup is a free, multi-threaded program that can be used to schedule and automate backups.
Incremental Backup: It supports incremental backups, which only back up the files that have been modified since the last backup.
Additional Features: Offers encryption, compression, and remote backup options.
Website: Cobian Backup
6. Macrium Reflect Free
Description: Macrium Reflect is a disk imaging software that also provides file backup capabilities.
Incremental Backup: The free version supports differential backups (backing up only changes since the last full backup), while the paid version supports full incremental backups.
Additional Features: Provides disk imaging, cloning, and scheduling features.
Website: Macrium Reflect
7. Goodsync
Description: GoodSync is a reliable file backup and synchronization software that offers real-time data synchronization.
Incremental Backup: It supports both one-way and two-way sync, ensuring only new or changed files are backed up.
Additional Features: Can sync with cloud services and remote locations.
Website: GoodSync

These tools are widely used and can be configured to back up only the files that have changed, which helps save time and storage space on your USB flash or SSD drive.

*********************************************************************

Du:
How about xcopy?
ChatGPT:

Yes, you can use the xcopy command in Windows to perform incremental backups, where only files that have changed or been created since the last backup are copied. Here's how you can set it up:

Basic xcopy Command for Incremental Backup
The basic syntax for using xcopy for incremental backup is:

cmd

xcopy [source] [destination] /D /E /C /H /R /Y

Explanation of the Parameters:
[source]: The directory you want to back up.
[destination]: The directory where you want to store the backup.
/D: Copies only files that are newer than the existing files in the destination. If no date is given, it will copy only the files that have been modified since the last backup.
/E: Copies all subdirectories, including empty ones.
/C: Continues copying even if errors occur.
/H: Copies hidden and system files.
/R: Overwrites read-only files in the destination.
/Y: Suppresses prompting to confirm overwriting a file.
Example:
Suppose you want to back up the C:\MyDocuments directory to a USB drive located at E:\Backup. You can use the following command:

cmd

xcopy "C:\MyDocuments" "E:\Backup" /D /E /C /H /R /Y

How It Works:
The xcopy command will scan the C:\MyDocuments directory and compare it with the E:\Backup directory.
It will only copy files that have been modified since the last time the command was run or files that are not already present in the destination.
Automating the Process with a Batch File:
You can automate this process by creating a batch file (backup.bat) with the xcopy command:

Open Notepad and paste the xcopy command.
Save the file as backup.bat.
You can then run this batch file whenever you want to perform an incremental backup, or you can schedule it using Task Scheduler.
Example Batch File (backup.bat):
cmd

@echo off
xcopy "C:\MyDocuments" "E:\Backup" /D /E /C /H /R /Y
echo Backup completed successfully!
pause

Notes:
xcopy is a powerful tool but does not offer advanced features like versioning or differential backups. For more sophisticated needs, dedicated backup software might be more appropriate.
Ensure that the destination drive has enough space to accommodate the files being backed up.
This approach is simple and effective for straightforward backup tasks.

ChatGPT Backup_.zip

I suspect other members have more knowledgeable answers?

Best regards
Michael

causality ≠ correlation ≠ coincidence

JohnS
Guru

Joined: 18/11/2011
Location: United Kingdom
Posts: 4038

Posted: 09:26am 04 Sep 2024

Copy link to clipboard

Print this post

On Linux, rsync (can find & copy just the changes).

Use it over ssh if you want encryption as the data passes.

John

Plasmamac

Guru

Joined: 31/01/2019
Location: Germany
Posts: 573

Posted: 11:02am 04 Sep 2024

Copy link to clipboard

Print this post

For win : driveimage.xml if you clone a drive , or need to save a partition

Plasma

lizby
Guru

Joined: 17/05/2016
Location: United States
Posts: 3363

Posted: 11:53am 04 Sep 2024

Copy link to clipboard

Print this post

TassyJim said I'm a cheapscape.
For Windows, XCOPY or ROBOCOPY and a schedule

Ditto, with a scheduled XCP.BAT file, with the newer (/D), no prompt for overwrite (/Y), subdirectories (/S), and modified (/M) flags set; e.g.:

xcopy /D /Y /S /M c:\dl\micromite\*.* %ddir%\dl\micromite\

I rarely add a low-level folder, so don't often have to change the batch file. "%ddir% is set to the drive (and perhaps folder) I want to copy to. I have variants to copy to a net drive or to USB or SD card.

I have an UNXCP.BAT file to copy to a different PC.

PicoMite, Armmite F4, SensorKits, MMBasic Hardware, Games, etc. on fruitoftheshed

ville56
Senior Member

Joined: 08/06/2022
Location: Austria
Posts: 221

Posted: 12:29pm 04 Sep 2024

Copy link to clipboard

Print this post

syncthing could be worth a try, runs on windows, Linux/Unix, Android and there are packages available even for off-the-shelf NAS boxes. Has lots of configuration possibilites.

73 de OE1HGA, Gerald

twofingers

Guru

Joined: 02/06/2014
Location: Germany
Posts: 1577

Posted: 01:18pm 04 Sep 2024

Copy link to clipboard

Print this post

TassyJim said ... For Windows, XCOPY or ROBOCOPY and a schedule

@ChatGPT:
briefly explain to me how "robocopy" works. What are the advantages over xcopy?

Quote Robocopy (Robust File Copy) is a command-line utility in Windows that is designed for efficiently copying files, directories, and even entire drives from one location to another. It was introduced as a more powerful and flexible alternative to xcopy.

Key Features of Robocopy
Resilience and Error Handling:

Robocopy is designed to handle interruptions and errors gracefully. If the network connection drops or a file is locked, Robocopy can resume the copying process once the issue is resolved, without starting over.
Selective Copying:

It can copy only files that have changed or been newly created since the last copy operation, using the /MIR (mirror) or /XO (exclude older) options.
It can also copy files based on attributes like archive, system, hidden, etc.
Logging:

Robocopy can generate detailed logs of the copy process, which can be extremely useful for troubleshooting and auditing.
Directory Tree Mirroring:

With the /MIR switch, Robocopy can mirror an entire directory tree, meaning it can replicate the source directory structure in the destination, including any deletions.
Multi-Threaded Copying:

Robocopy supports multi-threaded copying (using the /MT option), which can significantly speed up the process by copying multiple files simultaneously.
Retry Options:

If a file is in use or locked, Robocopy automatically retries the operation a specified number of times with a delay between attempts.

Comparison with xcopy
Error Recovery:

Robocopy is much more robust in recovering from errors compared to xcopy, which might fail completely if it encounters an issue like a network disruption.
Efficiency:

Robocopy is more efficient in handling large amounts of data, especially over network connections, due to its ability to resume transfers and retry failures.
Multi-Threading:

Robocopy supports multi-threaded operations, which xcopy does not, allowing for faster copying of large numbers of files.
Logging and Detailed Reporting:

While xcopy offers basic logging, Robocopy provides extensive logging and reporting options, making it easier to monitor and troubleshoot copy operations.
Directory Mirroring:

Robocopy can mirror entire directory structures, including deletions, which is more complex to achieve with xcopy.
Basic Robocopy Syntax

cmd

robocopy [source] [destination] [options]

Example Command:
To copy all files from C:\Source to D:\Destination, while preserving the directory structure and copying only new or updated files:

cmd

robocopy C:\Source D:\Destination /MIR /MT:8

/MIR mirrors the directory structure.
/MT:8 uses 8 threads to speed up the process.
Overall, Robocopy is the preferred tool when dealing with complex or large-scale copy operations, especially in professional and network environments.

Better formatted:
ChatGPT Backup - Robocopy.zip

Some thoughts on backup strategies
ChatGPT Backup - strategies.zip

Regards
Edited 2024-09-04 23:24 by twofingers

causality ≠ correlation ≠ coincidence

Goksteroo
Senior Member

Joined: 15/03/2021
Location: Australia
Posts: 114

Posted: 02:05pm 04 Sep 2024

Copy link to clipboard

Print this post

SyncBack Free works well on windows. Been using it for years without issues. Define what and how to backup then set a schedule for it to run - easy peasy. There is also a pro version but the free version works well for home/single user use.

twofingers

Guru

Joined: 02/06/2014
Location: Germany
Posts: 1577

Posted: 04:37pm 04 Sep 2024

Copy link to clipboard

Print this post

One more point is worth considering: the dependence on the manufacturer of the backup software. What if the manufacturer decides to change the format or its pricing policy or goes bankrupt?

EDIT:
Important: There is one reason not to use Robocopy: Robocopy has problems with open files.

(The Robocopy GUI ist still available on archive.org)

see also: https://en.wikipedia.org/wiki/RichCopy
(and its limits)

Edited 2024-09-05 04:16 by twofingers

causality ≠ correlation ≠ coincidence

Grogster

Admin Group

Joined: 31/12/2012
Location: New Zealand
Posts: 9593

Posted: 12:05am 05 Sep 2024

Copy link to clipboard

Print this post

Wow - lots more replies then I expected - thanks.

I will read through this thread in more detail and look at a few of the options.
Never thought of asking chatGPT....

Smoke makes things work. When the smoke gets out, it stops!

Gizmo

Admin Group

Joined: 05/06/2004
Location: Australia
Posts: 5117

Posted: 03:37am 05 Sep 2024

Copy link to clipboard

Print this post

I've been using batch files for this purpose for years.

This is a example...

xcopy /c /d /f /e /y "D:\BusinessData" "F:\Backup\BusinessData\"

The switches are...
/c continue on errors
/d copy files with a newer date
/f display whats been backed up
/e copy all sub folders
/y overwrite existing

I put this batch file into the scheduler.

I've seen many backup software options that are just not that good at what they do. My advise has always been to stay away from them, just use batch files.

Footnote added 2024-09-05 13:39 by Gizmo
Just adding, when you add the batch file to the scheduler, you need to give it admin rights, so it will run even if you are not logged in.

The best time to plant a tree was twenty years ago, the second best time is right now.
JAQ

Grogster

Admin Group

Joined: 31/12/2012
Location: New Zealand
Posts: 9593

Posted: 04:03am 05 Sep 2024

Copy link to clipboard

Print this post

twofingers said EDIT:
Important: There is one reason not to use Robocopy: Robocopy has problems with open files.

(The Robocopy GUI ist still available on archive.org)

Does XCOPY have that issue too?
iDrive sync copies open files, and I leave several files open all the time, so I need something that WON'T fall over, if it tries to copy a file that is already open in an application.

That's not a deal breaker though - I most certainly CAN just close all applications that have files open, BEFORE I run the ROBOCOPY or XCOPY batch file. There is no pain in having to do that, but it is preferable to be able to copy files that are still open.

Smoke makes things work. When the smoke gets out, it stops!

TassyJim

Guru

Joined: 07/08/2011
Location: Australia
Posts: 6270

Posted: 06:47am 05 Sep 2024

Copy link to clipboard

Print this post

I believe that XCOPY will copy open files but there are different 'open' states including buffers that haven't been flushed.

It should be simple for you to test on your files.

Jim

VK7JH
MMedit

Grogster

Admin Group

Joined: 31/12/2012
Location: New Zealand
Posts: 9593

Posted: 06:50am 05 Sep 2024

Copy link to clipboard

Print this post

TassyJim said It should be simple for you to test on your files.

Good point, well made.

I will open some files in my CAD software, then attempt to XCOPY them to another location and see what happens.

Smoke makes things work. When the smoke gets out, it stops!

twofingers

Guru

Joined: 02/06/2014
Location: Germany
Posts: 1577

Posted: 07:02am 05 Sep 2024

Copy link to clipboard

Print this post

Grogster said

twofingers said EDIT:
Important: There is one reason not to use Robocopy: Robocopy has problems with open files.

...)

...
Does XCOPY have that issue too?

Wikipedia

Quote Limitations
XCOPY fails with an "insufficient memory" error when the path plus filename is longer than 254 characters.[14] An option "/J" copies files without buffering;[15] moving very large files without the option (available only after Server 2008R2) can consume all available RAM on a system.

No open files
XCOPY will not copy open files. Any process may open files for exclusive read access by withholding the FILE_SHARE_READ [16]

XCOPY does not support the Windows Volume Shadow Copy service which effectively allows processes to have access to open files, so it is not useful for backing up live operating system volumes.

I would prefer robocopy to xcopy.
But a backup strategy is also important, not just a backup program. I suspect that because of the complexity, many people forego backup altogether.

Regards
Michael

causality ≠ correlation ≠ coincidence

Volhout
Guru

Joined: 05/03/2018
Location: Netherlands
Posts: 5066

Posted: 11:43am 05 Sep 2024

Copy link to clipboard

Print this post

Somtimes backups become worthless...

12 years ago I bought a new 21" MAC (apple). And the machine has a nice feature. You buy a (at that time 90 euro) USB harddisk, and plug it is. The MAC shows a popup. "do you want to use this drive as backup drive?". I answered Y, and never looked at it since.

The machine made incremental backups ever since. Apple claimed: when your machine dies, buy a new MAC, plug in the USB drive, and you are back in business (after hours of waiting).

Time has passed since 2012...

I cannot buy any Intel MAC anymore, only ARM (M1/M2/M3) and have a nice backup drive with 32bit and 64bit x86 code, and data.....

Volhout

PicomiteVGA PETSCII ROBOTS

atmega8

Guru

Joined: 19/11/2013
Location: Germany
Posts: 724

Posted: 05:13pm 05 Sep 2024

Copy link to clipboard

Print this post

Try Veeam Backup & Replication.
The free Community Edition!

PhenixRising
Guru

Joined: 07/11/2023
Location: United Kingdom
Posts: 1362

Posted: 06:58am 06 Sep 2024

Copy link to clipboard

Print this post

This post reminded me that I used to use a file-sync program but I couldn't remember the name. I had reason to fire-up the old XP laptop, yesterday and there was the folder; Allway Sync.

Did a search and now I see that it's Good Sync (also on the above ChatGPT list).

I installed the free version and I have to say that I'm impressed. I used it to backup/sync with my Dropbox and Google Drive.

The Dropbox backup had to be re-started a couple of times due to a complaint of "too many files open" but this was a first-time copy that was >1.5GB

Now, it feels solid.

Page 1 of 2

Print this page