Difference between revisions of "Realtime event backup to cloud storage"

From ZoneMinder Wiki
Jump to navigationJump to search
Line 61: Line 61:
Logging is pretty self-explanatory.  Be sure to set up log rotation after you are done setting lsyncd up, if required.
Logging is pretty self-explanatory.  Be sure to set up log rotation after you are done setting lsyncd up, if required.


maxProcesses = 1 is the default.  If you are syncing multiple sources or targets you may want to increase this.
''maxProcesses = 1'' is the default.  If you are syncing multiple sources or targets you may want to increase this.


insist = true allows the service to start even if the target is not ready.  I got errors without this which went away when I added it.  YMMV.
''insist = true'' allows the service to start even if the target is not ready.  I got errors without this which went away when I added it.  YMMV.


source = The directory where ZM events are stored.
''source = ''The directory where ZM events are stored.


target = The location to copy events to.  Must be accessible via rsync.
''target = ''The location to copy events to.  Must be accessible via rsync.


targetdir = (not shown) subdirectory to use on the target.  I'm just using the root directory in a dedicated bucket.  You may not want this and should specify something here.
''targetdir = ''(not shown) subdirectory to use on the target.  I'm just using the root directory in a dedicated bucket.  You may not want this and should specify something here.


delay - this is how often to run rsync if it is not triggered by any filesystem changes.  Default is 15 seconds.  For many users this will be close enough to real-time for the purpose.  This can be combined with (or substituted) for another value in the settings field;
''delay - ''this is how often to run rsync if it is not triggered by any filesystem changes.  Default is 15 seconds.  For many users this will be close enough to real-time for the purpose.  This can be combined with (or substituted) for another value in the settings field;


maxDelay - # (not shown) will queue this # of file changes before calling rsync.  Between delay and/or maxDelay it is possible to tune the timing of your file copies to your exact preference.  inotify waits for write_close  to add files to the transfer list so there are no issues with rsync trying to upload uncompleted files.  I typically only record short 10 second clips.  If you use your cameras to continuously record be aware that files in the process of being written will not be copied until they are closed.
''maxDelay - # '' (not shown) will queue this # of file changes before calling rsync.  Between delay and/or maxDelay it is possible to tune the timing of your file copies to your exact preference.  inotify waits for write_close  to add files to the transfer list so there are no issues with rsync trying to upload uncompleted files.  I typically only record short 10 second clips.  If you use your cameras to continuously record be aware that files in the process of being written will not be copied until they are closed.


Two other very important variables in this file are;
Two other very important variables in this file are;


delete = false this is pretty self explanatory, file deletes will not be synced from source to target.  In the case of my S3 bucket I have lifecycle rules that handle this.  If you prefer for your remote storage location to synchronize the deletions made by Zoneminder you can set this to true.   
''delete = false'' this is pretty self explanatory, file deletes will not be synced from source to target.  In the case of my S3 bucket I have lifecycle rules that handle this.  If you prefer for your remote storage location to synchronize the deletions made by Zoneminder you can set this to true.   


init = false causes the lsyncd to not do a complete initial synchronization between source and target on service startup.  On S3 this is expensive in terms of API calls and may take a while.  I know I'm going to miss some events during system maintenance/OS updates etc anyway so I don't consider this worth the cost, but you can set to true if you want it.
''init = false'' causes the lsyncd to not do a complete initial synchronization between source and target on service startup.  On S3 this is expensive in terms of API calls and may take a while.  I know I'm going to miss some events during system maintenance/OS updates etc anyway so I don't consider this worth the cost, but you can set to true if you want it.


Having delete and init set to false also should make this config fairly safe to run against an existing target if you just blindly copy it to your machine without heeding my previous warning. You can also add options to be added directly to the rsync calls. You can sync multiple directories to multiple targets with multiple options.  There are exclude options which you may find useful.  It is far too much to cover here so read the docs, but at this point you should get the idea.
Having delete and init set to false also should make this config fairly safe to run against an existing target if you just blindly copy it to your machine without heeding my previous warning. You can also add options to be added directly to the rsync calls. You can sync multiple directories to multiple targets with multiple options.  There are exclude options which you may find useful.  It is far too much to cover here so read the docs, but at this point you should get the idea.

Revision as of 19:30, 11 December 2025

After recently migrating from another popular NVR software on Windows to Zoneminder on Linux I started looking for a way to do near-real-time backups of my events to offsite storage, simply as a way to have a copy should the local ZM server be compromised by break-in, fire, etc. Software such as Dropbox, Google Drive, etc. make it relatively easy to do this on Windows but the setup on Linux takes bit more effort.

I wanted to keep it as simple as possible and don't need my backup solution to interface with the ZM database at all. After experimenting with several other options, I have settled on an application called lsyncd. It runs as a service and leverages inotify and rsync to watch a directory or directories and mirror only new changes to another storage location. Lsyncd should be available as a package for most Linux distributions. I used the instructions here for my Rocky Linux system;

https://docs.rockylinux.org/10/guides/backup/mirroring_lsyncd/

At this point it is worthwhile to at least skim through the docs.

https://lsyncd.github.io/lsyncd/manual/config/file/

https://linux.die.net/man/1/rsync

Both lsyncd and rsync are highly configurable and incredibly powerful. For many people the use case will be very simple like my own which I will describe here - watch a single event directory and copy new files to a remote location, but much more complex configurations are certainly possible. In my case I'll be writing to an Amazon S3 bucket which I have already setup using S3fs-FUSE according to the documentation here;

https://zoneminder.readthedocs.io/en/latest/userguide/options/options_storage.html

Note you do not need to add the S3FS as a storage location in ZM to use lsyncd to copy files there, that is only required for ZM to write events directly to S3. Configuring rsync is beyond the scope of this document but this solution will work with any target that rsync is capable of writing to. Be sure to have your remote location configured and accessible via rsync before proceeding.

Once lsyncd is installed you will need to create a configuration file in /etc/lsyncd.conf. I've included mine as an example below. Note this script uses LUA syntax, not bash. Be VERY CAREFUL if you are testing remote write/sync/copy operations to a destination that already has important data on it. I suggest initial testing with an empty bucket or target filesystem. You have been warned.

settings {

   logfile = "/var/log/lsyncd.log",

   statusFile = "/var/log/lsyncd-status.log",

   statusInterval = 10,

   maxProcesses = 1,

   insist = true,

   }



sync {

   default.rsync,

   source = "/home/Cameras",

   target = "/media/aws",

   delete = false, 

   delay = 10,    

   init = false,

   rsync = {

     archive = true,

     compress = false

   }

}

Logging is pretty self-explanatory. Be sure to set up log rotation after you are done setting lsyncd up, if required.

maxProcesses = 1 is the default. If you are syncing multiple sources or targets you may want to increase this.

insist = true allows the service to start even if the target is not ready. I got errors without this which went away when I added it. YMMV.

source = The directory where ZM events are stored.

target = The location to copy events to. Must be accessible via rsync.

targetdir = (not shown) subdirectory to use on the target. I'm just using the root directory in a dedicated bucket. You may not want this and should specify something here.

delay - this is how often to run rsync if it is not triggered by any filesystem changes. Default is 15 seconds. For many users this will be close enough to real-time for the purpose. This can be combined with (or substituted) for another value in the settings field;

maxDelay - # (not shown) will queue this # of file changes before calling rsync. Between delay and/or maxDelay it is possible to tune the timing of your file copies to your exact preference. inotify waits for write_close to add files to the transfer list so there are no issues with rsync trying to upload uncompleted files. I typically only record short 10 second clips. If you use your cameras to continuously record be aware that files in the process of being written will not be copied until they are closed.

Two other very important variables in this file are;

delete = false this is pretty self explanatory, file deletes will not be synced from source to target. In the case of my S3 bucket I have lifecycle rules that handle this. If you prefer for your remote storage location to synchronize the deletions made by Zoneminder you can set this to true.

init = false causes the lsyncd to not do a complete initial synchronization between source and target on service startup. On S3 this is expensive in terms of API calls and may take a while. I know I'm going to miss some events during system maintenance/OS updates etc anyway so I don't consider this worth the cost, but you can set to true if you want it.

Having delete and init set to false also should make this config fairly safe to run against an existing target if you just blindly copy it to your machine without heeding my previous warning. You can also add options to be added directly to the rsync calls. You can sync multiple directories to multiple targets with multiple options. There are exclude options which you may find useful. It is far too much to cover here so read the docs, but at this point you should get the idea.

Once the service is configured you can control it with systemctl like any other systemd service. You can check the logs for startup or running errors with journalctl. If syncs are working correctly you can watch them as uploads are logged very shortly after the events are saved to local disk;

[root@nvr system]# tail -50 /var/log/lsyncd.log
/1/2025-12-11/8707/8707-video.mp4
/1/2025-12-11/8707/snapshot.jpg
/1/2025-12-11/8707/alarm.jpg
/6/2025-12-11/8708/
/6/2025-12-11/
/6/
/6/2025-12-11/8708/8708-video.mp4
/6/2025-12-11/8708/snapshot.jpg
/6/2025-12-11/8708/alarm.jpg
/8/2025-12-11/8706/8706-video.mp4
/8/2025-12-11/8706/
/8/2025-12-11/
/8/
Thu Dec 11 14:30:25 2025 Normal: Finished a list after exitcode: 0
Thu Dec 11 14:31:50 2025 Normal: Calling rsync with filter-list of new/modified files/dirs
/2/2025-12-11/8709/
/2/2025-12-11/
/2/
/
/2/2025-12-11/8709/8709-video.mp4
/2/2025-12-11/8709/snapshot.jpg
/2/2025-12-11/8709/alarm.jpg
Thu Dec 11 14:32:00 2025 Normal: Finished a list after exitcode: 0
Thu Dec 11 14:32:04 2025 Normal: Calling rsync with filter-list of new/modified files/dirs
/2/2025-12-11/8710/
/2/2025-12-11/
/2/
/
/2/2025-12-11/8710/8710-video.mp4
/2/2025-12-11/8710/snapshot.jpg
/2/2025-12-11/8710/alarm.jpg
Thu Dec 11 14:32:11 2025 Normal: Finished a list after exitcode: 0
Thu Dec 11 14:32:45 2025 Normal: Calling rsync with filter-list of new/modified files/dirs
/2/2025-12-11/8711/
/2/2025-12-11/
/2/
/
/2/2025-12-11/8711/8711-video.mp4
/2/2025-12-11/8711/snapshot.jpg
/2/2025-12-11/8711/alarm.jpg
Thu Dec 11 14:32:51 2025 Normal: Finished a list after exitcode: 0
Thu Dec 11 14:37:36 2025 Normal: Calling rsync with filter-list of new/modified files/dirs
/3/2025-12-11/8712/
/3/2025-12-11/
/3/
/
/3/2025-12-11/8712/8712-video.mp4
/3/2025-12-11/8712/snapshot.jpg
/3/2025-12-11/8712/alarm.jpg
Thu Dec 11 14:37:41 2025 Normal: Finished a list after exitcode: 0
[root@nvr system]# 

I'm happy to have offsite copies of all my events now. While they are not tracked in the Zoneminder database, they are still saved in folders by monitor # and date for easy retrieval should the ZM database become unavailable. Props to the ZM devs for their great software, and I hope this writeup proves useful to some other users in the future.