clonetool

Block-level sync for a large file or block device between two machines (or two paths on the same machine), driven from a third, passive "manager" machine. Single static Go binary, no runtime dependencies beyond the system ssh client for remote endpoints.

How it works

• You run clonetool sync on the manager. Source, destination, and manager may be three entirely different machines — the manager never reads or writes a single block itself. It only spawns and talks to two control agents (clonetool agent --role control, one per side, local subprocess or over ssh) that do the stat/prepare/transfer work.
• The two agents compare content by SHA-256 hash per fixed-size block. A block is only re-read once: it's hashed from the buffer it was read into, and that same buffer is what gets sent on if it differs — never read twice.
• No state is kept between runs. Every sync re-reads and re-hashes the destination's current content and compares the source against that, so a re-run only moves the blocks that actually differ and nothing needs to be trusted from a previous run. (The destination side does the destination hashing; in push mode it streams that table to the source, in pull mode the destination agent hashes locally.)
• Bulk data goes directly between source and destination, not through the manager. Each run tries:
  1. push — the source agent connects straight to the destination host over SSH and streams changed blocks to it.
  2. pull (fallback) — if push isn't possible (no route/keys in that direction), the destination agent connects to the source host instead and pulls.
  3. If neither direction works, the job fails with both reasons named. Run the manager on the source or destination host, or set up SSH connectivity in at least one direction.
  • When source and destination are both local to the manager, no SSH is used at all — the source agent spawns the write-side helper as a plain local subprocess.
• Self-deploy: if a remote endpoint has no runnable clonetool on PATH (or wherever --remote-bin points), the manager streams this binary to ~/.clonetool/bin/clonetool on that host over the existing SSH connection and uses it — no install, no root. Disable with --deploy=false. If the copied binary won't execute there (wrong CPU architecture) the error says so; build one for the remote's arch (CGO_ENABLED=0 GOOS=linux GOARCH=arm64 go build) and put it on PATH or pass --remote-bin.
• sudo: if reading or writing an endpoint that is a block device fails with a permission error, --sudo=auto (the default) transparently restarts that side's agent — and the helper it spawns on the peer — under sudo. Locally that may prompt for a password on your terminal; on a remote host it uses sudo -n, so passwordless sudo (NOPASSWD) must be configured there (a password prompt can't work — the transfer protocol owns the SSH stdout stream). --sudo=always elevates from the start; --sudo=never never does.
• Sizing rules:
  • Destination is a block device: it can't be resized, so if the source is larger the job fails; otherwise exactly min(source, dest) bytes are synced and the remainder of the device is left untouched.
  • Destination is a regular file: it's truncated (created if missing) to exactly the source's size, growing or shrinking it. Shrinking an existing non-empty file prompts for confirmation unless --yes is passed.

Build

CGO_ENABLED=0 go build -o clonetool .

Copy the resulting binary to the manager host. Source and destination hosts get it automatically (see self-deploy above), or place it yourself and point --remote-bin at it. The binary is architecture-specific — cross-compile (GOOS/GOARCH) if your hosts differ.

Usage

clonetool sync --source LOC --dest LOC [options]

LOC is either a local path (/dev/sdb, ./image.bin) or [user@]host:path for a path reached over SSH.

# Same machine
clonetool sync --source /dev/sda --dest /dev/sdb

# Two remote machines, orchestrated from a third
clonetool sync --source db1:/dev/vdb --dest backup-host:/srv/db1.img

# Re-run any time; only changed blocks move
clonetool sync --source db1:/dev/vdb --dest backup-host:/srv/db1.img

Options:

Flag	Default	Meaning
--block-size	4M	Block size (accepts K/M/G suffixes).
--job	—	Optional label shown in progress/log output.
--yes	off	Don't prompt before shrinking an existing destination file.
--sudo	auto	Block-device privilege escalation: auto (on a permission error), always, or never. Remote elevation needs passwordless sudo.
--deploy	true	Copy this binary to remote hosts that lack a runnable clonetool. --deploy=false to disable.
--connect-timeout	8	SSH connect timeout (seconds) used for the push/pull direction probe.
--ssh	ssh	ssh binary to use.
--ssh-opt	—	Extra -o OPT passed to ssh (repeatable).
--remote-bin	clonetool	Path to clonetool on remote hosts.
--manager-host	local hostname	Address a peer should use to reach this machine, needed only when source or dest is local to the manager and the other side is remote and ends up needing to dial back in (pull fallback).

clonetool version prints the binary's GOOS/GOARCH (used internally for the self-deploy check).

Caveats

• Block-device size detection (BLKGETSIZE64) is Linux-only.
• If a destination path doesn't exist yet, it's created as a regular file — clonetool won't create device nodes, so double-check device paths for typos before running.
• SSH host keys are accepted on first connect (StrictHostKeyChecking=accept-new) and rejected if they later change, same as normal SSH behavior.
• Because every run re-hashes the whole destination, a re-sync costs a full read of both sides even when little changed — the win is in the bytes transferred, not the bytes read. That read shows on the status line as a scanning destination phase before syncing begins.
• agent is an internal subcommand spawned automatically by sync; it's not meant to be run by hand, though it will work standalone for debugging.