104 lines
4.8 KiB
Markdown
104 lines
4.8 KiB
Markdown
# clonetool
|
|
|
|
Block-level sync for a large file or block device between two machines (or
|
|
two paths on the same machine), driven from a third, passive "manager"
|
|
machine. Single static Go binary, no runtime dependencies beyond the
|
|
system `ssh` client for remote endpoints.
|
|
|
|
## How it works
|
|
|
|
- You run `clonetool sync` on the **manager**. Source, destination, and
|
|
manager may be three entirely different machines — the manager never
|
|
reads or writes a single block itself. It only spawns and talks to two
|
|
**control agents** (`clonetool agent --role control`, one per side,
|
|
local subprocess or over `ssh`) that do the stat/prepare/transfer work.
|
|
- The two agents compare content by **SHA-256 hash per fixed-size block**.
|
|
A block is only re-read once: it's hashed from the buffer it was read
|
|
into, and that same buffer is what gets sent on if it differs — never
|
|
read twice.
|
|
- Bulk data goes **directly between source and destination**, not through
|
|
the manager. Each run tries:
|
|
1. **push** — the source agent connects straight to the destination
|
|
host over SSH and streams changed blocks to it.
|
|
2. **pull** (fallback) — if push isn't possible (no route/keys in that
|
|
direction), the destination agent connects to the source host
|
|
instead and pulls.
|
|
3. If neither direction works, the job fails with both reasons named.
|
|
Run the manager on the source or destination host, or set up SSH
|
|
connectivity in at least one direction.
|
|
- When source and destination are **both local** to the manager, no SSH
|
|
is used at all — the source agent spawns the write-side helper as a
|
|
plain local subprocess.
|
|
- Each named **job** keeps a hash table for the destination
|
|
(`~/.clonetool/jobs/<job>.state` by default) so a re-run doesn't need to
|
|
read the destination again — it just compares freshly-hashed source
|
|
blocks against last known state. **The destination must not be modified
|
|
by anything else between syncs of the same job** — that assumption is
|
|
what lets the tool skip reading it. Running the same job against a
|
|
different destination is refused (pass `--force` to deliberately rebind
|
|
it, which discards the hash history).
|
|
- Sizing rules:
|
|
- Destination is a **block device**: it can't be resized, so if the
|
|
source is larger the job fails; otherwise exactly `min(source, dest)`
|
|
bytes are synced and the remainder of the device is left untouched.
|
|
- Destination is a **regular file**: it's truncated (created if
|
|
missing) to exactly the source's size, growing or shrinking it.
|
|
Shrinking an existing non-empty file prompts for confirmation unless
|
|
`--yes` is passed.
|
|
|
|
## Build
|
|
|
|
```
|
|
CGO_ENABLED=0 go build -o clonetool .
|
|
```
|
|
|
|
Copy the resulting binary to the manager, source, and destination hosts
|
|
(same path, or point `--remote-bin` at wherever it lives on each host —
|
|
clonetool does not deploy itself).
|
|
|
|
## Usage
|
|
|
|
```
|
|
clonetool sync --job NAME --source LOC --dest LOC [options]
|
|
```
|
|
|
|
`LOC` is either a local path (`/dev/sdb`, `./image.bin`) or
|
|
`[user@]host:path` for a path reached over SSH.
|
|
|
|
```
|
|
# Same machine
|
|
clonetool sync --job disk1 --source /dev/sda --dest /dev/sdb
|
|
|
|
# Two remote machines, orchestrated from a third
|
|
clonetool sync --job backup --source db1:/dev/vdb --dest backup-host:/srv/db1.img
|
|
|
|
# Re-run any time; only changed blocks move
|
|
clonetool sync --job backup --source db1:/dev/vdb --dest backup-host:/srv/db1.img
|
|
```
|
|
|
|
Options:
|
|
|
|
| Flag | Default | Meaning |
|
|
|---|---|---|
|
|
| `--block-size` | `4M` | Block size (accepts `K`/`M`/`G` suffixes). Changing it on an existing job discards its hash history. |
|
|
| `--state-dir` | `~/.clonetool/jobs` | Where job hash-tables live. |
|
|
| `--yes` | off | Don't prompt before shrinking an existing destination file. |
|
|
| `--force` | off | Rebind a job to a different source/dest, discarding its hash history. |
|
|
| `--connect-timeout` | `8` | SSH connect timeout (seconds) used for the push/pull direction probe. |
|
|
| `--ssh` | `ssh` | ssh binary to use. |
|
|
| `--ssh-opt` | — | Extra `-o OPT` passed to ssh (repeatable). |
|
|
| `--remote-bin` | `clonetool` | Path to clonetool on remote hosts. |
|
|
| `--manager-host` | local hostname | Address a peer should use to reach this machine, needed only when source or dest is local to the manager *and* the other side is remote and ends up needing to dial back in (pull fallback). |
|
|
|
|
## Caveats
|
|
|
|
- Block-device size detection (`BLKGETSIZE64`) is Linux-only.
|
|
- If a destination path doesn't exist yet, it's created as a regular
|
|
file — clonetool won't create device nodes, so double-check device
|
|
paths for typos before running.
|
|
- SSH host keys are accepted on first connect (`StrictHostKeyChecking=accept-new`)
|
|
and rejected if they later change, same as normal SSH behavior.
|
|
- `agent` is an internal subcommand spawned automatically by `sync`; it's
|
|
not meant to be run by hand, though it will work standalone for
|
|
debugging.
|