lfs-mirror-extend(1)

NAME

lfs-mirror-extend - add mirror(s) to an existing file

SYNOPSIS

DESCRIPTION

This command adds mirror(s) to existing file(s) specified by FILES_FROM.

The file(s) to be extended can already be a mirrored file, or just a regular non-mirrored file. If it's a non-mirrored file, then the command will convert it to a mirrored file.

When --ec is specified, the command operates in one of three erasure-coding modes:

• With --mirror-id MIRROR_ID --ec DATA+PARITY, a parity mirror is attached to the existing data mirror identified by MIRROR_ID. The parity mirror inherits the extents and stripe configuration of the target data mirror. The target must not itself be a parity mirror, and it must not already be EC-protected: attaching a second parity mirror of a different geometry to one data mirror, to change a file's EC protection without rewriting its data, is not implemented yet.
• With -N[MIRROR_COUNT] --ec DATA+PARITY, MIRROR_COUNT new data+parity mirror pairs are added (one parity mirror per data mirror). MIRROR_COUNT defaults to 1.
• With --ec DATA+PARITY alone (no -N, no --mirror-id), a parity mirror is attached to the file's lone data mirror, at a cost of PARITY/DATA of its size in additional storage. The target mirror is resolved per file; files with two or more mirrors produce a per-file error directing the user to disambiguate with --mirror-id or -N.

In all three EC modes the new mirrors are brought in sync before the command returns, matching the behavior of a regular (non-EC) extend: new data mirrors are populated with a copy of the file data, and new parity mirrors are computed from their data mirrors as lfs-mirror-resync(1) does.

A data mirror that is stale or nosync cannot be given a parity mirror. Parity is computed from the content of the named mirror, and resynchronizing that mirror afterwards does not mark its parity stale in turn, so the parity would go on protecting superseded data; a nosync mirror is skipped by resync, which would leave the parity uncomputed. Run lfs-mirror-resync(1) on the file first.

FILES_FROM may be:

FILENAME [FILENAME ...]

File names are listed on command line. Multiple file names are separated by space char.

-0, --null

Read file names from stdin by default. Each file name is followed by a NUL char. Usually is used after a pipeline from lfs find --print0 command.

--files-from=LIST_FILE

Read file names from file LIST_FILE. One line for each file name. If LIST_FILE is '-', read from stdin. If --null is specified at the same time, file names are separated by a NUL char.

SETSTRIPE_OPTIONS

specify the specific layout for the mirror. These are the same options that are used by lfs-setstripe(1). If SETSTRIPE_OPTIONS are not specified, then the stripe options inherited from the previous component will be used. If VICTIM_FILE is specified and exists, then the command will merge the layout from that file as a mirror added to the mirrored file. After the command is finished, the victim file will be removed.

OPTIONS

--chunk-size=SIZE_MB

The buffer chunk size in MB used to transfer data. It is 1 (MB) by default. But if --aio-tasks=0, the default value is 64 (MB).

--ec, --ec-stripe-count DATA+PARITY

Enable Erasure Coding (EC) for the mirror(s) added by this command, with DATA data stripes and PARITY parity stripes per RAID Set. The format is DATA+PARITY or DATA:PARITY. The semantics of the DATA+PARITY specification are identical to those described in lfs-setstripe(1).

--ec-expert DATA+PARITY

Equivalent to --ec but allows EC stripe counts that exceed the supported limits, as described in lfs-setstripe(1).

-F, --fid

FID(s) are provided from command line/stdin/LIST_FILE instead of file path(s). Option --lustre-dir=LUSTRE_MOUNT_POINT must be specified at the same time.

-f, --file VICTIM_FILE

First, the contents of VICTIM_FILE will be verified to be the same as FILENAME. If they are, the layout of VICTIM_FILE will be merged as a mirror added to the mirrored file. Otherwise the layout of VICTIM_FILE will be unchanged. However, --no-verify can be specified to override this check. Once the command is complete VICTIM_FILE will be removed. This option cannot be specified with SETSTRIPE_OPTIONS.

-N, --mirror-count[[=]MIRROR_COUNT]

Specifies that the next MIRROR_COUNT mirrors will have the following layout. This option can be repeated multiple times to separate mirrors that have different layouts. The MIRROR_COUNT argument is optional and will default to 1 if not specified. This lets a single mirror be specified easily with -N. The --mirror-count|-N option itself is also optional, and if not specified only 1 mirror will be created. The MIRROR_COUNT can be specified with a preceding '=' to indicate the number of total mirrors desired for the layout, such as --mirror-count==TOTAL_MIRROR_COUNT or -N=TOTAL_MIRROR_COUNT. This will create mirrors until the total number of mirrors in the layout is at least TOTAL_MIRROR_COUNT. This also means that if there are already at least TOTAL_MIRROR_COUNT mirrors in the layout then no new mirrors will be created.

--mirror-id MIRROR_ID

Identify the existing data mirror that a new parity mirror should be attached to. Requires --ec, and may be passed at most once per command. Cannot be combined with -N or -f. MIRROR_ID must be 1 or greater; --mirror-id 0 is rejected. The target mirror must be a data mirror that is not already EC-protected; otherwise the command fails.

For a non-mirrored (single-mirror) file omit --mirror-id entirely and rely on --ec auto mode to attach parity to the lone mirror.

--no-verify

This option indicates not to verify the mirror(s) from victim file(s) in case the victim file(s) contains the same data as the original mirrored file. This option can save siginificant time on file comparison if the file size is large, but it should only be used when the file contents are known to be the same.

-D, --non-direct

Use buffered, not O_DIRECT, read and write operations when migrating a file. This adds CPU and memory overhead for the data copy between kernel and userspace buffers, but avoids synchronous read and write operations that may slow down small file operations.
This option cannot be used on encrypted files when the encryption key is not available. It will result in -ENOKEY being returned.

--stats

This option enables progress updates every 5 seconds, in YAML format.

--stats-interval=STATS_INTERVAL

This option enables progress updates every STATS_INTERVAL seconds, in YAML format.

-W, --bandwidth BANDWIDTH_MB[MG]

Limit the read and write bandwidth a mirror extend consumes to no more than BANDWIDTH_MB MiB/s. An optional suffix can be used to specify units in Mebibytes or Gibibytes per second.

EXAMPLES

Add 2 mirrors to /mnt/lustre/file1. If file1 is a non-mirrored file, then the command will convert it to a mirrored file first and then add mirrors. Each mirror has the same default striping pattern with STRIPE_COUNT and STRIPE_SIZE inherited from filesystem-wide default values, and OST POOL_NAME inherited from parent directory:

# lfs mirror extend -N2 /mnt/lustre/file1

Add 3 PFL mirrors to /mnt/lustre/file1. Each mirror has the same specified PFL layout:

# lfs mirror extend -N3 -E 1M -c 1 -E 32M -c 4 -S 16M -E eof -c -1 /mnt/lustre/file1

Add 3 plain layout mirrors to /mnt/lustre/file1. The first mirror has a single stripe and 4MB stripe size. The second mirror has two stripes and locates on OSTs with indices 2 and 3 allocated from the flash OST pool. It also has 4MB stripe size inherited from the first mirror. The third mirror has two stripes and 4MB stripe size inherited from the previous mirror, and also has inherited OST POOL_NAME from parent directory:

# lfs mirror extend -N -c 1 -S 4M -N -c 2 -o 2,3 -p flash -N -p none /mnt/lustre/file1

Add 5 PFL mirrors to /mnt/lustre/file1. The first and second mirrors have the same PFL layout. All of the components are allocated from the flash OST pool. The last three mirrors have the same PFL layout. All of these components have a stripe size of 16MB and use OSTs in the archive pool:

# lfs mirror extend -N2 -E 4M -c 2 --pool flash -E eof -c 4 -N3 -E 16M -c 4 -S 16M --pool archive -E eof -c -1 /mnt/lustre/file1

Merge the layouts from /mnt/lustre/file2 and /mnt/lustre/file3, which contain the same data as /mnt/lustre/file1, use the layouts as mirrors and add them to /mnt/lustre/file1 without verification:

# lfs mirror extend --no-verify -N -f /mnt/lustre/file2 -N -f /mnt/lustre/file3 /mnt/lustre/file1

Attach an 8+2 parity mirror to mirror ID 1 of /mnt/lustre/file1. The parity is computed from the data mirror before the command returns:

# lfs mirror extend --mirror-id 1 --ec 8+2 /mnt/lustre/file1

Attach an 8+2 parity mirror to a non-mirrored file in place (auto mode). The single existing data mirror is auto-resolved as the target; the storage cost is (8 + 2) / 8 = 1.25x the file size, vs 2x for a plain mirror:

# lfs mirror extend --ec 8+2 /mnt/lustre/file1

Add 2 new data+parity mirror pairs to /mnt/lustre/file1, each data mirror having a single PFL component striped across 4 OSTs and protected by a 4+2 EC parity mirror:

# lfs mirror extend -N2 -E -1 -c 4 --ec 4+2 /mnt/lustre/file1

AVAILABILITY

The lfs mirror extend command is part of the lustre(7) filesystem package since release 2.11.0

SEE ALSO

lfs(1), lfs-mirror-create(1), lfs-mirror-delete(1), lfs-mirror-resync(1), lfs-mirror-split(1), lfs-mirror-verify(1), lfs-setstripe(1)