LFS-MIGRATE(1) Lustre User Utilities LFS-MIGRATE(1) # [NAME](#NAME) lfs-migrate - migrate files or directories between MDTs or OSTs. # [SYNOPSIS](#SYNOPSIS) `lfs migrate` \[**-h**] \[**-v**] \[**-A** \[**-K** *CAP*] \[**-M** *MIN\_FREE*] \[**-X** *MAX\_FREE ]]* \[*SETSTRIPE\_OPTIONS* ... ] *FILES\_FROM* `lfs migrate` **-m** *START\_MDT\_INDEX* \[**-cdHv**] *DIRECTORY* # [DESCRIPTION](#DESCRIPTION) Migrate OST objects between OSTs for the file(s) specified by *FILES\_FROM*, or with **-m** migrate *DIRECTORY* and all inodes/subdirectories therein to other MDTs (recursively by default). # [OPTIONS](#OPTIONS) ## [OST MIGRATE OPTIONS](#OST_MIGRATE_OPTIONS) The **lfs migrate** command can be used for moving files from one (or more) OSTs to other OSTs (e.g. for space balancing between OSTs, or to evacuate an OST for hardware reasons), to change the stripe count or other layout parameters of a file (e.g. to increase the bandwidth of a file by striping it over multiple OSTs), or to move the file between different classes of storage (e.g. SSD vs. HDD OSTs, or local vs. remote OSTs in different pools). *FILES\_FROM* may be: [*FILENAME* \[...\]](#FILENAME) File names are listed on command line. Multiple file names are separated by space char. **-0**, **--null** Read file names from stdin by default. Each file name is followed by a NUL char. Usually is used after a pipeline from **lfs find --print0** command. [**--files-from**=***LIST\_FILE***](#files-from) Read file names from file *LIST\_FILE*. One line for each file name. If *LIST\_FILE* is **-**, read filenames from standard input. If **--null** is also specified, file names are separated by a NUL char in the input. In OST object migration mode, the command supports the same *SETSTRIPE\_OPTIONS* listed in [**lfs-setstripe**(1)](/man/enterprise-packages/lustre/lfs-setstripe.1) to specify the layout of the target file. The migrate command differs from **lfs setstripe** in that **lfs migrate** will copy the data from the existing file(s) using the new layout parameters to the new OST(s). In contrast, **lfs setstripe** is used for creating new (empty) files with the specified layout. For OST object migration, there are additional options available: If setstripe is restricted to specified users, this also applies to **lfs migrate.** See [**lfs-setstripe**(1)](/man/enterprise-packages/lustre/lfs-setstripe.1) for details. [**-A**](#A) Automatically determine the stripe count for the file, using the algorithm count = sqrt(filesize\_in\_GB) + 1. This option may not be specified at the same time as the **-c** or **-R** options. [**-K CAP**](#K) When **-A** is set, limit the migrated file to use on each OST at most 1/*CAP* of the available space of the smallest OST. If this option is not set, a default value of 100 is used, limiting the object size to 1% of available space. [**-b**, **--block**](#b) Block access to the file by other applications during data migration (default). This prevents other processes from accessing the file during migration, which prevents data data writes to the old file objects from being lost. This should be used if an OST needs to be completely emptied prior to its removal, to ensure all requested files are migrated off the OST. [**-D**, **--non-direct**](#D) Do **not** use **O\_DIRECT** read and write operations when migrating a file. The **O\_DIRECT** option avoids data copy from kernel buffers into userspace, which can impose CPU and memory overhead on the copy operation, but makes read and write operations synchronous. Using the **--non-direct** option uses buffered read/write operations, which may improve migration speed at the cost of more CPU and memory overhead. This option cannot be used on encrypted files when the encryption key is not available. It will result in **-ENOKEY**. [**-F**, **--fid**](#F) FID(s) are provided from command line/stdin/ *LIST\_FILE* instead of file path(s). Option **--lustre-dir**=***LUSTRE\_MOUNT\_POINT*** *may be specified at the same time.* [**-h**, **--help**](#h) Print usage message. [**--lustre-dir**=***LUSTRE\_MOUNT\_POINT***](#lustre-dir) The mount point of Lustre file system on which FIDs are. If there is only one mounted Lustre file system, this option is not necessary. This option must be specified together with **--fid**. [**-M**, **--min-freeMIN\_FREE**](#M) When **-A** is set, only consider OSTs with free space greater than the *MIN\_FREE* value to be available for migration. The value is specified in KB. If this option is not set, a default of 256MB is used. [**-X**, **--max-freeMAX\_FREE**](#X) When **-A** is set, *MAX\_FREE* is the maximum amount of free space that can be considered available for the migration of the file on each OST. The value is specified in KB. This option is useful for testing, by simulating OSTs that are nearly full. [**-n**, **--non-block**](#n) Abort migration if concurrent file access is detected. This can be used with OST space balancing migration to avoid interfering with file access by applications if there is not a requirement to migrate any particular file to the new layout. [**--restripe**](#restripe) Restripe the file using the default directory striping layout. This option copies the default layout from the parent directory and applies it to the file being migrated. This option may not be specified at the same time as the **-c** or **-S** or **-p** or **-A** or **-E** options. [**--stats**](#stats) This option enables progress updates every 5 seconds, in YAML format. [**--stats-interval**=*STATS\_INTERVAL*](#stats-interval) This option enables progress updates every *STATS\_INTERVAL* seconds, in YAML format. [**-v**, **--verbose**](#v) Print each filename as it is migrated. [**-W**, **--bandwidth-limit**, **--bandwidth** *BANDWIDTH\_MB*\[**MG**\]](#W) Limit the read and write bandwidth a migrate job consumes to no more than *BANDWIDTH\_MB* MiB/s. An optional suffix can be used to specify units in **M**ebibytes or **G**ibibytes per second. ## [MDT MIGRATE OPTIONS](#MDT_MIGRATE_OPTIONS) [**-m**, **--mdt-index=*MDT\_INDEX*\[,*MDT\_INDEX*,...\]**](#m) The specified *DIRECTORY* **and (by default) all subdirectories and inodes** will be migrated to the MDT with the specified *MDT\_INDEX*. This is useful if new MDTs have been added to a filesystem and existing user or project directories should be migrated off old MDTs to balance the space usage and future metadata workload. If *MDT\_INDEX* is -1, the MDT index will be selected to balance by free space and inodes among available MDTs. If multiple *MDT\_INDEX* values are specified in a comma-separated list, then all subdirectories will be **striped** across all of the specified MDT indices as if an equivalent **-cN** option were given. This is typically **NOT** what you want to do. [**-c**, **--mdt-count=COUNT**](#c) All directories and subdirectories in the tree will be striped across *COUNT* MDTs, always using *MDT\_INDEX* as the primary MDT for the directory. If *MDT\_INDEX is* **-1** then *COUNT* directory stripes will be chosen from MDTs proportional to the amount of free inodes and space on each MDT. If multiple *MDT\_INDEX* values are specified in a comma-separated list, then the number of specified *MDT\_INDEX* values must match *COUNT*. [**-C**, **--mdt-overcount=COUNT**](#C) The number of directory stripes to create, creating > 1 stripe per MDT if *COUNT* exceeds the number of MDTs in the file system. **0** means to use the filesystem-wide default stripe count (default 1), and **range -1 to -5** means to stripe over all available MDTs in multiple of MDT count. For example, **-1** means one stripe per MDT, **-2** means two stripes per MDT, and so on. The maximum is 5 stripes per MDT. This allows a single directory to more fully use the capability of each MDT. [**-d**, **--directory**](#d) Only migrate the specified *DIRECTORY* and the non-directory inodes that are directly located within it. Similar to '**ls -d**' and '**lfs getstripe -d**'. [**--clear-fixed**](#clear-fixed) Do not set LMV\_HASH\_FLAG\_FIXED upon migrating the directory. It allows a striped dir with only one stripe to be converted to a plain directory. [**-H**, **--mdt-hash=HASH\_TYPE**](#H) Use *HASH\_TYPE* for the new directory layout. [**all\_char (type 1)**](#all_char) Sum of ASCII characters modulo number of MDTs. This provides weak hashing of the filename, and is suitable for only testing or when the input is known to have perfectly uniform distribution (e.g. sequential numbers). [**fnv\_1a\_64 (type 2)**](#fnv_1a_64) Fowler-Noll-Vo (FNV-1a) hash algorithm. This provides reasonably uniform, but not cryptographically strong, hashing of the filename. (default) [**crush (type 3)**](#crush) CRUSH hash algorithm. This is a consistent hash algorithm, so minimum sub files need to relocate during directory restripe. [**crush2 (type 4)**](#crush2) CRUSH hash algorithm (as above), with optimizations that allow temporary names created by the [*mktemp*(3)](/man/enterprise-packages/lustre/mktemp.3) family of functions to have both temporary and final names hash to the same MDT index. This allows renaming the temporary name to the final name without creating a remote directory entry or doing a cross-MDT rename (e.g. **test\_foo.XXXXXX** or **.test\_foo.XXXXXXXXXX** is on the same MDT as **test\_foo**). ## [NOTES](#NOTES) Only the root user can migrate directories. Files that have been archived by HSM or are currently opened will fail to migrate, user can run the same migrate command again to finish migration when files are ready. Both inode and directory entry will be migrated. During migration directory and sub files can be accessed like normal ones, but the migration itself cannot be interrupted. It is not currently possible to migrate files with an **mdt** component (Data-on-MDT, DoM). If it is necessary to migrate such files off a particular MDT, they must first be migrated to have a non-DoM file layout and then the inodes migrated separately. See **EXAMPLES** for details on how to migrate DoM files between MDTs. ## [WARNING](#WARNING) Each migrated file or directory will have a new FID, and hence a new inode number. As a consequence, files archived by Lustre HSM that depend on the FID as the identifier in the HSM archive cannot currently be migrated. Having a new inode number may also cause backup tools to consider the migrated file(s) to be a new, and cause them to be backed up again. # [EXAMPLES](#EXAMPLES) This migrates the data in **file1** into a new layout with 2 stripes: ``` # lfs migrate -c 2 /mnt/lustre/file1 ``` ``` ``` This migrates **file2** using automatic striping (stripe count calculated based on file size): ``` # lfs migrate -A /mnt/lustre/file2 ``` ``` ``` This migrates **file3** using automatic striping with custom capacity limits (5% of OST space): ``` # lfs migrate -A -K 20 /mnt/lustre/file3 ``` ``` ``` This migrates the data in **file2** into a three component composite layout (number of stripes depends on file size): ``` # lfs migrate -E 256M -c 1 -E 16G -c 4 -E eof -c 40 /mnt/lustre/file2 ``` ``` ``` Recursively move the subdirectories and inodes contained in directory **remotedir** from its current MDT to a 2-striped directory on MDT0000 and MDT0002. The **testremote** directory and all of its subdirectories will be striped across both MDTs: ``` # lfs migrate -m 0,2 testremote ``` ``` ``` Move **./testremote** and the first level of sub files from their current MDT to the MDT with index 0 and 2. Different from above case, the layout of subdirectories under **./testremote** won't be changed: ``` # lfs migrate -m 0,2 -d ./testremote ``` ``` ``` Set a default PFL layout (without any DoM component) on the directory **topdir**: ``` # lfs setstripe -E 256M -c 1 -E 16G -c 4 -E eof -c 40 topdir ``` ``` ``` then find and migrate all regular files that have an **mdt** component to copy the default layout from the specified **topdir**: ``` # lfs find dir -type f -L mdt -0 | lfs migrate -0 --copy topdir ``` ``` ``` and finally migrate the directory **topdir** and all files and subdirectories in that tree to MDT0002. This allows migrating files with DoM components off an MDT: ``` # lfs migrate -m 2 topdir ``` ``` ``` # [AVAILABILITY](#AVAILABILITY) The **lfs migrate** command is part of the [**lustre**(7)](/man/enterprise-packages/lustre/lustre.7) filesystem package since release 2.4.0 # [SEE ALSO](#SEE_ALSO) [**lctl**(8)](/man/enterprise-packages/lustre/lctl.8), [**lfs**(1)](/man/enterprise-packages/lustre/lfs.1), [**lfs-getdirstripe**(1)](/man/enterprise-packages/lustre/lfs-getdirstripe.1), [**lfs-mkdir**(1)](/man/enterprise-packages/lustre/lfs-mkdir.1), [**lfs-setdirstripe**(1)](/man/enterprise-packages/lustre/lfs-setdirstripe.1), [**lfs-setstripe**(1)](/man/enterprise-packages/lustre/lfs-setstripe.1) 2025-12-20 Lustre --- Document: lfs-migrate(1) from lustre 2.17.59-r1 (Chainguard OS) Source: https://github.com/lustre/lustre-release