lfs-find(1)

NAME

lfs-find - Lustre client utility to list files with specific attributes

SYNOPSIS

DESCRIPTION

lfs find is similar to the standard find(1) utility and is used to list files and directories with specific attributes, both regular POSIX attributes such as ownership, timestamps, and size using the same options as find(1), as well as Lustre-specific attributes such as stripe count and size, OST and MDT location, and composite layout attributes.

OPTIONS

-A, --atime

File was last accessed N*24 hours ago (if no units are given), or N*seconds, minutes, hours, days, weeks, or years ago within a margin of error of 24h if no unit is specified. Multiple units can be specified, for example 8h20m is equivalent to 500m. If multiple units are specified, the margin of error is based on the smallest unit used, so -atime 3d has a margin of error of one day, while -atime 72h has a margin of error of one hour. If both -atime +OLDER and -atime -NEWER are used, then files with atime within the range OLDER-NEWER are matched.

--attrs

File has ATTRS attribute flags. Supported attributes are (non exhaustive list): Compressed (c), Immutable (i), Append_Only (a), No_Dump (d), Encrypted (E), Automount (M)

-b, --blocks

Blocks allocated by the file is N 512-byte blocks (if no units are given), or chars (bytes), Kibi-, Mebi-, Gibi-, Tebi-, Pebi-, or Ebi-bytes if that suffix is given.

-B, --btime, --Btime, --crtime

File was created N*24 hours ago, see --atime for full details and options.

-C, --ctime

File's status was last changed N*24 hours ago, see --atime for full details and options.

--component-count, --comp-count

The file has N components in its layout.

--component-end, --comp-end

The file has component end offset N (in bytes) for any component.

--component-flags, --comp-flags

The file has components with the specified flag set. If ^flag is used, print only components not matching flag. Multiple flags can be specified, separated by commas. Valid flag names are:

--component-start, --comp-start

The file has component start offset N (in bytes) for any component.

--foreign

File has a foreign (non-Lustre/free format) layout and is of the given TYPE, if specified. Presently only none or symlink are defined types, though 32-bit numeric types can also be used.

-g, --gid

File has specified numeric group ID GID.

-G, --group

File belongs to specified GROUPNAME, or numeric group GID.

-h, --help

Print usage message.

-L, --layout

File has a layout of the given type, one of:

-l, --lazy

Use file size and blocks from MDT, if available, to avoid extra RPCs.

--ls

Print files in a format similar to 'ls -l', showing the inode number, allocated blocks, permissions, link count, owner, group, size, modification time, and path. This is equivalent to specifying the --printf '%18i %10k %M %4n %-8u %-8g %13s %t %p  option. This option is for human convenience, but the exact formatting is not guaranteed to be constant or match 'find -ls' or 'ls -l' exactly. In particular the timestamp format may change, so if a consistent output is needed as input to a parser then the format should be explicitly specified.

--links

File has N links.

--maxdepth

Limits find to decend at most N levels of directory tree.

--mindepth

Starts find at N levels of directory tree.

-m, --mdt, --mdt-index

File or directory inode is located on the specified MDT(s).

-H, --mdt-hash

DNE striped directory uses the given filename hashing function, one of:

This implicitly selects only directories to be matched, like -type d and not other file types.

-T, --mdt-count

The DNE striped directory has the given number of MDT shards. This implicitly selects only directories to be matched, like -type d and not other file types.

-N, --mirror-count

The file has N mirrors in its layout.

--mirror-state

The file has a state of state. If ^state is used, print only files not matching state. Only one state can be specified. Valid state name is:

-M, --mtime

File's data was last modified N*24 hours ago, see --atime for full details and options.

-n, --name

Filename matches the given filename, or regular expression using standard glob(7) file name regular expressions and wildcards.

--newer[XY] reference

Succeeds if timestamp X of the file being considered is newer than timestamp Y of the file reference. The letters X and Y can be any of the following letters:

a

The access time of the file reference

b|B

The birth time of the file reference

c

The inode status change time of reference

m

The modification time of the file reference

t

reference is interpreted directly as a time

Some combinations are invalid; for example, it is invalid for X to be t. Specifying -newer is equivalent to -newermm. When reference is interpreted directly as a time, currently it must be in one of the following formats: "%Y-%m-%d %H:%M:%S", "%Y-%m-%d %H:%M", "%Y-%m-%d", "%H:%M:%S", "%H:%M", to represent year, month, day, hour, minute, seconds, with unspecified times at the start of that minute or day, unspecified dates being "today", and "@%s" or "%s" the seconds since the Unix epoch (see strftime(3) for details of the time formats). Otherwise, it will report an error. If you try to use the birth time of a reference file, and the birth time cannot be determined, a fatal error message results. If you specify a test which refers to the birth time of files being examined, this test will fail for any files where the birth time is unknown.

-O, --ost

File has an object on the specified OST(s). The OST names can be specified using the whole OST target name, or just the OST index number, or OST index range like 0-3. If multiple OSTs are given in a comma-separated list, the file may have an object on any of the given OSTs. Specifying multiple OSTs allows scanning the filesystem only once when migrating objects off multiple OSTs for evacuation and replacement using lfs-migrate(1).

--perm MODE

File's permission are exactly MODE (octal or symbolic).

--perm/MODE

All of the permission bits MODE are set for the file.

--perm-MODE

Any of the permission bits MODE are set for the file. If no permission bits in MODE are set, this test matches any file.

--pool

Layout was created with the specified POOL name. For composite files, this may match the pool of any component. -P, --print Prints the file or directory name to standard output if it matches all specified parameters, one file per line with a trailing linefeed. This is the default behaviour for any matching files.

-0, --print0

Print full file name to standard output if it matches the specified parameters, followed by a NUL character. This is for use together with xargs(1) with the -0 option to handle filenames with embedded spaces or other special characters.

--printf FORMAT

Print FORMAT to standard output for each matching file, interpreting `´ escapes and `%' directives. Unlike --print, the --printf option does not automatically add a newline to the end of the string. The escapes and directives are:

%LA

Comma-separated list of file's named attribute flags, or hex value of any unknown attributes.

%Lc

File or directory stripe count. For a composite file, this is the stripe count of the last instantiated component.

%LF

File Identifier (FID) associated with the file.

%Lh

Directory's hash type (or `none´ for an unstriped directory).

%Li

File´s starting OST index (or starting MDT index for a directory). For a composite file, this is the starting OST index of the last instantiated component.

%Lo

List of all OST/MDT indices associated with a file or directory.

%Lp

File´s OST pool name. For a composite file, this is the pool associated with the last instantiated component. (NOTE: This can also be used for directories, but since MDT pools are not currently implemented, nothing will be printed.)

%LP

Numeric project ID assigned to the file or directory.

%LS

File's stripe size. For a composite file, this is the stripe size of the last instantiated component.

--projid

File belongs to the project specified by PROJID or PROJNAME. A username or group name can be used in place of a project name by specifying u:USERNAME or user:USERNAME or g:GROUPNAME or group:GROUPNAME respectively, though this name will not be shown in place of the numeric ID.

-s, --size

File size is N 512-byte blocks (if no unit is given), or N chars (bytes), Kibi-, Mebi-, Gibi-, Tebi-, Pebi-, or Ebi-bytes if a suffix is given.

-k, --skip

Skip PERCENT of the total files that match the other filters. This is useful together with lfs-migrate(1) to allow processing only a fraction of the files to rebalance files when new OSTs are added to the filesystem.

-c, --stripe-count

File has N stripes allocated. For composite files, this matches the stripe count of the last initialized component.

-i, --stripe-index

File has stripe on OST index N. Multiple OST indices can be specified in a comma-separated list, which indicates that the file has a stripe on .B any of the specified OSTs. This allows a single namespace scan for files on multiple different OSTs, if there are multiple OSTs that are being replaced.

-S, --stripe-size

Stripe size is N bytes, or Kibi-, Mebi-, Gibi-, Tebi-, Pebi-, or Ebi-abytes if a suffix is given. For composite files, this matches the stripe size of the last initialized non-extension component.

-z, --extension-size, --ext-size

Extension size is N bytes, or Kibi-, Mebi-, Gibi-, Tebi-, Pebi-, or Ebi-bytes if a suffix is given. For composite files, this matches the extension size of any extension component.

-t, --type

File has type: block, character, directory, file, pipe, symlink, or socket.

--threads N

Use N threads for parallel directory traversal. If not specified, the default thread count is calculated as min(MDT_COUNT * 4, CPU_COUNT / 2), where CPU_COUNT is the total number of CPUs configured for Lustre. This allows faster scanning of large directory trees by processing multiple directories concurrently while respecting the available CPU resources.

-u, --uid

File has specified numeric user ID UID.

-U, --user

File owned by specified USERNAME, or numeric user UID.

--xattrNAME[=VALUE]

File has an extended attribute with name matching the regular expression (regex(7)) NAME, and optionally value matching the regular expression VALUE. The regular expressions must match the complete attribute names and values, and not just a substring. This option may be specified multiple times, and the file must match all provided arguments.

NOTES

Specifying ! before an option negates its meaning (files NOT matching the parameter). Using + before a numeric value means 'more than N', while - before a numeric value means 'less than N'. If neither is used, it means 'equal to N', within the bounds of the unit specified (if any).

Numeric suffixes are in binary SI (power-of-two) units.

For compatibility with find(1) it is possible to specify long options with either a single or double leading dash.

The order of parameters does not affect how the files are matched. lfs find will first scan the directory for any specified filename, and then fetch MDT inode attributes for each matching filename. If it can make a positive or negative decision for a file based only on the MDT attributes (e.g. newer than specified time, user/group/project ID) it will not fetch the OST object attributes for that file.

BASH COMPLETION

The lfs find command supports bash-completion(1) for tab completion of option names and in some cases their arguments, with proper package installation and supporing shell usage. This can simplify usage when specifying complex search commands.

To list mountpoints, type 'lfs find TAB'. To list long options (more descriptive and easily understood) type '--TAB', and to list short options type '-TAB'. Some arguments also allow context-specific completion to the command-line option, for example generating a list of OST devices for the -ost option, or a list of OST pools for the -pool option.

BUGS

The lfs find command isn't as comprehensive as find(1). In particular, it doesn't support complex boolean expressions with -o (logical OR), only logical AND of most expressions. The parameter ordering does not affect how the files are matched. Specifying multiple identical options generally clobbers the earlier option (e.g. it is not possible to specify a range of stripe counts or multiple pools).

EXAMPLES

Efficiently lists all files in a given directory and its subdirectories, without fetching any file attributes:

# lfs find /mnt/lustre

Prints a formatted find string, listing the username, access mode and inode number:

# lfs find /mnt/lustre -printf %u %M %i 0

Recursively list all regular files in given directory more than 30 days old:

# lfs find /mnt/lustre -mtime +30 -type f -print

Recursively find all files in test that have objects on OST0002 or OST0003 and migrate them to other OSTs. See lfs-migrate(1) for more details:

# lfs find /mnt/lustre/test -o OST0002,OST0003 -print0 | lfs migrate -0

Recursively list all files ending with .mpg that have more than 3 components:

# lfs find -name *.mpg --component-count +3 /mnt/lustre

Recursively list all files that have at least one component with both 'init' and 'prefer' flags set, and doesn't have flag 'stale' set:

# lfs find --component-flags=init,prefer,^stale /mnt/lustre

Recursively list all mirrored files that have more than 2 mirrors:

# lfs find --mirror-count +2 /mnt/lustre

Recursively list all out-of-sync mirrored files:

# lfs find ! --mirror-state=ro /mnt/lustre

Recursively list all but foreign files/dirs of symlink type:

# lfs find ! --foreign=symlink /mnt/lustre

Recursively list all files with the specified "user.job" extended attribute:

# lfs find -xattr user.job=202310.* /mnt/lustre

Recursively list all files in /var/www that have any SELinux extended attribute, but that do NOT have an SELinux extended attribute with a value containing "httpd":

# lfs find -xattr security.selinux ! -xattr security.selinux=.*httpd.* /var/www

Recursively list all files in a directory with detailed information in a format similar to 'ls -l':

# lfs find /mnt/lustre --ls

AVAILABILITY

lfs find is part of the lustre(7) filesystem package since release 0.10.0

SEE ALSO

lfs(1), lfs-getdirstripe(1), lfs-getstripe(1), lfs-migrate(1), xargs(1), projid(5), lustre(7), regex(7)