OS Manuals

llapi_scan_namespace(3)

llapi_scan_namespace - scan a Lustre namespace, one record per object

LLAPI_SCAN_NAMESPACE(3) Lustre Library Functions LLAPI_SCAN_NAMESPACE(3)

llapi_scan_namespace - scan a Lustre namespace, one record per object

#include <lustre/lustreapi.h>
typedef int (*llapi_scan_cb_t)(const struct llapi_scan_rec *rec,
                               void *data);
int llapi_scan_namespace(const char *path,
                         const struct llapi_scan_param *sp,
                         llapi_scan_cb_t cb, void *data);








llapi_scan_namespace() walks the namespace below
    path and calls cb once for each object found, passing it a
    struct llapi_scan_rec describing that object and the caller's
    data unchanged.


sp may be NULL for the defaults. Otherwise its
    lfsp_size field must be set to sizeof(struct
    llapi_scan_param). The caller must own that many bytes, because the
    library reads them. A caller built against an older, shorter definition of
    the structure is accepted and the fields it did not set read as zero. A
    longer one is accepted too, up to 4096 bytes, as long as every byte past the
    end of this library's definition is zero — which is the caller saying
    it set no field this library could not honour — and is refused with
    -EINVAL otherwise. The size is therefore not a version number: an
    application built against a newer header runs against an older library for
    as long as it asks only for what that library has. lfsp_max_depth
    limits the descent, counted as find(1) counts it: 1 is the start
    point and its children, 2 their children in turn, and 0 is unlimited.
    lfsp_thread_count sets the number of scan threads (0 or 1 scans in
    the calling thread), and lfsp_flags takes
    LLAPI_SCAN_F_STOP_ON_ERROR to abandon the scan when an object cannot
    be read rather than skipping it. lfsp_padding must be zero: those
    bytes are already counted in lfsp_size, so a field carved out of them
    later can only be refused here, never noticed.


lfsp_want is the demand mask: the LLAPI_SCAN_*
    fields the consumer will read, or 0 for everything except
    LLAPI_SCAN_MDT_INDEX, LLAPI_SCAN_PROJID and
    LLAPI_SCAN_HSM, which each cost an ioctl or an open per object and
    are gathered only when named. LLAPI_SCAN_LMV is not one of them: it
    costs a second ioctl, but only on a directory, which is the one place it
    means anything. A scan wanting nothing outside LLAPI_SCAN_DIRENT_MASK
    performs no ioctl per object, which is what makes a name-only search cheap.
    A field not asked for may still be delivered.


A bit of lfsp_want this library does not know, or knows but
    this scan cannot answer for, is not an error: it is dropped, and
    lfsp_got reports what is left. If lfsp_got is not NULL it is
    filled, before the first record, with the subset of lfsp_want this
    scan will answer for — a property of the scan rather than of the
    library, since what a scan can answer for depends on the entry point and,
    for a scan of a target, on what the target turns out to be. Where
    lfsp_want is 0 it reports what the scanner's default came to. A bit
    in lfsp_got says the scan may fill the field, not that any one object
    has it: an object with no project id still leaves LLAPI_SCAN_PROJID
    clear in lfsr_valid. So an application can ask an older library for
    everything it might want and decide once whether the subset it is offered
    will do, rather than learning field by field that a bit never arrives.


lfsp_filter is an optional pre-filter, called before any
    I/O on the object with a record carrying only lfsr_path,
    lfsr_name and, when the directory entry knew the object type,
    LLAPI_SCAN_TYPE. It returns 0 to gather and deliver the object, 1 to
    skip it without gathering (descent into a directory is unaffected), or a
    negative value to stop the scan.


Each field of the record has a bit, one of the LLAPI_SCAN_*
    constants, which is set only when the scanner could answer for that field. A
    consumer must test the bit rather than compare against zero: a scanner that
    cannot answer and a scanner that answers zero mean different things.


Two masks report those bits, each owning the half of the record it
    describes. The fields of lfsr_stx are reported by
    lfsr_stx.stx_mask, and the low LLAPI_SCAN_* constants are the
    kernel's STATX_* values, so the two are the same bits under either
    name. Everything else is reported by lfsr_valid, whose own low half
    is reserved and always zero. lfsp_want takes either half.


lfsr_size holds the size of the record the scanner filled,
    so a consumer built against an older definition of the structure continues
    to work; fields are only ever appended.


lfsr_stx begins the record, so a struct llapi_scan_rec
    * may be cast to a struct statx * and handed to anything that
    takes one — &rec->lfsr_stx says the same
    thing and costs no more. Every other field therefore sits at an offset
    sizeof(struct statx), which the library asserts at build time is
    still the 256 bytes it has been since Linux 4.11: a statx that grew would
    move them under a consumer already built against the old ones, and
    lfsr_size reports how much of a record was filled, never that a field
    moved.


For a regular file, LLAPI_SCAN_SIZE and
    LLAPI_SCAN_BLOCKS are set only when the MDT can answer
    authoritatively. The file data lives on OSTs, so in the general case the
    size is not the MDT's to give, and the record says so rather than reporting
    zero. When the MDT holds a lazy size-on-MDT value instead,
    LLAPI_SCAN_LAZY_SIZE (and LLAPI_SCAN_LAZY_BLOCKS) says so and
    the value is in lfsr_stx.stx_size. STATX_SIZE stays clear for
    it, as it does in llite when the MDT's answer is not authoritative. A
    consumer that needs a strict size for every regular file must stat it, which
    is what lfs-find(1) does for --size; lfsr_parent_fd and
    lfsr_name are there so it can do so with fstatat(2).
    lfsr_parent_fd is the parent's descriptor only while
    lfsp_thread_count is 1: above that each subdirectory is a work unit
    of its own and the walk enters it with no parent open, so a directory's
    record carries -1. A consumer that needs the parent above one thread opens
    lfsr_path itself.


The layout is delivered raw, as lfsr_lmm and
    lfsr_lmmsize, for the caller to decode with
    llapi_layout_get_by_xattr(3). That call takes a non-const buffer and
    byte-swaps it in place, so a consumer copies lfsr_lmmsize bytes and
    decodes the copy: lfsr_lmm points into the scan's own buffer, which
    the walk reuses for the next object. A directory's stripe arrives likewise
    as lfsr_lmv and lfsr_lmvsize, under LLAPI_SCAN_LMV,
    which is set only for a directory that has one.


LLAPI_SCAN_HSM fills lfsr_hsm_states from enum
    hsm_states and lfsr_hsm_archive_id, for a regular file that has
    them. State 0 with the bit set means the scan asked and the file has no HSM
    state, which is not the same as the bit being clear.






Each field is set only when its bit is set, in
    lfsr_stx.stx_mask for the fields of lfsr_stx and in
    lfsr_valid for the rest.



  

  
lfsr_fid — the object's FID.

  

  
the S_IFMT part of lfsr_stx.stx_mode, from the directory
      entry, with no I/O behind it.

  

  
lfsr_stx.stx_mode in full.

  
  
lfsr_stx.stx_nlink.

  

  
/LLAPI_SCAN_GID lfsr_stx.stx_uid and
      lfsr_stx.stx_gid.

  

  
lfsr_projid.

  

  
/LLAPI_SCAN_BLOCKS lfsr_stx.stx_size and
      lfsr_stx.stx_blocks, authoritatively; the LAZY_ pair says
      the value is a size-on-MDT one instead.

  

  
/MTIME/CTIME/BTIME lfsr_stx.stx_atime,
      lfsr_stx.stx_mtime, lfsr_stx.stx_ctime and
      lfsr_stx.stx_btime, each a whole struct statx_timestamp. The
      MDT reports whole seconds today, so tv_nsec is 0; the field is
      there for when it reports more.

  

  
lfsr_stx.stx_attributes, set when the object's flags were answered
      — OBD_MD_FLFLAGS in the MDT's reply, which it sets only when
      it filled them. The value is masked down to the STATX_ATTR_* bits
      named in lfsr_stx.stx_attributes_mask, which is the scanner's own
      declaration of what it can represent; flags outside it are masked off
      rather than reported. This bit is in lfsr_valid because statx has
      no mask bit of its own for stx_attributes.

  

  
lfsr_lmm and lfsr_lmmsize.

  

  
lfsr_lmv and lfsr_lmvsize, with
      LLAPI_SCAN_LMV_FOREIGN set beside it when the buffer is a struct
      lmv_foreign_md (declared in <linux/lustre/lustre_idl.h>)
      rather than an lmv_user_md. Naming LLAPI_SCAN_LMV_FOREIGN in
      lfsp_want asks for LLAPI_SCAN_LMV too, and lfsp_got
      reports both.

  

  
lfsr_mdt_index.

  

  
lfsr_hsm_states and lfsr_hsm_archive_id.




lfsr_path and lfsr_name are always set.
    lfsr_parent_fd is an open descriptor on the parent directory and
    lfsr_fd one on the object, or -1 when the scan holds none; neither is
    the consumer's to close, and both are valid only for the duration of the
    callback.








path does not have to name a Lustre filesystem, and a walk
    that crosses onto one that is not — a bind mount, a local disk below
    the mount point — does not stop there. Attributes then come from
    stat(2) instead of the MDT, and the record says so: a field that
    source cannot answer leaves its bit clear rather than reading as zero. In
    particular a stat has no FID, so LLAPI_SCAN_FID is clear and
    lfsr_fid must not be read; the same holds for
    LLAPI_SCAN_LAYOUT, LLAPI_SCAN_LMV, LLAPI_SCAN_MDT_INDEX
    and LLAPI_SCAN_HSM.


What such an object does carry is what a stat answers
    — type, mode, link count, owner, size, blocks and the three times
    — plus LLAPI_SCAN_PROJID where lfsp_want names it and
    the filesystem supports project quotas. It is the same walk with
    stat(2) as the attribute source, not a second scanner.










llapi_scan_namespace() returns:



  
0

  
on success.

  
<0

  
a negative errno on failure, or the negative value cb or
      lfsp_filter returned to stop the scan, passed back unchanged. The
      two are not distinguishable here: a filter stops only by returning a
      negative, its positive values meaning skip. A consumer that must tell them
      apart should stop from cb instead, whose positive values come back
      under >0 below, or pick a value this page's ERRORS does not
      list.

  
>0

  
the value a callback returned to stop the scan, unchanged.











  

  
path or cb is NULL, or lfsp_size is smaller than the
      fields up to and including lfsp_want, or is larger than 4096 bytes,
      or is larger than the library's definition of the parameter structure and
      a byte past the end of it is not zero, or lfsp_flags sets a bit
      this library does not define, or lfsp_padding is not zero. Neither
      changes lfsp_size, so refusing them is what gives flags and the
      reserved bytes the same guarantee fields already have: a caller never has
      one silently ignored.

  

  
path is longer than PATH_MAX.










With lfsp_thread_count greater than 1 the callback is
    invoked concurrently from several threads and must be thread-safe. The
    record passed to each call is private to that call; data is shared
    between them.






The record is valid only for the duration of the callback, and so
    is everything it points at. A consumer that keeps any part of either must
    copy it. What follows says when each part stops being the object's, because
    the answers differ and two of them are shorter than the rule above
  suggests.


The record itself is a local of the scan's own traversal callback,
    so the fields held by value — lfsr_stx, lfsr_fid,
    lfsr_projid and the rest — are gone when the callback returns.
    Copying the structure is enough for those, and enough for nothing else.


lfsr_path is the traversal's path buffer, rewritten as the
    walk moves on. For a directory that is its own descent: the buffer has the
    first child appended to it before any callback below it runs, so a
    directory's path does not last even until its next sibling. lfsr_name
    points into that buffer rather than beside it, and does not outlive it.


lfsr_lmm and lfsr_lmv are the buffers the MDT's
    answer arrives in, reused from one object to the next rather than one per
    object. Within the callback the record is honest about them: an object with
    no layout leaves lfsr_lmm NULL rather than pointing at the last
    object's bytes. The buffer itself is rewritten by the next object's
    gather, though, which is later than the callback returns — so a
    pointer kept past that describes whichever object was gathered most
    recently, and the gather clears the layout's magic without clearing the body
    behind it. With lfsp_thread_count above 1 a kept pointer can be freed
    memory instead: each subdirectory is walked with buffers of its own,
    released when that subdirectory is done. Treat either as an accident of the
    implementation and not as room.


lfsr_fd is the shortest-lived. For a directory it is the
    traversal's own descriptor and lasts while the subtree is walked. For
    anything else it is a descriptor the gather opened only because
    lfsp_want asked for a field an ioctl answers, and it is closed as
    soon as the callback returns — so a kept copy names a closed
    descriptor whose number the scan is about to reuse. lfsr_parent_fd is
    the parent directory's and lasts while that directory is walked. Neither is
    the consumer's to close.










To count the regular files under a directory, testing the validity
    mask before reading a field:




        static int count_cb(const struct llapi_scan_rec *rec, void *data)


        {


                __u64 *n = data;


                if ((rec->lfsr_stx.stx_mask & STATX_TYPE) &&


                    S_ISREG(rec->lfsr_stx.stx_mode))


                        (*n)++;


                return 0;


        }


        __u64 n = 0;


        int rc = llapi_scan_namespace("/mnt/lustre", NULL, count_cb, &n);









llapi_scan_namespace() is part of the lustre(7) user
    application interface library since release 2.18.0.








lfs-find(1), llapi_layout_get_by_xattr(3),
    lustreapi(7)







  

    2026-08-18
    Lustre User API
  






lustre 2.17.60-r0 · enterprise-packages · ingested 2026-10-22