TOK(3ast) TOK(3ast)

# [NAME](#NAME)

tok - space separated token stream routines

# [SYNOPSIS](#SYNOPSIS)

#include <ast.h>

void* tokopen(char* string) char* tokread(void* tok) void tokclose(void* tok)

int tokscan(char* string, char\** next, const char* format, ...);

Sfio\_t* tokline(const char* input, int flags, int* line);

# [DESCRIPTION](#DESCRIPTION)

tokopen returns a pointer to a space separated token stream on the 0 terminated string string. tokread returns a pointer to the next space separated token in the token stream tok as returned by tokopen. 0 is returned when no tokens remain. tokread temporarily modifies string by inserting 0's to terminate each token. tokclose closes the token stream and restores string to its original state.

tokscan scans the string string for tokens specified in format. It is a more forgiving [*sscanf*(3)](/man/os/ksh/sscanf.3). If next != 0 then it will point to the next unread character in string on return. The number of scanned tokens is returned. -1 is returned if string was not empty and format failed to match and tokens.

*space* in format matches 0 or more *space* or *tab* characters. *newline* in format eats the remainder of the current line in string. "...", '...' and \\*character* quotes are interpreted. A quoted *carriage-return* is converted to *newline*. *newline* in string is equivalent to end of string except when quoted. *\\newline* is a line splice.

% in format prefixes format conversion characters; each conversion character corresponds to a tokscan argument following the format argument. The format conversions are:

%c

A single char.

%hd %d %ld

\[short, int, long] base 10 integer.

%hn %n %ln

\[short, int, long] C-style base integer.

%ho %o %lo

\[short, int, long] base 8 integer.

%s

String.

%hu %u %lu

\[short, int, long] C-style base unsigned integer.

%v

The next two arguments are a pointer to a char\** argument vector and the maximum number of elements in the vector.

%hx %x %lx

\[short, int, long] base 16 integer.

%s and %v data may also be counted length strings of the form **(count:data)** where *count* is the number of characters in *data* and the terminating ) may also be a *tab*, or the data may [be (null)](/man/os/ksh/be.null) which represents the NULL string.

tokline returns an [*sfio*(3)](/man/os/ksh/sfio.3) stream to a file or string that splices *\\newline* into single lines, allows "..." and '...' to quotes to span *newlines* (done by translating quoted *newline* to *carriage-return*; tokscan above converts quoted *carriage-return* back to *newline*), and deletes *# ... newline* comments. This is done by pushing an *sfio* discipline onto a string or file stream. Seeks are disabled on the resulting stream. If flags == SFIO\_READ then input is a file name; If flags == SFIO\_STRING then input is a 0 terminated string; otherwise input is an open Sfio\_t* stream. If line != 0 then it points to a line count that is initialized to 0 and is incremented for each input line.

# [SEE ALSO](#SEE_ALSO)

[sfio(3)](/man/os/ksh/sfio.3)

---

Document: tok(3ast) from ksh 1.0.10-r37 (Chainguard OS)
Source: https://github.com/ksh93/ksh