Mercurial > hg > ucx / changeset

--- a/docs/Writerside/topics/string.h.md	Sun Feb 23 13:47:10 2025 +0100
+++ b/docs/Writerside/topics/string.h.md	Sun Feb 23 14:03:15 2025 +0100
@@ -252,8 +252,42 @@
 bool cx_strtok_next_m(CxStrtokCtx *ctx, cxmutstr *token);
 ```
 
-> Documentation work in progress.
->{style="warning"}
+You can tokenize a string by creating a _tokenization_ context with `cx_strtok()`,
+and calling `cx_strtok_next()` or `cx_strtok_next_m()` as long as they return `true`.
+
+The tokenization context is initialized with the string `str` to tokenize,
+one delimiter `delim`, and a `limit` for the maximum number of tokens.
+When `limit` is reached, the remaining part of `str` is returned as one single token.
+
+You can add additional delimiters to the context by calling `cx_strtok_delim()`, and
+specifying an array of delimiters to use.
+
+> Regardless of how the context was initialized, you can use either `cx_strtok_next()`
+> or `cx_strtok_next_m()` to retrieve the tokens. However, keep in mind that modifying
+> characters in a token returned by `cx_strtok_next_m()` has only defined behavior, when the
+> underlying `str` is a `cxmutstr`.
+
+### Example
+
+```C
+#include <cx/string.h>
+
+cxstring str = cx_str("an,arbitrarily;||separated;string");
+
+// create the context
+CxStrtokCtx ctx = cx_strtok(str, CX_STR(","), 10);
+
+// add two more delimters
+cxstring delim_more[2] = {CX_STR("||"), CX_STR(";")};
+cx_strtok_delim(&ctx, delim_more, 2);
+
+// iterate over the tokens
+cxstring tok;
+while(cx_strtok_next(&ctx, &tok)) {
+    // to something with the tokens
+    // be aware that tok is NOT zero-terminated!
+}
+```
 
 ## Conversion to Numbers