Wed, 09 May 2018 20:15:10 +0200
adds new shift operations for UcxBuffer (including tests and a usage example in modules.md)
1 ---
2 title: Modules
3 ---
5 UCX provides several modules for data structures and algorithms.
6 You may choose to use specific modules by inclueding the corresponding header
7 file.
8 Please note, that some modules make use of other UCX modules.
9 For instance, the [Allocator](#allocator) module is used by many other modules
10 to allow flexible memory allocation.
11 By default the header files are placed into an `ucx` directory within your
12 systems include directory. In this case you can use a module by including it
13 via `#include <ucx/MODULENAME.h>`.
14 Required modules are included automatically.
16 <div id="modules" align="center">
18 ----------------------- ---------------------- ---------------------------- -------------------------
19 [Allocator](#allocator) [AVL Tree](#avl-tree) [Buffer](#buffer) [List](#list)
20 [Logging](#logging) [Map](#map) [Memory Pool](#memory-pool) [Properties](#properties)
21 [Stack](#stack) [String](#string) [Testing](#testing) [Utilities](#utilities)
22 ----------------------- ---------------------- ---------------------------- -------------------------
24 </div>
26 ## Allocator
28 *Header file:* [allocator.h](api/allocator_8h.html)
29 *Required modules:* None.
31 A UCX allocator consists of a pointer to the memory area / pool and four
32 function pointers to memory management functions operating on this memory
33 area / pool. These functions shall behave equivalent to the standard libc
34 functions `malloc`, `calloc`, `realloc` and `free`.
36 The signature of the memory management functions is based on the signature
37 of the respective libc function but each of them takes the pointer to the
38 memory area / pool as first argument.
40 As the pointer to the memory area / pool can be arbitrarily chosen, any data
41 can be provided to the memory management functions. One example is the
42 [UCX Memory Pool](#memory-pool).
44 ## AVL Tree
46 *Header file:* [avl.h](api/avl_8h.html)
47 *Required modules:* [Allocator](#allocator)
49 This binary search tree implementation allows average O(1) insertion and
50 removal of elements (excluding binary search time).
51 All common binary tree operations are implemented. Furthermore, this module
52 provides search functions via lower and upper bounds.
54 ### Filtering items with a time window
56 Suppose you have a list of items which contain a `time_t` value and your task
57 is to find all items within a time window `[t_start, t_end]`.
58 With AVL Trees this is easy:
59 ```C
60 /* ---------------------
61 * Somewhere in a header
62 */
63 typedef struct {
64 time_t ts;
65 // other important data
66 } MyObject;
68 /* -----------
69 * Source code
70 */
72 UcxAVLTree* tree = ucx_avl_new(ucx_longintcmp);
73 // ... populate tree with objects, use '& MyObject.ts' as key ...
76 // Now find every item, with 30 <= ts <= 70
77 time_t ts_start = 30;
78 time_t ts_end = 70;
80 printf("Values in range:\n");
81 for (
82 UcxAVLNode* node = ucx_avl_find_node(
83 tree, (intptr_t) &ts_start,
84 ucx_longintdist, UCX_AVL_FIND_LOWER_BOUNDED);
85 node && (*(time_t*)node->key) <= ts_end;
86 node = ucx_avl_succ(node)
87 ) {
88 printf(" ts: %ld\n", ((MyObject*)node->value)->ts);
89 }
91 ucx_avl_free_content(tree, free);
92 ucx_avl_free(tree);
93 ```
95 ## Buffer
97 *Header file:* [buffer.h](api/buffer_8h.html)
98 *Required modules:* None.
100 Instances of this buffer implementation can be used to read from or to write to
101 memory like you would do with a stream. This allows the use of
102 `ucx_stream_copy()` from the [Utilities](#utilities) module to copy contents
103 from one buffer to another or from file or network streams to the buffer and
104 vice-versa.
106 More features for convenient use of the buffer can be enabled, like automatic
107 memory management and automatic resizing of the buffer space.
108 See the documentation of the macro constants in the header file for more
109 information.
111 ### Add line numbers to a file
113 When reading a file line by line, you have three options: first, you could limit
114 the maximum supported line length.
115 Second, you allocate a god buffer large
116 enough for the most lines a text file could have.
117 And third, undoubtedly the best option, you start with a small buffer, which
118 adjusts on demand.
119 An `UcxBuffer` can be created to do just that for you.
120 Just pass the `UCX_BUFFER_AUTOEXTEND` option to the initialization function.
121 Here is a full working program, which adds line numbers to a file.
122 ```C
123 #include <stdio.h>
124 #include <ucx/buffer.h>
125 #include <ucx/utils.h>
127 int main(int argc, char** argv) {
129 if (argc != 2) {
130 fprintf(stderr, "Usage: %s <file>\n", argv[0]);
131 return 1;
132 }
134 FILE* input = fopen(argv[1], "r");
135 if (!input) {
136 perror("Canno read input");
137 return 1;
138 }
140 const size_t chunksize = 256;
142 UcxBuffer* linebuf =
143 ucx_buffer_new(
144 NULL, // the buffer should manage the memory area for us
145 chunksize, // initial buffer size should be the chunk size
146 UCX_BUFFER_AUTOEXTEND); // the buffer will grow when necessary
148 size_t lineno = 1;
149 do {
150 // read line chunk
151 size_t read = ucx_stream_ncopy(
152 input, linebuf, fread, ucx_buffer_write, chunksize);
153 if (read == 0) break;
155 // handle line endings
156 do {
157 sstr_t bufstr = ucx_buffer_to_sstr(linebuf);
158 sstr_t nl = sstrchr(bufstr, '\n');
159 if (nl.length == 0) break;
161 size_t linelen = bufstr.length - nl.length;
162 sstr_t linestr = sstrsubsl(bufstr, 0, linelen);
164 printf("%zu: %" PRIsstr "\n", lineno++, SFMT(linestr));
166 // shift the buffer to the next line
167 ucx_buffer_shift_left(linebuf, linelen+1);
168 } while(1);
170 } while(1);
172 // print the 'noeol' line, if any
173 sstr_t lastline = ucx_buffer_to_sstr(linebuf);
174 if (lastline.length > 0) {
175 printf("%zu: %" PRIsstr, lineno, SFMT(lastline));
176 }
178 fclose(input);
179 ucx_buffer_free(linebuf);
181 return 0;
182 }
183 ```
185 ## List
187 *Header file:* [list.h](api/list_8h.html)
188 *Required modules:* [Allocator](#allocator)
190 This module provides the data structure and several functions for a doubly
191 linked list. Among the common operations like insert, remove, search and sort,
192 we allow convenient iteration via a special `UCX_FOREACH` macro.
194 ## Logging
196 *Header file:* [logging.h](api/logging_8h.html)
197 *Required modules:* [Map](#map), [String](#string)
199 The logging module comes with some predefined log levels and allows some more
200 customization. You may choose if you want to get timestamps or source file and
201 line number logged automatically when outputting a message.
204 ## Map
206 *Header file:* [map.h](api/map_8h.html)
207 *Required modules:* [Allocator](#allocator), [String](#string)
209 This module provides a hash map implementation using murmur hash 2 and separate
210 chaining with linked lists. Similarly to the list module, we provide a
211 `UCX_MAP_FOREACH` macro to conveniently iterate through the key/value pairs.
213 ## Memory Pool
215 *Header file:* [mempool.h](api/mempool_8h.html)
216 *Required modules:* [Allocator](#allocator)
218 Here we have a concrete allocator implementation in the sense of a memory pool.
219 This pool allows you to register destructor functions for the allocated memory,
220 which are automatically called on the destruction of the pool.
221 But you may also register *independent* destructor functions within a pool in
222 case, some external library allocated memory for you, which you wish to be
223 destroyed together with this pool.
225 ## Properties
227 *Header file:* [properties.h](api/properties_8h.html)
228 *Required modules:* [Map](#map)
230 This module provides load and store function for `*.properties` files.
231 The key/value pairs are stored within an UCX Map.
233 ### Example: Loading properties from a file
235 ```C
236 // Open the file as usual
237 FILE* file = fopen("myprops.properties", "r");
238 if (!file) {
239 // error handling
240 return 1;
241 }
243 // Load the properties from the file
244 UcxMap* myprops = ucx_map_new(16);
245 if (ucx_properties_load(myprops, file)) {
246 // error handling
247 fclose(file);
248 ucx_map_free(myprops);
249 return 1;
250 }
252 // Print out the key/value pairs
253 char* propval;
254 UcxMapIterator propiter = ucx_map_iterator(myprops);
255 UCX_MAP_FOREACH(key, propval, propiter) {
256 printf("%s = %s\n", (char*)key.data, propval);
257 }
259 // Don't forget to free the values before freeing the map
260 ucx_map_free_content(myprops, NULL);
261 ucx_map_free(myprops);
262 fclose(file);
263 ```
264 ## Stack
266 *Header file:* [stack.h](api/stack_8h.html)
267 *Required modules:* [Allocator](#allocator)
269 This concrete implementation of an UCX Allocator allows you to grab some amount
270 of memory which is then handled as a stack.
271 Please note, that the term *stack* only refers to the behavior of this
272 allocator. You may still choose if you want to use stack or heap memory
273 for the underlying space.
275 A typical use case is an algorithm where you need to allocate and free large
276 amounts of memory very frequently.
278 ## String
280 *Header file:* [string.h](api/string_8h.html)
281 *Required modules:* [Allocator](#allocator)
283 This module provides a safe implementation of bounded string.
284 Usually C strings do not carry a length. While for zero-terminated strings you
285 can easily get the length with `strlen`, this is not generally possible for
286 arbitrary strings.
287 The `sstr_t` type of this module always carries the string and its length to
288 reduce the risk of buffer overflows dramatically.
290 ### Initialization
292 There are several ways to create an `sstr_t`:
294 ```C
295 /* (1) sstr() uses strlen() internally, hence cstr MUST be zero-terminated */
296 sstr_t a = sstr(cstr);
298 /* (2) cstr does not need to be zero-terminated, if length is specified */
299 sstr_t b = sstrn(cstr, len);
301 /* (3) S() macro creates sstr_t from a string using sizeof() and using sstrn().
302 This version is especially useful for function arguments */
303 sstr_t c = S("hello");
305 /* (4) ST() macro creates sstr_t struct literal using sizeof() */
306 sstr_t d = ST("hello");
307 ```
309 You should not use the `S()` or `ST()` macro with string of unknown origin,
310 since the `sizeof()` call might not coincide with the string length in those
311 cases. If you know what you are doing, it can save you some performance,
312 because you do not need the `strlen()` call.
314 ### Finding the position of a substring
316 The `sstrstr()` function gives you a new `sstr_t` object starting with the
317 requested substring. Thus determining the position comes down to a simple
318 subtraction.
320 ```C
321 sstr_t haystack = ST("Here we go!");
322 sstr_t needle = ST("we");
323 sstr_t result = sstrstr(haystack, needle);
324 if (result.ptr)
325 printf("Found at position %zd.\n", haystack.length-result.length);
326 else
327 printf("Not found.\n");
328 ```
330 ### Spliting a string by a delimiter
332 The `sstrsplit()` function (and its allocator based version `sstrsplit_a()`) is
333 very powerful and might look a bit nasty at a first glance. But it is indeed
334 very simple to use. It is even more convenient in combination with a memory
335 pool.
337 ```C
338 sstr_t test = ST("here::are::some::strings");
339 sstr_t delim = ST("::");
341 ssize_t count = 0; /* no limit */
342 UcxMempool* pool = ucx_mempool_new_default();
344 sstr_t* result = sstrsplit_a(pool->allocator, test, delim, &count);
345 for (ssize_t i = 0 ; i < count ; i++) {
346 /* don't forget to specify the length via the %*s format specifier */
347 printf("%*s\n", result[i].length, result[i].ptr);
348 }
350 ucx_mempool_destroy(pool);
351 ```
352 The output is:
354 here
355 are
356 some
357 strings
359 The memory pool ensures, that all strings are freed.
361 ## Testing
363 *Header file:* [test.h](api/test_8h.html)
364 *Required modules:* None.
366 This module provides a testing framework which allows you to execute test cases
367 within test suites.
368 To avoid code duplication within tests, we also provide the possibility to
369 define test subroutines.
371 ## Utilities
373 *Header file:* [utils.h](api/utils_8h.html)
374 *Required modules:* [Allocator](#allocator), [String](#string)
376 In this module we provide very general utility function for copy and compare
377 operations.
378 We also provide several `printf` variants to conveniently print formatted data
379 to streams or strings.
381 ### A simple copy program
383 The utilities package provides several stream copy functions.
384 One of them has a very simple interface and can, for instance, be used to copy
385 whole files in a single call.
386 This is a minimal working example:
387 ```C
388 #include <stdio.h>
389 #include <ucx/utils.h>
391 int main(int argc, char** argv) {
393 if (argc != 3) {
394 fprintf(stderr, "Use %s <src> <dest>", argv[0]);
395 return 1;
396 }
398 FILE *srcf = fopen(argv[1], "r"); // insert error handling on your own
399 FILE *destf = fopen(argv[2], "w");
401 size_t n = ucx_stream_copy(srcf, destf, fread, fwrite);
402 printf("%zu bytes copied.\n", n);
404 fclose(srcf);
405 fclose(destf);
408 return 0;
409 }
410 ```
412 ### Automatic allocation for formatted strings
414 The UCX utility function `ucx_asprintf()` and it's convenient shortcut
415 `ucx_sprintf` allow easy formatting of strings, without ever having to worry
416 about the required space.
417 ```C
418 sstr_t mystring = ucx_sprintf("The answer is: %d!", 42);
419 ```
420 Still, you have to pass `mystring.ptr` to `free()` (or the free function of
421 your allocator, if you use `ucx_asprintf`).
422 If you don't have all the information ready to build your string, you can even
423 use a [UcxBuffer](#buffer) as a target with the utility function
424 `ucx_bprintf()`.
425 ```C
426 UcxBuffer* strbuffer = ucx_buffer_new(NULL, 512, UCX_BUFFER_AUTOEXTEND);
428 for (unsigned int i = 2 ; i < 100 ; i++) {
429 ucx_bprintf(strbuffer, "Integer %d is %s\n",
430 i, prime(i) ? "prime" : "not prime");
431 }
433 // print the result to stdout
434 printf("%s", (char*)strbuffer->space);
436 ucx_buffer_free(strbuffer);
437 ```