| 1 | # Worker Utilization |
| 2 | |
| 3 | This library is to be used when there are 1 or more worker threads accepting requests |
| 4 | of some kind and servicing them. The goal is to provide a very simple way to monitor |
| 5 | worker threads utilization, as a percentage of the time they are busy and the amount |
| 6 | of requests served. |
| 7 | |
| 8 | ## Design goals |
| 9 | |
| 10 | 1. Minimal, if any, impact on the performance of the workers |
| 11 | 2. Easy to be integrated into any kind of worker |
| 12 | 3. No state of any kind at the worker side |
| 13 | |
| 14 | ## How to use |
| 15 | |
| 16 | When a working thread starts, call: |
| 17 | |
| 18 | ```c |
| 19 | void worker_register(const char *name); |
| 20 | ``` |
| 21 | |
| 22 | This will create the necessary structures for the library to work. |
| 23 | No need to keep a pointer to them. They are allocated as `__thread` variables. |
| 24 | |
| 25 | Then job types need to be defined. Job types are anything a worker does that can be |
| 26 | counted and their execution time needs to be reported. The library is fast enough to |
| 27 | be integrated even on workers that perform hundreds of thousands of actions per second. |
| 28 | |
| 29 | Job types are defined like this: |
| 30 | |
| 31 | ```c |
| 32 | void worker_register_job_type(size_t id, const char *name); |
| 33 | ``` |
| 34 | |
| 35 | `id` is a number starting from zero. The library is compiled with a fixed size of 50 |
| 36 | ids (0 to 49). More can be allocated by setting `WORKER_UTILIZATION_MAX_JOB_TYPES` in |
| 37 | `worker_utilization.h`. `name` can be any string up to 22 characters. This can be |
| 38 | changed by setting `WORKER_UTILIZATION_MAX_JOB_NAME_LENGTH` in `worker_utilization.h`. |
| 39 | |
| 40 | Each thread that calls `worker_register(name)` will allocate about 3kB for maintaining |
| 41 | the information required. |
| 42 | |
| 43 | When the thread stops, call: |
| 44 | |
| 45 | ```c |
| 46 | void worker_unregister(void); |
| 47 | ``` |
| 48 | |
| 49 | Again, no parameters, or return values. |
| 50 | |
| 51 | > IMPORTANT: cancellable threads need to add a call to `worker_unregister()` to the |
| 52 | > `pop` function that cleans up the thread. Failure to do so, will result in about |
| 53 | > 3kB of memory leak for every thread that is stopped. |
| 54 | |
| 55 | When you are about to do some work in the working thread, call: |
| 56 | |
| 57 | ```c |
| 58 | void worker_is_busy(size_t id); |
| 59 | ``` |
| 60 | |
| 61 | When you finish doing the job, call: |
| 62 | |
| 63 | ```c |
| 64 | void worker_is_idle(void); |
| 65 | ``` |
| 66 | |
| 67 | Calls to `worker_is_busy(id)` can be made one after another (without calling |
| 68 | `worker_is_idle()` between them) to switch jobs without losing any time between |
| 69 | them and eliminating one of the 2 clock calls involved. |
| 70 | |
| 71 | ## Implementation details |
| 72 | |
| 73 | Totally lockless, extremely fast, it should not introduce any kind of problems to the |
| 74 | workers. Every time `worker_is_busy(id)` or `worker_is_idle()` are called, a call to |
| 75 | `now_realtime_usec()` is done and a couple of variables are updated. That's it! |
| 76 | |
| 77 | The worker does not need to update the variables regularly. Based on the last status |
| 78 | of the worker, the statistics collector of netdata will calculate if the thread is |
| 79 | busy or idle all the time or part of the time. Works well for both thousands of jobs |
| 80 | per second and unlimited working time (being totally busy with a single request for |
| 81 | ages). |
| 82 | |
| 83 | The statistics collector is called by the telemetry thread of netdata. So, |
| 84 | even if the workers are extremely busy with their jobs, netdata will be able to know |
| 85 | how busy they are. |