Response time jitter is one of the most sneaky source of problems when designing a real time system. When using a RTOS like ChibiOS/RT one must be aware of what the jitter is and how it can affect the performance of the system. A good place to start is this Wikipedia article.
The total execution time of an interrupt handler includes:
The interrupt latency is the time interval from an interrupt event to the execution of the handler code. Unfortunately this time is not constant in most cases, see the following graph: In this scenario the jitter (busy state) is represented by the sum of:
This is the time between an event, for example an interrupt, and the execution of the thread that will process it. Imagine the following graph as the continuation of the previous one: In this scenario all the jitter sources previously discussed are also present and there is the added jitter caused by the activity of the higher priority threads.
For each of the previously described jitter sources there are possible mitigation actions.
An obvious mitigation action is to optimize the interrupt handler code as much as possible for speed. Complex actions should never be performed in interrupt handlers, an handler should just serve the interrupt and wakeup a dedicated thread in order to handle the bulk of the work.
Another possible mitigation action is to evaluate if a specific interrupt handler really needs to interact with the OS, if the handler uses full stand-alone code then it is possible to remove the OS related overhead.
OS kernels usually protect some critical internal data structures by disabling (fully in simple architectures, to some extent in more advanced microcontrollers) the interrupt sources. Because of this the kernel itself is a jitter cause, a good OS design minimizes the jitter caused by the kernel by using adequate data structures, algorithms and coding practices. A good OS design is not the whole story, some OS primitives may generate more or less jitter depending on the system state, for example the maximum number of threads on a certain queue, the maximum number of nested mutexes and so on. Some algorithms employed internally can have constant execution time but others may have linear execution time or be even more complex.
At thread level, the response time is still affected by the interrupt-related jitter but mainly by the activity of the higher priority threads and contention on protected resources.
It is possible to improve the system overall response time and reduce jitter by carefully assigning priorities to the various threads and carefully designing mutual exclusion zones, or even better, design out mutual exclusion at all if possible. The use of the proper synchronization mechanism (semaphores, mutexes, events, messages and so on) also helps to improve the overall system performance. The use of the Priority Inheritance algorithm can improve the overall response time and reduce jitter but it is not a magic wand, a proper system design comes first.