Introduction
There is a widespread belief that simpler is better for RTOSs. But if you find that you are spending time programming around your RTOS’s deficiencies or debugging problems due to your RTOS not behaving as expected, then your RTOS is not serving you well.
It is often the case that a simple RTOS behaves well 99.999% of the time, which is good enough to get it out of the lab. It is the 0.001% that is the problem. It leads to infrequent glitches that take days or weeks to find and fix or to find a work-around.
Yogi Berra once said “Ninety percent of short putts don’t go in”. One has to have watched a long putt travel perfectly on line and stop one inch from the hole to understand the frustration behind his quote. We face a similar frustration in our work: five 9’s is good enough for gold, but not for embedded systems. In what follows we look into some of the pitfalls of RTOSs that cause the 0.001% problems.
Binary Semaphores
Using binary semaphores for mutual exclusion creates two problems:
- Any task can signal a binary semaphore thus defeating its protection.
- A task will block itself and all equal and lower-priority tasks if it tests the same semaphore twice.
Mutexes are designed to avoid these pitfalls and should always be used instead. What about binary event and binary resource semaphores? These could also cause the same problems. However, it is unlikely that a task is going to wait for the same event or the same resource in two different places in its code.
Priority Change
In some RTOSs, when a task’s priority is changed, the task is not requeued. Hence, if the task is in a queue, it stays where it is. But, if the task is not in a queue when its priority is changed and it is then enqueued, it ends up where it should be. Inconsistencies like this can cause problems that are difficult to find because they seldom occur and may show up as completely unrelated failures.
Priority Step Down
Most RTOSs implement priority promotion for mutexes. As a result, when a higher-priority task waits at a mutex, the mutex owner’s priority is increased to the higher-priority. If an even higher-priority task waits at the mutex, the owner’s priority is increased again. Thus owner priority stair-steps up. What happens if the highest priority task is dequeued? Does the owner priority stair-step down? Probably not. Why is this important?
The reason the highest priority task was dequeued could be because it timed out. That being the case, it is reasonable to expect that it would immediately deal with the timeout in order to not miss its deadline. But if the mutex owner’s priority has not stepped down, the timed-out task will not be able to run until the mutex owner finishes[1]. If the mutex owner finishes quickly, the timed-out task may still be able to make its deadline. If not, the timed-out task will miss its deadline. This failure may depend upon an unusual sequence of events, thereby being a difficult problem to find and fix.
A related problem is that tasks having priorities above that of the new top-priority waiting task, but not above the incorrect owner priority, are needlessly blocked so they might miss their deadlines. This kind of secondary effect can be really hard to track down.
Priority Promotion Propagation
That’s a mouthful. What it means is that priority promotion is propagated to the next linked mutex. Two mutexes are linked if the owner of one is waiting for another. It is probably rare in current embedded systems that a task would need to get two mutexes in order to access a critical region. Consequently, most RTOSs probably do not propagate priority promotion. But not doing so defeats the whole idea of it. If the priority of the linked mutex owner is not promoted, then unlimited priority inversion[2] is still possible and priority promotion has failed to prevent it. This problem may not be possible in an initial embedded system, yet some day a change is made (e.g. adding an open source component) and the problem becomes possible, though still rare and hard to find.
Priority Step Down Propagation
Also needed for the same reasons as priority step down.
Ceiling Priority
A few RTOSs implement both priority promotion and ceiling priority for mutexes. Ceiling priority has the effect of assigning priority to the object being protected by the mutex. Thus, when a task becomes the mutex owner, its priority is boosted to the ceiling. In contrast, priority inheritance waits until a high-priority task needs the object. The advantage of ceiling priority is that a lower-priority owner is likely to finish sooner. Thus the object is more likely to be available when the higher-priority task needs it. If both choices are available, it may be decided that object priority, not priority promotion, is what is actually needed for some objects in a system.
A ceiling priority can be the same as the highest priority task that uses the object, or higher, or lower. In addition, it can be combined with priority promotion such that ceiling priority covers tasks up to a certain priority level and tasks above that level are covered by priority promotion.
Abandoned Mutexes
A task should not be deleted, stopped, or suspended while it owns a mutex, else tasks waiting for the mutex cannot run. Some RTOSs maintain a list of owned mutexes to make it easy to release them, in these cases. For other RTOSs it is necessary to search all mutexes to find those owned by the task.
Task Lock
Locking a task is an efficient way to protect a critical region from a preempting task that also uses the region. Task lock should be implemented with a counter that counts up each time the task is locked and down each time the task is unlocked. Otherwise, if the lock is a Boolean, a task locked twice or more will be unlocked once and thus is unlocked when it is expected to be locked.
If the current task is deleted, stopped, or suspended, the lock counter must be cleared so that the next task is not inadvertently locked. Therefore, when the task is restarted or resumed, it will not be locked. But, if the stop or suspend did not actually occur (e.g. SemTest() passed) the RTOS should still clear the lock counter or else operation will be inconsistent (i.e. the resource protected some times and not others). Some RTOSs may not do this.
Other Unlimited Priority Inversions
RTOSs have many objects at which tasks can wait, yet none of these objects have priority promotion. Thus they can cause unlimited priority inversions. This may be a reason why event-driven designs are becoming popular. In these designs, an event handler (aka active object) waits for an event, then runs to completion while processing it. It does not wait on any RTOS objects. The result is that operation is more predictable[3].
Some RTOSs provide special features to support event handling [Ref 1]; others do not. For example, most RTOSs defer part of interrupt processing to tasks. This brings in the many problems related to tasks such as dispatch time, preemption, and priority inversion. These can result in output jitter that is difficult to fix. The SMX RTOS allows deferred interrupt processing to by done by light-weight link service routines (LSRs) that run ahead of all tasks [Ref 2].
When a Mutex is Wrong
A mutex is not always the best solution for resource protection. For example, a client-server structure based upon messaging can be a preferrable way to protect non-reentrant resources vs. a mutex. The advantage is that a client can request a service then go about its business. This works particularly well if the service is something of low importance like outputting a status message or recording events.
Timeouts
It is beneficial to assign carefully considered timeouts to all task waits. This means not using INF or DFLT, but to think about what timeout should be in each case. The RTOS should report timeouts as errors, otherwise it may be difficult to determine that a problem is due to a timeout.
Ideally, an RTOS provides a timeout priority which is above the normal task priority. For object timeouts or data-transfer protocols, a timeout is an event that should be delt with immediately. However, if the task is blocked from running by another task, then it cannot do this.
As an example, partition portals use a semaphore at each end to coordinate between the server and client tasks and these tasks have equal priorities. If the client semaphore times out because the server is hung, the client is helpless to correct the situation unless the RTOS has bumped up its priority.
More Pitfalls
This is not a comprehensive list of RTOS pitfalls. I would like to hear about other pitfalls you have encountered. The “simpler is better” credo is a fallacy when applied to RTOS selection. RTOS pitfalls are likely to be the cause of many problems that developers are having with RTOSs. Trouble is likely when an RTOS does not behave as reasonably expected.
Conclusion
RTOS size and performance are no longer dominant determinants for RTOS selection due to great advances in hardware. Now the main determinants are development cost, maintenance cost, and security. Simple RTOSs are riddled with pitfalls, thus making them not simple to use. The SMX RTOS protects against these pitfalls and the SecureSMX extension adds powerful security features to it. Both are available under a dual GPL/commercial license and full source code may be freely downloaded from www.github.com/Micro-Digital/SecureSMX.
References
1. “An Event-Friendly RTOS”, Ralph Moore, LinkedIn, August 2026.
2. “LSRs Improve Interrupt Handling”, Ralph Moore, LinkedIn, November 2025,
Notes
[1] An alternative solution to this problem is to automatically boost the priority of a timed-out task.
[2] This is where one or more medium priority tasks preempt the current object owner, keeping a high-priority task waiting indefinitely for the mutex.
[3] However, operation is still dependent upon event priorities and characteristics of the underlying RTOS.