About SMX® v4
See the smx datasheet and smx Special Features for a summary of the features that make smx unique versus other kernels.
A few heap issues have been fixed and improved: Automatic chunk merge has been fixed to avoid downward bin leakage due to chunk splitting. smx_HeapBinSort() has been fixed and improved to reduce sort times. smx_HeapManager() was added to main.c and is called by the Idle task to handle auto merge control and heap and bin scanning and fixing.
The heap has been greatly improved and now has comparable performance to dlmalloc and potentially greater performance when customized to specific embedded system requirements. In addition, fast large bin sorting during idle has been added and improvements have been made to debug support and self-healing. In the latter case, heap scanning has been simplified and heap bin scanning has been added. This heap, under the name "eheap" is being made available for free in source code form for non-commercial use in order to facilitate testing it in actual systems and experimentation.
The heap has been replaced with our new bin heap for fast allocations vs. the linear heaps typically implemented by RTOSs. Its structure is similar to that used in GPOSs, such as Linux, but it has been designed to fit the requirements of embedded systems. It works with small to large amounts of RAM and offers high performance, deterministic behavior, small code size, ruggedness, and strong debug support. It is intended to provide very high performance for object-oriented languages such as C++ and Java, as well as for other intensive heap applications. It provides the standard heap services malloc(), calloc(), realloc(), free(), and init(), and it has various services to monitor the heap and optimize its behavior, such as to control bin population. It is self-healing and does scanning and fixing automatically during idle time. It also has a special HCB type that can be used for allocations to aid in debugging.
The heap has 3 main regions: small bins, upper bins, and top bin. For blocks in the small bins, it offers nearly the same performance as a block pool. Allocations from the upper bins use a binary search that is nearly as fast and deterministic. The top bin is for large allocations. Bin configuration is easily tuned to suit the application.
Timers have been enhanced with new operations, such as duplicate, reset, and start from an absolute time. A pulse timer is a new type that has been added to allow generating pulses like hardware timers and can be used in place of them for relaxed timing requirements. A new smx service, smx_TimerSetPulse() permits converting a cyclic timer to a pulse timer and is then used to control its pulse width and period simultaneously. This permits easily implementing pulse-width modulation (PWM), pulse-period modulation (PPM), and frequency modulation (FM).
smx_SysPowerDown(sleep_mode) was added to provide a power-down framework, at the RTOS level. It is called from the idle task when there is no useful work to do. It in turn calls sb_PowerDown(sleep_mode), which does the actual power down process and time recording. This is hardware and application dependent and thus must be user-implemented.
When power is restored, control returns to the sb function, which must initialize the tick counter and return ticks lost to smx_SysPowerDown(), which then performs tick recovery. This process handles events which timed out during the power-off period in an efficient manner that preserves the proper order of timed events. Execution time for tick recovery is dependent upon the number of events that timed-out, not upon the number of ticks lost .
Error management reports timeout as an error, and the TCB.err field was changed to indicate the result of the last smx service: OK, timeout, or error type. This makes it easier to determine the proper action at the point of call.
smx_QueueClear() has been replaced with specific clears for event groups, event queues, message exchanges, mutexes, pipes, and semaphores. This provides a more intuitive API, and each clear service is able to do a more complete job.
FPU register autosave has been implemented for the Cortex-M4 and Cortex-M7. This has also been retrofitted into v4.2.1.
smx v4.2.1 is a minor release to fix and improve a few things. The block pool API changed slightly. More examples from the smx User's Guide were implemented in ESMX and corrected in the manual.
Free SMX Learning Kits were released for learning and experimental use.
smx v4.2 adds significant new kernel features and improvements to existing code for size, performance, and reliability. These are summarized below.
Exchange Messaging has been extended to support broadcasting and multicasting. Multicasting is supported by proxy messages, which can be used for distributed message assembly. These capabilities are unique to smx. Resource exchanges have been eliminated; new messages are obtained from smx block pools like smx blocks, except they are mated with MCBs. (In fact, they can be obtained from the same pools as smx blocks.) smx_MsgMake() permits making messages from any source. This leads to the concept of "message migration". The idea is that an input ISR obtains a block from a base pool, fills it, and passes it to an LSR. The LSR makes it into a message and sends it to an exchange. From there it can be passed from task to task via exchanges. The final task calls smx_MsgRel() and the data block and the MCB are put back where they belong. The final task need not know where the data block came from. This is a no-copy operation from start to finish. Output works inversely. This is a ground-breaking concept, unique to smx.
Block Pools have also been extensively improved. smx raw blocks were replaced with smxBase blocks. Getting and releasing base blocks is interrupt-safe and thus amenable to being used from ISRs. smx block pools are based on base block pools but offer more safety and flexibility, e.g. the API is task-safe. Similarly to smx messages, smx blocks can be made from anything (base block, static block, heap block, etc.) using smx_BlockMake().
Event Groups have been completely redesigned. Slots have been eliminated to simplify usage. Pre-clear and post-clear masks have been added. The former permit dealing reliably with modes and the latter permit both modes and flags. Modes (mutually exclusive flags) are not auto cleared; flags can be auto cleared. In addition, testing for AND/OR combinations has been added, which is unique to smx.
Semaphores have been completely redesigned. Previously, the smx counting semaphore was used for different purposes. Now, there are 6 types of semaphores, each explicitly designed for a particular use: Event (binary and multiple), Resource (binary and multiple), Threshold, and Gate. They introduce new concepts for semaphore usage.
Multi-Level Queues, except rq, have been replaced with priority or FIFO queues, as appropriate. This not only simplifies usage, but also ensures operations are done according to priority, as expected. Multi-level queues were designed for efficiency, but if the user didn't use them (and instead used a single-level queue), tasks and messages would be handled in FIFO order not priority order. These queues are typically small, so it was decided that search time is less of a concern than ease of use and expected operation.
One-Shot Tasks are now easier to use effectively. Stop calls were changed to stop the task in all cases, which simplifies use. They have been documented more clearly and completely in the smx User's Guide, and many good examples of using them have been peppered throughout. One-shot tasks offer significant RAM savings because they share stacks, and in many places their use is more logical than standard tasks.
Manuals have been substantially rewritten, particularly the smx User's Guide. Obsolete material has been replaced with accurate, detailed information on how to use smx effectively. Many new examples have been created.
Code size was reduced by about 40%, despite the increased functionality and the fact that smx was already efficiently implemented. This was achieved by sharing code by similar SSRs as much as possible, particularly in the suspend and stop versions of calls; by changing many macros to functions; and by making other similar changes after carefully studying the code. Sharing common subroutines is also safer since changes only need to be made in one place. Control block pools are now implemented with base pools, which resulted in further code simplification.
While implementing new features and streamlining existing code, significant low-level study and experimentation were done, including analyzing disassembly and testing performance while experimenting with details such as control block sizes and field sizes.
x86 code has been fully removed from smx and smxBase, significantly simplifying some files. x86 is where smx started almost 30 years ago and was the only architecture it supported for nearly a decade. Over many years, a great deal of energy was expended supporting this architecture.
While updating smx++ to v4.1, it was simplified and improved (more info).
smx is one of the few kernels that also offer a C++ API.
smx v4.1 represents a major step forward in simplifying smx, making it more user-friendly, and adding new features that will make your job easier. These are summarized below.
All versions of smx now use the system stack for ISRs, LSRs, the scheduler, and error handling. This significantly reduces RAM usage
since it is no longer necessary to increase each task's stack size to accommodate the worst case stack usage of nested ISRs and LSRs. Now task stacks can be sized for just the task's needs. This is especially important because we advocate designing your application to use many tasks.
In addition to RAM savings, this improves reliability, since changes to ISRs and LSRs that cause increased stack usage do not require adjusting task stack sizes. Also, stack usage during error handling is something one might not account for in sizing stacks, so running that code on the system stack could avoid a run-time failure due to an error. smxAware shows system stack usage in its task stack usage displays.
(Note that ARM-M (Cortex-M) versions have always had a system stack which was used for ISRs, LSRs, and the scheduler, because it is inherent in the architecture.)
Implementing the system stack has required significant changes to the scheduler, including addition of the pre-scheduler. It improves efficiency by bypassing the task scheduler in cases where the sched flag indicates a reschedule might be necessary, but ct remains the top task.
Shared stack handling has been improved, in the case where a stack is unavailable at the time of task dispatch. Now if a stack is unavailable because it needs to be scanned, the scheduler bumps the idle task up to do (or complete) a stack scan, then proceeds to dispatch the task. Higher priority tasks that become ready will preempt this operation.
We expect these to be the last changes to the scheduler for the foreseeable future. As in v4.0.1, a single configuration option will restore the previous scheduler and associated sections of code, if a problem with it is suspected. This code will be removed in v4.2.
The event buffer is used by smxAware to display graphical timelines showing how the system is running. The macros and functions used to add records to the event buffer during execution appear frequently in the code, so they have been streamlined to make them as efficient as possible. Part of this was creating additional versions of macros to store fewer parameters. Previously the record size was fixed; now it is variable. This allows storing all parameters of SSRs. New user macros were added to store from 0 to 6 values of interest in the application. Having variable-sized records makes it possible to store more records in a given amount of RAM. In a typical sample, we observed a density increase of 34%.
In addition, since SSRs are called frequently, we implemented a means to control which
are logged, to avoid wasted records and time. Each can be assigned to 1 of 8 SSR groups, and a flag variable allows controlling which groups are logged. Group assignment is done statically, but selection of which to log can be adjusted for the current debug session using checkboxes in smxAware.
The Protosystem is the foundation for application development, so minimizing complexity benefits the application and helps users get a quick start. Several files were moved into the smx library, and files that remain have been simplified, particularly main.c. This makes it easier for new users to see how to build their application upon the Protosystem, and it improves the readability of the code for all users.
The number of (required) system tasks has been reduced to one: the idle task. Previously there were tasks for handling timeouts, profiling, stack scanning, and exit. Now all are handled by idle or LSRs. This gives significant RAM savings for those stacks and TCBs, which is important for minimal RAM systems.
All smx services that wait have a parameter to specify a timeout, in ticks, when the operation will abort. In previous versions, the true accuracy of the timeout depended upon how often the timeout task ran (a configuration setting). To minimize overhead, this was typically set to multiple ticks (e.g. 10). The original purpose for timeouts was to be a safeguard to catch error conditions when a task waited much longer than reasonable. However, we found users were expecting tick accuracy, so we greatly improved the efficiency of timeout handling, and we made it an LSR to avoid task switching overhead. Now timeouts are tick-accurate and efficient.
Precise Time Measurement
API functions are provided to do precise time measurement of short sections of code (less than 1 tick), accurate to the input clock of the tick timer. Some places in the smx code are instrumented, such as the scheduler, and more will be added over time.
Previously, smx offered only coarse profiling, which showed %busy, %overhead, and %idle. Now it has task profiling that is accurate to a tick counter timer clock, which is typically one to several instruction clocks. Each task has a run time count field in its TCB, which accumulates incremental counts when the task runs. There are also run time counts for ISRs and LSRs. (ISR count is the total for all ISRs, and likewise for LSRs.) Remaining counts are attributed to overhead. All counters are cleared at the start of a profile frame and transferred to the run time count array, where they overwrite the oldest column. The profile frame is one second, by default, but it can be changed to any number of ticks. Profiling uses an LSR instead of a task.
The profiling information is displayed graphically and in tabular form in smxAware. The coarse profiling data is still displayed on the terminal (if present), but now it is calculated using the precise profiling data.
There is now a single error handler rather than individual error service
routines. This eliminates the error jump table and generally
simplifies the error manager. Also, error reporting to the terminal is now
done through new console output routines that enqueue messages
rather than sending them directly to the UART. When using a polled
UART driver, the old way could cause long delays at critical times.
This was a main motivation for creating the improved message output
functions (see below). Other minor improvements were made, such as
shortening error strings to save flash.
Message Output to Console
The smx kernel and middleware output error, warning, and status messages as appropriate
for the debug level selected for each. These are very useful for
diagnosing run-time problems. Previously, they relied on console output functions
that directly wrote to the UART, which is a polled driver in many BSPs.
This introduced long delays into operations, and could cause unbounded priority
inversions. Even though diagnostic message output is used primarily during debug, we decided that these problems needed to be fixed.
console output functions are de-coupled from the UART. Fixed messages
(in ROM) are output through the output message queue (OMQ), and variable
messages (in RAM) are output through the output message buffer (OMB).
A display function is called from the idle task (and can be called from
other low priority tasks) to output messages that have accumulated.
The OMQ has only pointers to messages, so this saves time and RAM for
displaying them, so on limited RAM systems, it is preferrable to use
fixed messages. The OMB is a ring buffer into which messages are copied,
requiring a little more time and more RAM to hold the messages, but is
essential for printing messages that vary, such as values read from
peripheral registers, etc. Since buffer overflow can occur at a high
debug level, configuration options are provided to control which
facilities are available, and even to allow re-coupling to the UART
in cases where it is necessary to avoid losing messages.
Stack Overflow Handling
More sophisticated handling has been added for stack overflow.
If the scheduler is about to suspend a task, but the stack pointer has
encroached into the Register Save Area (RSA) (which is "above" the stack), the context cannot be saved
because it would overwrite the top elements of the stack, so the task is
restarted. If the stack has actually overflowed above the RSA, the
application is exited, to limit system damage. (These behaviors can
be modified by altering the error manager. Also note that during
development pads are typically used to prevent overflow from going
outside the stack block, and by release, stacks should have been
tuned to sufficient sizes based upon stack high water marks.)
SDAR and ADAR
DAR functions have been added to smxBase, and smx has been modified to use them.
SDAR was created for smx system objects, and ADAR is primarily for
application objects. They are allocated in separate sections and can
be easily located separately for performance and safety. SDAR holds
smx control blocks, ready queue, LSR queue, and error buffer, and it is small so it can be located
in internal SRAM on most processors, for speed. The heap, stack pool, and event buffer are located from ADAR, which would be typically put
into external SDRAM, if available. Otherwise, both are
located in internal SRAM. The user can easily change their location
in the linker command file. The separation of smx objects from application objects is a step in the direction of MPU or MMU protection.
A Memory display was added to show a summary of smx RAM usage. Graphical bars indicate usage of heap, stacks, SDAR, ADAR, LSR queue, and each type of control block. For heap and LSR queue displays, the high water mark is indicated by a thin line past the end of the bar and a numeric value. The Profile display was replaced with one that shows the new smx profiling data graphically and in tabular form. The graphical display shows bars for each task, all ISRs, and all LSRs. The user can step forward and backward through the frames. The first display shows the average of all frames. SSR filtering has been added for SSR groups (see Event Buffer above). See the smxAware datasheet for details and for information about its other features. smxAware continues to be included with smx at no additional cost, to aid users in developing smx applications.
- The tick timer is used for event buffer timestamps, precise profiling,
time measurement, and polled delays, in addition to its main use to
generate the smx tick. This means even with all these timing-related
features, smx still uses only one hardware timer, leaving the rest
for application use.
- app and smx config files have been compacted and simplified by moving descriptions of
settings out to SMX Quick Start (Configuration section).
- Task state field was added to the TCB. Application code can check it,
and it can be viewed in the debugger watch window. (smxAware has always
displayed it, but a bit of code was required to determine it.)
- cbtype, state, and errno were made enums for debugger display of names rather than numbers.
- smx internal macros in xsmx.h were simplified and their number was
reduced. As part of this, exits in SSR code were made explicit, not hidden in condition test macros.
- Several small improvements were made to the BSPs. One was to modify
sb_DelayUsec() to use the tick timer counter rather than a dummy loop,
which was highly inaccurate. Another was to create term.c
in each BSP, a simple file to replace crt.c and kbd.c that were shared
by many BSPs and had many conditional checks.
- Prefixes were added to a few more names for consistency and to avoid name conflicts.
- smxSim (Win32) code was removed to reduce clutter. smxSim has been discontinued.
The manuals have been carefully updated for all these changes. Additional improvements
have been made to some chapters of the smx User's Guide, and the glossary
in the smx Reference Manual has been rewritten.
Despite the significant gains made so far in v4, we are just getting started on the
development we have planned for the smx kernel. Much of the work has been
focused on infrastructure, the scheduler, and low-level details. These
have been extensive and time-consuming changes to make.
We are now starting on smx v4.2, which will focus on the following:
- Adding new, powerful features.
- Enhancing Safey, Security, and Reliability (SS&R) features.
- Power management.
- Adding performance enhancements such as multicore support.
- Continuing to simplify and make smx easier to use.
Hence, you can expect smx to keep growing with your project needs.
The scheduler was rewritten so it is faster, cleaner, and more easily portable. This is a project that was done carefully over many months and thoroughly tested. Still, there is a single configuration option to restore the old scheduler and associated sections of code if a problem with it is suspected.
ObjectAction naming is now used in the smx kernel and smxBSP APIs, such as smx_TaskCreate() and smx_TaskStart(). This groups related functions together naturally, making it easy to see what operations are possible for each type of object. smx Kernel API.
Prefixes such as smx_ and sb_ were added to these APIs, as were already present in many of our middleware products. This gives them their own namespace to avoid conflicts and also helps with code readability since it is immediately clear which product defines each symbol. The underscore helps to visually isolate the name from the prefix, and it is helpful in searches.
We have eliminated short names in preprocessor defines since they are more likely to clash with names in your code and third-party libraries, and errant preprocessor substitutions cause strange compile-time errors that are sometimes difficult to diagnose.
smx Kernel Improvements
Pipes were redone. Previously they supported only byte-sized data. Now they support packets of any size from 1 to 255 bytes, making it possible to not only send fundamental data types but also your own custom data structures. Pipes are equivalent to message queues in some RTOSes. (smx continues to provide more sophisticated messaging capabilities using exchanges.)
Scheduler locking has been changed to add a nesting count, and a flag was added to smx_TaskCreate() to specify whether to start the task locked or unlocked. Forgetting to unlock tasks inhibits preemptive scheduling, which has been the most
common support issue in v3. Now you must explicitly specify it when creating each task.
The heap now has a high water mark to show maximum heap usage since startup, making it easy to configure heap size. Also, a backward link was added to the HCB so blocks can be merged with the preceding block when freed rather than when scanning the heap during alloc or check functions.
The C++ features that had been used in some API functions have been removed. Some compilers are offered in lower-cost versions without C++ support. This was already handled in v3, but with conditionals in the code to call these functions one way if C++ and a different way for C.
Now all have a C interface in either case.
Future v4.x will add new features that have long been planned. We needed to release at this point to gain the benefits of the new codebase with all of its other improvements.
Centralized Porting Layer
We recognize that many projects have a need for the excellent SMX middleware, even though they have already chosen a different RTOS or no RTOS.
Previously, each middleware product had its own porting layer to handle differences in target hardware, OS, and tools. This meant it was necessary to port each product individually. In v4, all share a central porting layer, so that once it is implemented, all of our middleware products are supported.
(Note that when purchasing the smx kernel for a supported target board and tools, no porting is required.)
First we worked out a common porting layer. Then we implemented it for more than 10 common RTOSes. This revealed significant differences among them requiring changes to the porting layer to support them. (All RTOSes are not equal!) We went through several iterations of refinements to the porting layer to achieve this. Since it is a representative set of RTOSes, it is likely that the porting layer will support your custom RTOS or other RTOS if not one of these. Also, this means the work is already done for all of these RTOSes, and we will continue to port to others as well. In this case, our middleware is nearly a drop-in solution for your existing project!
Note that there are still a few product-specific things to port in each, as you would expect.
This is a new module that is the foundation of all SMX products. It contains common definitions, macros, and functions so it is not duplicated in our different products. This reduces ROM space by eliminating redundant code; it improves reliability since there are not multiple versions of these that are possibly implemented differently; and it improves consistency and readability since familiarity with a definition in one product carries to others — there are fewer definitions, macros, and functions to be aware of.
smxBase also contains the porting layer just described and BSP code.
Compatibility with v3
Because it is common for our customers to order additional modules after their initial release, we made it possible to ship middleware from the v4 codebase to drop into existing v3 releases with few changes required.
This will shorten the time we need to maintain two codebases (which is error-prone). New orders of middleware for existing projects
will come from the v4 codebase, so the v3 codebase is only used to ship updates of products the customer already has (so they do not need to be ported again). For the v4 middleware added to an existing project, the porting will be done in smxBase, so if yet another product is purchased, the common porting will have been already done, giving a v4 benefit to existing customers using v3 releases.
For those purchasing SMX middleware for another RTOS or standalone, we are now able to supply BSP code used by smx, reducing porting work further.
A great deal of cleanup has been done to the codebase.
Obsolete products have been removed, as well as conditionals for obsolete tools. Segmented x86 support has been removed, and with it arcane keywords and terminology, such as near and far. The smx kernel now supports only 32-bit flat architectures. (Middleware continues to be portable to any.) Historical quirks have been removed. Assembly support has been minimized except what is needed to write ISRs, and all definitions have been merged into a single .inc file for each architecture. This is less for us to maintain and less to distract you. Some files have been renamed for better consistency among products. smx kernel functions have been grouped into related files, since most modern linkers can deadstrip individual functions from object files.
All of these mean a cleaner and simpler release to work in.
The directory structure has mostly remained the same, since it continues to work well. However, a few changes were made. All Protosystems were merged into one, now called APP. This eliminates a lot of duplicate code for us to maintain and ensures uniformity among different versions of smx. BSP code has been moved out of the Protosystem into its own directory which is more appropriate so libraries can access it, and since it is now supplied with standalone releases.
All product manuals and BSP notes have been updated to v4. This was a large effort with all the name changes. The smx Reference Manual is easier to use due to the API renaming. Some simplification has been done to it and other manuals. Like the code, obsolete sections have been removed. SMX documentation continues to be among the best in the industry.
In summary, we have made significant improvements to the infrastructure of the SMX codebase as well as specific improvements to the smx kernel. We have both improved the integration among our own products and made it easier to use them ala carte.
Register now for access to the manuals and to be contacted by a product expert. Registration also enables us to mail you printed literature and is necessary to qualify for a free evaluation kit.
smx is protected by US Patents 9,336,072, 9,424,027, and one or more patents pending.
Back to v4 Home Page