Abstract
This paper presents a design and implementation of a Mixed-Criticality System (MCS) extended from C/OS III. It is based on a MCS model that uses an adaptive reservation mechanism to cope with the uncertainties in task execution times and to increase the resource utilization in MCS. The implementation takes advantage of the tasks’ recent execution records to predict their required computational resource in the near future and adjusts their reserved budget according to their criticality levels. The designed system focuses on soft real-time tasks. An overrun tolerance algorithm is used to limit the deadline miss ratios between a rise to the task’s actual consumption and the change to the amount of reservation. More than two criticality levels can be handled without introducing obvious additional overhead at each added level. The case study evaluation demonstrates that the reserved resource for each task is always close to its actual consumption; the tasks’ deadline misses are bounded by the different requirements specified by the criticality levels; during overload conditions, high-criticality tasks are guaranteed to have sufficient resource reservation. Although there is still room for improvement if it comes to processing overhead, this research brings some inspirations in both modelling and implementation aspects of MCS.
1. Introduction
Embedded systems are in high numbers around us today. They range from smart watches to flight control systems. Among different types of embedded systems, some of them do not permit for malfunctions as they could harm human lives or the environment. Examples of these so called safety-critical systems can be found in medical devices, aviation, and transport industry. In the related industry standards, the safety-critical functions are classified into multiple criticality or safety levels in relation to the hazard they generate by their failure (e.g., DO-178B, IEC 61508). The concept of a Mixed-Criticality System (MCS) is therefore defined and aims at a software platform that allow functions with different criticality levels to coexist on a shared hardware platform [1].
The goal of a MCS is to only provide high levels of reliability to the sole functions requiring it. A fair amount of research has been devoted to this area. One comprehensive overview is addressed in [2]. In general, three fundamental research issues are considered important: the resource isolation, the resource distribution and sharing among the isolated parts, and the balance between isolation and sharing. A wide range of resource kinds are involved in these topics, including power and energy [3–8], communication bus and memory access [9–20], and processor usage [21–42] in both the single- and multiprocessors platforms.
The research in this paper is limited to the temporal isolation and sharing problem on a single processor platform. It has been a major focus in MCS research. The research history in this area can be traced back to Vestal’s paper [1]. It was followed by a series of publications that covered both processor resource distribution schemes with fixed priority (FP) [24, 26–29, 31–39] and earliest deadline first (EDF) [21–23, 25, 40–42] based scheduling in MCSs.
Not all of these studies use the same system model but their defined models share some common features. In general, a task is characterized by multiple WCET values with different levels of confidence. In order to avoid severe underutilization of the resource, every task in the system executes with a budget that equals its low-confidence WCET at the beginning. Only when a given task with a high-criticality overruns does the system raise the reservation of high-criticality tasks to their high-confidence WCETs and triggers operations such as discarding low-criticality tasks, extending their deadlines, lowering their priorities, or shrinking their execution times.
These models have certain limitations. The first issue is related to WCET assessment. For time-randomised hardware, defining different WCET budget with different confidence levels can be done with probabilistic WCET (pWCET) [44, 45]. But for time-deterministic hardware the existing WCET estimation techniques (both static and measurement-based techniques) can not associate the confidence levels with the WCET estimates in an easy way [2]. Although recent studies [46, 47] present that the pWCET can also be used for time-deterministic hardware, the technique is immature and limited to very simple systems [48]. Even for the time-randomised platform which has the best fit for pWCET techniques, there is a lack of trustworthiness of the provided results due to their strong dependence on the assumptions made in the measurement, particular hardware features, or a number of other aspects [48]. Moreover, if the target task’s execution time varies a lot from one instance to another, which is highly possible due to the impact of unpredictable environment and high complexity of the system, using offline WCET estimates as a task’s bandwidth budget can cause low resource usage [36, 49–51].
The second issue is about a pessimistic assumption that is made in the general MCS models. In [52], Burns classifies the event that a task executes longer than its expected WCET as a fault. As defined in these models, all high-criticality tasks’ executions will consume up to high-confidence WCET values after one high-criticality task encounters such a fault. As pointed out by Easwaran and Shin [53], this is highly unlikely to happen in practice if these tasks are independent of each other. Therefore, the paralysation or service degradation of low-criticality tasks based on this assumption is sometimes overdone.
The third issue is that the traditional models do not take into account the fact that at certain criticality level, sometimes even the highest ones, the tasks can tolerate some deadline misses. In a dual-criticality system, for example, a high-criticality task should never miss a deadline according to the classical assumption. However, as stated in [54], the industry standards actually do not have specific definitions for the relationship between deadline misses and critical failures. It conceivable that a small number of deadline misses has little or no effect on safety. If this is indeed the case, then avoiding all the deadline misses is certainly too pessimistic and causes a waste of resource.
In this paper, we design and implement a software framework extended from C/OS III. It introduces adaptive reservation concept into the area of MCSs. Considering the issues mentioned, the designed system supports a MCS model different from the existing ones. It allows tasks to have deadline misses within certain limits but still based on the conjecture that the strictness of the timing constraints lines up with the criticality levels of the tasks. Instead of reserving resource according to several offline WCET estimates, a prediction mechanism is adopted to calculate the required computation resource for each task individually during runtime. Whether the tasks’ requirements will be fulfilled is determined by the admission control according to their criticality levels and the system’s available resource. In the overload condition, tasks are sacrificed in a reverse order of criticality levels to release resource and maintain a schedulable system. Until the prediction mechanism reacts to a task’s resource consumption change, a slack (reserved but unused processor bandwidth) distribution algorithm is adopted to handle the budget overrun events and lower the risk of deadline miss.
The organization of this paper is as follows. Section 2 provides an overview of related work; Section 3 introduces notions and the MCS model adopted in this paper; Sections 4 and 5 describe all the services provided by the designed system and the basic structure used for task isolation; Sections 6 to 10 explain the implementation details of these services; Section 11 presents the case study evaluation results for this framework; and Section 12 presents the final conclusions and future work.
2. Related Work
In this section we first review the adaptive resource reservation and slack allocation algorithms in traditional real-time systems, their potential in resolving the resource efficiency, and overrun tolerance issues in MCS. Then we show the usage of slack distribution algorithms in the context of MCS. At last, we review one existing solution towards replacing the greedy algorithms that simultaneously increase the budget for all high-criticality tasks.
In traditional real-time systems, there is a vast literature using the online admission control and adaptive reservation to deal with the uncertainty of execution time and to avoid the disadvantage of offline WCET assessment. They compute and adjust the budget of each task according to its actual demand. Typical implementation employs feedback and prediction mechanisms together with online schedulability testing to measure and predict the resource reservations during runtime [55–58]. This kind of technique has been barely considered in the area of MCS. One possible reason would be that there is always delay between the change of demand and the system’s reaction towards the demand. During the delay, overrun events can happen which in turn cause deadline violations. Since critical tasks have high standard for deadline satisfaction, additional endeavor needs to be made for introducing this technique into MCS.
For handling the transient overrun faults as well as increasing resource utilization in reservation based scheduling, slack allocation algorithms have been studied [59–66]. These researches present that, by redistributing the reserved but unused resource, most overrun events within each reservation unit can be well handled without violating the timing constraint. In our approach in particular, we adopt the algorithm presented in [66] which claims to have better performance than IRIS (Idle-time Reclaiming Improved Server) [65], BEBS (Best-Effort Bandwidth Server) [60], GRUB (Greedy Reclamation of Unused Bandwidth) [61], RBED [62], CBS (Constant Bandwidth Server) [59], and CASH (CApacity ScHaring) [66] in bounding deadline misses.
To take advantage of both adaptive reservation schemes and slack allocation algorithms, some studies combine these two techniques together, as is the case in our paper. In [67–70], the authors present work that dynamically adapts the reservation parameters on top of CBS [59]. In [71, 72], Palopoli et al. integrate a feedback control mechanism together with ShRUB (Shared Reclamation of Unused Bandwidth), a variant of GRUB [61]. This kind of approach has certain benefit that it can avoid resource overallocation with a budget adapted to the needs and tolerate the budget overrun until the system reacts to the measurements. Unfortunately, since these mentioned works do not take into account the notion of criticality levels, they are not directly applicable for MCS.
Targeting for MCS, slack distribution algorithms have already been adopted in multiple papers [25, 35, 36, 42, 51]. The basic idea of them is to let the low-criticality tasks execute on the slack of the high-criticality tasks. In [35, 36], De Niz et al. propose zero slack rate-monotonic scheduling and later combine it with additional support for Quality-of-Service (QoS) Resource Allocation Model. In [25], Park and Kim introduce CBEDF (Criticality Based EDF) on top of EDF scheduling. In [42], Lipari and Buttazzo reuse the policy of GRUB [61] in MCS. In [51], Groesbrink et al. propose an algorithm for QoS-aware system where slack is adaptively allocated to tasks that can benefit from additional resource.
In order to replace the greedy algorithms that simultaneously increase the budget for all high-criticality tasks, one approach is proposed by Gu et al. [73]. They partition tasks into separate components. Then for tasks inside each component, they define the fault tolerance limit by a number of violations of WCETs. Only when the WCET violations within a component exceed the component’s expected limit are the tasks in other components affected. Our work is different from theirs in that it monitors each task individually and does not rely on the correctness of the offline WCET estimation.
3. Model
In a MCS, we define an ordered set of criticality levels: . A smaller represents a lower criticality. Only periodic tasks are considered in this paper. Each task is characterized by the criticality level, a relative deadline , and a period , where .
The system predicts the requirement of a task by a prediction mechanism during runtime and then grants the task a budget for each release period . Therefore, the processor usage for each task is . The major differences between our model and general MCS models are the following: the reserved resource does not equal the offline WCET estimate and it is modified adaptively and individually for each task during runtime.
A set of tasks is said to be schedulable under EDF scheduling if . Otherwise, the tasks will be suspended in a reverse order of criticality levels by admission control until the scheduling is feasible.
The overall adaptive reservation process for a task is shown in Figure 1. Upon the new job arrival of , the system first checks if has been suspended as a consequence of a processor overload condition. If it is not the case, then a budget is reserved for it. During the job execution, if the actual consumption does not equal the reserved budget , the fault tolerance service will be activated to deal with the overrun problem or redistribute the reserved but unused resource. When finishes the job execution in the current release period, the system determines whether should be replaced by a new estimation to apply from its next release period. The decision is made according to ’s execution records, the acceptable deadline miss ratio for a task at criticality , and the estimation interval defined by the user. This estimation process also takes into account the acceptable deadline miss ratio for a task at criticality . With the new estimated value of , the system does an overall calculation to predict whether the processor will be overloaded or if enough resource will be released for suspended tasks if any. According to these predictive calculations, the system then suspends or reactivates tasks taking into account their criticality levels. Such operations preferentially grant the required budget to a higher criticality level.
The detailed design and implementation are described in the following sections.
4. System Architecture
This section briefly describes all the services provided by the described system and their connections with each other.
Required by the adaptive reservation process (Figure 1), the system proposed provides the basic services that support reservation based scheduling and the extended services that are implemented above the basic ones and support adaptive modification to the reserved budget of each task. Figure 2 shows all the services included in the design.
The basic services refer to task isolation, processor allocation and resource monitoring, and general support for a scheduler. The extended services refer to the fault tolerance, resource adaptation, and overload handling.
Task isolation structure is used to ensure that each task only has access to the processor bandwidth assigned to it. This is the central idea of a reservation based system and its purpose is to minimize the interferences among tasks that are sharing the same hardware platform. As a result of task isolation, each task belongs to a reservation unit, which is called server in this design.
The processor allocation and monitoring service is responsible for distributing a specific amount of resources to a certain task (or server) and monitoring the actual consumption of tasks’ executions.
The scheduler is what is used in scheduling the servers. Instead of scheduling the tasks directly, a reservation based system has global scheduler that can only see the servers. Besides the algorithm design according to the selected scheduling policy (e.g., EDF, FP), there are some general supports needed for such a scheduler: the definition for server queue data structure, the design of queue management mechanism, the scheduling point selection, and the interaction with the local scheduler of the underlying OS (C/OS in this case).
The fault tolerance service is designed to tolerate the transient overrun faults before any change to the budgets assignment. It takes into account that servers other than the one being overrun might have spare budget. Or the overloaded server itself will have spare budget in the following execution. This kind of spare resource can be used to resolve the overrun event which avoids a waste of resource on the one hand and prevents performance lost on the other hand. Besides that, this service also serves as the foundation for the resource adaptation service. It loosens the requirement for the budget values reserved for each server, meaning that as long as the difference between the actual execution time and the reserved budget is within a range, it is bearable by the server. As a result, less conservative and more flexible budget estimations are allowed.
The resource adaptation service is used for predicting and adjusting servers’ budgets during runtime. The fault tolerance service can handle occasionally overrun events and reduces the wasted resource to a certain extent. This is under the assumption that the reserved budget is not too much lower than what is required in most real executions. If this is not the case, then there would be serious problems. On the one hand, a budget value which is being exceeded frequently will greatly add to the fault handling cost. If these overrun events are happening consecutively, there is even an accumulative difficulty in handling them which will further increase the risk of deadline miss. On the other hand, an overconservative budget value can increase the resource lost, with or without the fault tolerance mechanism. Even worse, overallocation to one or several servers sometimes gives a false impression that the system is overloaded. This is the major drawback when using WCETs as reserved budget values as in traditional MCS implementation. This stated condition motivates us to design this resource adaptation service that dynamically changes the budget value for every server via feedback control and online estimation.
The system overload handling service is designed to be activated when the system cannot provide required budget for all the servers. This service supports the schedulability test and budget reconfiguration with respect to the criticality levels. The objective is to react quickly upon the overload event and redistribute the resource to primarily consider the demands of high-criticality tasks. The schedulability test algorithms associated with this service highly depend on the correction of the budget values provided by the resource adaptation service.
5. Server Structure
Figure 3 shows the basic structure of the proposed system with server support. The servers are scheduled with the global scheduler which refers to “scheduler” in Figure 2. In the following paragraphs, the scheduler for the servers will be denoted as global scheduler and the C/OS III’s scheduler will be denoted as local scheduler.
Two types of servers are implemented in this paper: the periodic server and the idle server. The periodic servers receive processor bandwidth from the resource allocator periodically. The idle server only starts running when no other server requires the processor resource. C/OS III has an idle task which runs whenever there are no other tasks ready. In this design, both these two kinds of servers contain idle task, but the idle server contains only the idle task.
Each task is assigned a periodic server which is identified by a resource interface . This interface informs the global scheduler that the server houses a task with criticality and its execution requirement is in every period .
A server has three states as shown in Figure 4. It indicates some basic movements from one state to another. A periodic server is put into the ready state by calling the server-creation function. The global scheduler is responsible for selecting the server to be put into running state. The selected server then will be switched in upon system start or server switch. Any server that finishes the task’s execution will be put into pending state. When a server’s remaining period reaches zero, it is put back into the ready state. If the global scheduler then detects that the newly released server has become the most important (depends on EDF scheduling policy), it will preempt the currently running server. In the meantime, the currently running server will stay in the ready state and wait until it becomes the most important again. The servers can be deleted by calling the server delete function.
6. Resource Monitoring and Allocator
The resource monitoring of this framework can be divided into three parts. The first is the heart beat resource monitor (HBRM), which executes upon every system clock tick. Its purpose is to keep track of the remaining budget of the active server as well as the release time of each server’s new period. When a system clock tick event occurs, it will be triggered and will preempt the currently running task of the active server. A server has a pair of counters indicating the remaining period (relative deadline) and budget. Their initial values equal . The time unit for indicating the values is the system clock tick. Every time the HBRM is triggered by a system clock tick event, is decreased by one. Once reaches zero, server will be released, while when the HBRM detects that is in the running state, will be decreased by one. In the meantime, the queue to which this server belongs will be updated depending on global scheduling policy, fault tolerance policy, and resource adaptation policy. The procedure is shown in Algorithm 1.
| ||||||||||||||||
Figure 5 shows an example of a timing diagram of a periodic server. The parts with dark background show the work done by HBRM. More detailed description about the changes of will be introduced later in the following sections.
The second part is the Server Idle Detection (AID). It is used by us to detect ’s completion. From the moment the idle task starts running, the system will know that the current task has finished its execution. Then ’s remaining budget (slack), if greater than one, is available for redistribution among other servers. The redistribution policy and its implementation is introduced in Section 8.
The third part is Per-server Budget Usage Recorder (PBUR). In order to get the budget usage in each execution of , there are three possible ways:(1)Using the remaining budget of . Once the idle task starts running, if the current server is not the idle server, it indicates that has finished its executing. Then the value of is the budget actually used by .(2)Similar to HBRM. When is the currently running server, is incremented upon every system clock tick. Once the idle task starts running, is saved as the budget actually used by .(3)Using the hardware timer to mark the time that starts running, the time that is preempted for the time during the current execution, the time that resumes after the preemption, and the time that completes its execution. Assume that in one instance of periodic server it is preempted times. The actually used budget equals .
Our design chooses the third approach. The first has the advantage of limited memory reading times. However, it will be very complex to provide a budget reclamation and borrowing interface for the fault tolerance service, because the remaining budget is influenced not only by but also by the reclamation and borrowing slacks. The second is a straightforward approach, but changing the variable’s value after every clock tick event causes more overhead than the other two approaches. In the implementation of the third solution, a separate hardware timer is used other than the one for the system clock tick. The resolution (42 ns) is set to be higher than the system clock tick. Thus, it prevents a loss of accuracy during statistical calculation.
7. Scheduler Support
There are four scheduling points for the global scheduler: when a server is created; when a server in the ready state is deleted; after HBRM’s execution; and when the idle task starts running and the current server is not the idle server. The pseudocode in Algorithm 2 shows the overall scheduling process. It includes two steps: selection of the next running server and selection of the next running task. The function selects the running server . The detailed procedure is shown in Algorithm 3. represents the original scheduler of C/OS III. It is called every time after and decides which one among and should run.
| ||||||||
| ||||||||||||||||||||||||||||||
Algorithm 3 shows the global scheduler’s behaviour in EDF scheduling (with no support for fault tolerance service). It always selects the head item in the to be the next running server. If the head item in the exists and is not the currently running server, a server switch will be implemented. The local scheduler then switches to the corresponding server’s . as defined in C/OS III consists of a bitmap containing the priority levels that are ready and a table containing pointers to all the tasks ready.
Doubly linked queues (, , etc.) are employed for global scheduling. Each queue is a data structure of . It consists of one pointer to the first server in the queue. Correspondingly, a server’s basic data structure has three fields as shown in the list below. Server_TCB PrevPtr; NextPtr; ServerListPtr; ... ;
is a pointer to a data entry of the queue to which the server belongs. and are used for doubly linked queues. The last server in a queue points to .
Figure 6 presents an example of the insertion of a newly released server into , where EDF is the global scheduling policy. A pair of integers, and , need to be decremented upon system clock tick interrupt.
8. Fault Tolerance
The fault tolerance service relies on the resource monitor HBRM and AID. The purpose is to handle the situation where the task’s actual execution time exceeds the assigned budget of the server to which it belongs. The implementation of this service employs the BACKSLASH presented in [66] whose main principles are summarised as follows:(1)Allocate slack as early as possible.(2)Allocate slack to the one with the earliest original deadline.(3)Allow borrowing against a server’s own future resource reservations to complete the current execution. The server’s priority after borrowing is changed to the one from which the resources are borrowed.(4)Retroactively allocate slack to servers that have borrowed from their current budget during their last release period.
To implement the principles, other than and , two more queues are used: the server borrowing queue in ready state () and the server borrowing queue in pending state ().
When the HBRM detects that the budget of the running server has been exhausted () while ’s execution remains unfinished (as the second execution period of indicated in Figure 5), will be put into the . Meanwhile, borrows from its own future budget with an extended deadline (). Now, server simultaneously exists in two queues and . is sorted with the servers’ real deadlines and is sorted with extended deadlines. A different pair of pointers, and , are used in as shown in the server’s data structure below. Server_TCB ... BrwPrevPtr; BrwNextPtr; ... ;
When eventually finishes one execution, is put into the in order of real deadlines. The queue arrangements are done after the AID by the idle task and a schedule point is followed. In a different scenario that is detected idle without borrowing, its remaining budget will be distributed to other servers as slack. The slack receiver is selected from , , and . The overall process is expressed by Algorithm 4.
| ||||||||||||||||||||||
The slack donation is processed by Algorithm 5. The head server in is the first one that has its remaining budget increased. Depending on the value of , ’s remaining budget can be increased by up to the processor capacity that it consumed beyond during its last run. At the same time, is decreased by the same amount. Then, if is still greater than zero, the next available server in will increase its remaining budget. Until is empty, the slack goes to the server in the ready queue that has the shortest original deadline. After the slack donation process, the head server in the starts running.
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Compared with the example of the plain EDF described in Section 7, the ready queue management method is modified to be used together with this fault tolerance service, because the exact time at which a server in the starts to run can no longer be predicted. After the modification, all the servers’ remaining periods in the ready queue have to be decreased upon system clock tick. Figure 7 presents an insertion of a newly released server into the with the ready queue management method which adapts to this fault tolerance policy.
9. Resource Adaptation
The resource adaptation service is designed to adaptively change the reservation for each server during runtime. It relies on the whole of the support from the resource monitor HBRM, AID, and PBUR. The statistical tool adopted here is Chebyshev’s inequality which is providing a simple implementation possibility needed in our limited embedded environment.
Based on Chebyshev’s inequality, for a positive number , a random variable with the standard deviation , the bound follows formula (1). represents the possibility that ’s value is not inside the bound. Since the distribution of has equal lower and upper tails (Figure 8), can be estimated by (2). The budget’s upper bound here is then estimated by (3):
The statistical calculation resource is the budget actually used by successive past executions: , where donates the execution. We use (3) to estimate the future budget (i.e., ) and reassign the result to each server as the new reservation budget. The mean and standard deviation are calculated by
The configurable variables from the user side are and . A smaller can guarantee a lower overrun rate as well as a higher preassigned budget. For different , there is a corresponding . Normally they should meet the condition that when , then .
With regard to the choice of the value of , on the one hand, it should be a relatively big number (≥30) to have a better statistical precision. The execution frequency of the statistical recalculation for is , which can also be lowered by increasing . On the other hand, since the point at which ’s computation requirement dramatically changes is not predictable, the longer the budget reestimation calculation interval, the higher the chance that a high overrun rate will occur between two reestimation calculations. Examples are shown in Figure 9(a). With low frequency, when a server ’s budget requirement increases to a higher value only a few release periods after ’s execution, the earliest time for it to receive a higher budget as a reaction to the increase of actual consumption is almost release periods after. Therefore, from ’s to execution, might suffer from a high overrun rate. With high frequency, the duration of the overrun situation will obviously be shortened.
(a) Budget reestimation calculation on fixed intervals: The red arrows indicate the earliest time for the system to react to the increase of ’s consumption when configuring the system with small (below) and big estimation intervals (above)
(b) Dynamically triggered budget reestimation calculation
In order to achieve better statistical precision and react quickly upon changes, a solution with dynamic interval between two budget estimation calculations is proposed. Each time overruns its assigned budget (), a counter will increase by one. From the moment the overrun rate is higher than the expected percentage (), it triggers an immediate recalculation for budget prediction. In Figure 9(b), for instance, after the execution, the system arranges a recalculation after detecting that . If the overrun rate stays below , the budget estimation calculation will happen every executions, where is configured by the user.
The proposed dynamic interval policy guarantees for each task with criticality level the rate of overruns is bounded by . Therefore the rate of its deadline misses is also bounded by in a nonoverload system.
The value of is modified upon each server budget reassignment event with (5), where is the difference from the previously assigned budget
10. System Overload Handling
The overload handling service serves the situation where the resource adaptation service cannot assign the servers’ required budget due to system overload. It is designed to respect the tasks’ criticality levels in a MCS. The idea is to suspend low-criticality level tasks for some time in order to make resources available for a high-criticality level task.
A entry is used to identify whether servers at each criticality level are allowed () or not () to run the housed tasks. The number of criticality levels supported depends on the bit-size of which is limited by the processor data width. For example, on a 32-bit processor, it can support up to 32 criticality levels. If needed, this limitation can be easily overcome by using several such data entries. With an 8-bit data entry the criticalities are numbered from right to left as demonstrated in Figure 10. Since for the time being the existing standards consider no more than five criticality levels, one data entry is enough. When the number of criticality levels is less than the number of bits in , several upper bits are masked to 1 according to the difference of these two numbers. Besides that, a vector is used to store the amount of processor resources occupied by each criticality level when they are allowed to run.
is called after dynamic budget prediction. Algorithm 6 presents the process, where is the newly calculated budget value, is the current budget value for , and . Figure 11 is the flow chart for the function .
| ||||||||||||||||
We use special instructions: Count Leading Zeros (CLZ) and Counting Tailing Zeros (CTZ), which can be found in many processors, to speed up the overload handling process .
When a system overload is detected, the first step is using the CTZ instruction to count how many zeros are there in a entry starting from the right. For example, if is 8 bit, 0xFE will result in 1 which identifies that the servers at criticality zero already stopped running their tasks. Assume there are tailing zeros (Figure 12); then the system adds the values from to a budget sum. If the sum is smaller than the overloaded processor resource, set the corresponding bit to zero in . Then repeat the steps until the sum value equals or is greater than the overloaded processor usage. The process is shown in Algorithm 7. The corresponding part in Figure 10 is identified as .
| ||||||||||||||||||||||||||||||||||||
When a server has a recalculated budget lower than the current one, then this overload handling service first uses CLZ instruction to count how many zeros are there in from left. Assume there are leading zeros and is bit (Figure 13), then criticality level is the highest criticality which is being stopped from running its tasks. Then, will be added to a sum. If the sum is smaller than the spare processor resource, the bit is set to be 1. Then repeat the steps until the sum is equal to or greater than the spare processor resource. In another condition that the one requiring resource cannot reclaim enough processor bandwidth, it will suspend itself to avoid affecting servers with higher criticalities. The process is shown in Algorithm 8. The corresponding part is identified as in Figure 10.
| ||||||||||||||||||||||||||
This service adds an additional check upon each server’s execution; as soon as it detects the corresponding bit in is modified, the servers will immediately stop or restart the housed task and change the required budget. Being suspended means the server has zero budget, but the system still switches it from pending state to ready state at the beginning of its periods. When the system tries to start the tasks’ execution, will detect that the server is in the suspended mode and therefore put it back to the pending state.
11. Performance Evaluation
In this section, the system’s performance in supporting MCS is evaluated. The experiments are performed on a single core ARM-based platform, Beaglebone Black. The processor runs at a frequency of 1 GHz and the system’s clock tick frequency is set to be 1 kHz (default value of C/OS III).
Evaluation 1 measures the added overhead of server switch, , server budget and period update, server queue arrangement, slack detection and donation, and reserved budget calculation and prediction.
The overhead tests are based on the technique described in [43]. A 32-bit DMtimer, which is a timing register of AM3358, is used for measuring the time intervals. The timer is configured to run at 24 MHz; the resolution of the evaluation test results is 0.04 s. The results shown in this chapter are rounded to the nearest 0.1 s (±0.1 s). This accuracy is about three times faster than the tracing system overhead (see Table 1). The timer available is not synchronized with what we want to measure.
Figure 14 shows the test methodology. For the operation that needs to be tested, upon the start and the end of it, two time stamps are obtained and put to local memory. All the tested operations are related to scheduling and masked from interrupts. So this test will not be delayed by interrupt handlers. There is no task configured with higher priority than the one being tested. So the difference of and is the overhead caused solely by the tested operation.
Every test obtains more than 10000 samples making the results sufficient for statistical analysing.
Figure 15 shows the test procedure for tracing overhead. There is no operation between the two time stamp writing actions, so the difference between and is the tracing overhead. The aim of this test is to measure precisely the tracing system overhead to enhance the overall accuracy. Tracing system overhead is more hardware-related than software-related. The sources of tracing system overhead are timer register reading and memory writing
The test result (Table 1) shows that there is only very limited jitter in the overhead of the evaluation trace.
The overhead of an operation is obtained by subtracting the minimum tracing overhead (Table 1) from the test result.
In Figure 16, we compare the overheads of server switch when there are different numbers of servers. The results indicate that the value is not related to the number of servers. There is not much difference between the maximum and the average overhead due to the fact that the test set is small enough to fit into the cache. How the cache will affect the performance is beyond the scope of this paper and will not be discussed here.
In Figure 17, ’s overhead in two conditions is shown. The values on the left hand side contain no server switch time. These values are important due to the fact that is called upon every system clock tick and most of the time it returns with no need for switching between two servers. Here it has a value with complexity . The values on the right hand side show what happens when the server switch time is not subtracted from . The maximum value occurs when a currently running server is preempted by another newly released server with a shorter deadline. Then the system has to save the preempted server’s status before switching between their task-ready lists.
Figure 18 shows the overhead caused by the servers’ remaining budgets and periods updating. It is not a constant value due to two reasons: first, the and added by the fault tolerance service introduce uncertainty; second, every member’s remaining period in the needs to be decreased instead of only the first member’s as in the simple EDF example (Figure 6). The complexity of its maximum overhead is then , where is the largest possible number of servers that can be in the simultaneously and equals the number of active servers in the system.
Figure 19 denotes the accumulated overhead introduced by , , , and . Its maximum value is strongly related to how many servers are created in the system. This is as expected. The reason is that the function needs to go through the queues to find the right place to insert the newly released servers or those whose budgets are exhausted. The worst case happens when every server in the queues is checked. Thus the complexity is , where equals the sum of the number of servers in ready state and in pending state.
Figure 20 indicates the overhead introduced by AID and slack donation.
Figure 21 shows the overhead in deciding a server’s budget value in relation to the calculation interval (the amount of execution records used in the prediction). The value is highly dependent on the calculation interval. However, when considering processor time (%), which equals (Absolute Calculation Overhead/(Server of Executions Between each Calculation%, the values for different calculation intervals are almost the same. For instance, for a server that has a period equals 100 ms, the adaptation calculation will cost about 0.005% of the processor time, regardless of how many execution records are taken into account by the calculation. Therefore, if permitted by the system’s memory space, adopting a longer regular calculation interval is a good choice in consideration of a better statistical accuracy, and it will not increase the relative overhead.
Evaluation 2 detects, with our platform, whether the fluctuation in a task’s computation requirement can be effectively handled and whether the overrun faults can be tolerated in a nonoverload system. For this evaluation, a task is designed with variable consumption in each release period. Its processor bandwidth consumption is generated following a normal distribution with a 10% standard deviation from the mean. By design, the value of the mean changes between the and period, the and period, and the and period. For our system, the calculation parameter for this task is set to be 10%. The regular calculation interval in the resource adaptation service is set as 50. The system is not overloaded during the whole process.
The overrun ratios (ORs) and deadline missing ratios (DMRs) in Evaluation 2 are shown in Table 2. The results show that when the fluctuation appears (from the to the release period, from the to the release period, and from the to the release period), the OR is higher. However, the overall OR is less than 0.6%. The highest OR is detected in the period from the to the execution and is still less than 3%. No DMR event is detected. The bound, which is 10% for both OR and DMR as defined, holds in the tested case.
Figure 22(a) presents the task’s actual consumption and the assigned budget in 1–25000th release period. It shows that if a big rise in the consumption lasts for a certain number of periods, a change will be triggered to the assigned budget value. The budget prediction follows the actual change correctly and rapidly. The partial enlarged view in Figure 22(b) shows that when the OR gets too high, a reprediction event will start immediately due to the dynamic calculation interval policy.
(a)
(b)
Evaluation 3 detects the following: under overload condition, whether the system can guarantee the resource for high-criticality tasks; when there is enough resource, whether the low-criticality tasks can be restarted. The overheads caused by and are also measured. The regular calculation interval in the resource adaptation service is set as 100. There are four periodic tasks running in this case study. Their criticality levels and corresponding are listed in Table 3. is predefined to have a highly fluctuated resource consumption which causes system overload at certain time point. The budgets for the suspended tasks are set to be 2 clock ticks instead of zero for a better observation and values are adjusted correspondingly.
In Evaluation 3, the tasks’ ORs, DMRs, and the ratios that they are suspended (SRs) due to system overload are shown in Table 4. During the time that the tasks are not suspended, no deadline misses are detected for all the tasks.
Figure 23 indicates the changes of these four tasks’ actual consumption and assigned budgets across the time. Different from Figure 22, in Figure 23 the -axis denotes the time instead of release period numbers. It shows that small overrun faults can be tolerated for tasks with different criticality levels; when an overload condition is detected, the system can adapt quickly to the situation by suspending some servers in the order of low-criticality to high-criticality and ensure the task with the highest criticality, which is in this case, has enough resource; as soon as the system recovers enough resource, the stopped servers start running again in the order of high-criticality to low-criticality. From Figure 23 one can also tell that if offline WCET is used for the tasks’ budget value, will receive processor bandwidth ≥47%, while the other tasks with lower criticalities will be sacrificed during the whole process.
Figure 24 indicates the overhead caused by the overload handling service in Evaluation 3. Concluded from Figures 16, 17, 18, 19, 20, 21, and 24, the overall overhead caused by all the services will not be greater than 2% of processor resource in the worst case.
12. Conclusion and Future Work
This paper has presented a design that incorporates adaptive reservation into a MCS. Based on the MCS model proposed, a resource adaptation method is used to adaptively change the reserved budget for each task and a slack distribution policy is employed to handle overrun faults. As shown in the evaluation, the overhead caused by the added layer does not take more than 2% of the processor resource. We believe that these values, although have space to be further lowered, are still acceptable in many practical circumstances.
The case studies show the following: the tasks’ overrun ratios and deadline misses are bounded by user’s predefined values for each criticality level; the system is aware of the available resource and guarantees lower criticality tasks a good chance to receive their required resource; the resource usage efficiency of the system is high during nonoverload condition; upon system overload, the tasks with higher criticality levels can always receive enough resource.
In future work, we want to improve the system by adding criticality considerations in the fault tolerance service which means increasing the priority of higher criticality tasks in receiving slacks. As defined by the adopted slack distribution policy, only dynamic slack has been considered. In future work, we would like to also discover the potential applications of static slack (unreserved resources). We also plan to extend this work to other shared hardware resources, multiprocessor systems, and distributed systems.
Competing Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.