CSCI 1600 Embedded and Real Time Software Lecture
CSCI 1600: Embedded and Real Time Software Lecture 6: Verification I Steven Reiss, Fall 2019
Software Requirements are central to embedded systems May be mandatory (have real consequences) Car or plan crashes Broken devices Heart stops pumping Water is supplied unfiltered The nuclear plant does not shut down Failure might now be an option Types of Requirements Safety, Timing, Fairness 2 Lecture 30: Verification I 11/24/2020
Safety Requirements The car won’t crash Trains won’t crash Both traffic lights won’t be green at the same time You won’t blow a fuse The system won’t deadlock The heat and air conditioning won’t be on at the same time The doors won’t close on a person The elevator won’t stop between floors
Timing Requirements Forms X will occur within k ms of event Y X will occur every 10 +/- 1 ms Examples The solenoid will trigger within 10 ms of the bumper being hit The switches will be sensed every 20 ms The heat will come on within 5 minutes of detecting the room is too cool The elevator will get to the first floor in 5 minutes or less
Fairness Requirements Forms Eventually X will occur If X occurs, then eventually Y will occur X will occur infinitely often As long as Y is true, X will occur infinitely often Examples Eventually you will add K to the score when you hit Y As long as the heating system is on, the room will eventually be comfortable If you push a button, the elevator will eventually get to your floor
Example: HVAC System Other Possible Properties Show the heat is on if the room is too cool Show that air conditioning is on if the room is too hot Show neither is on if the room is comfortable Show that air conditioning doesn’t come on within 5 minutes of its last being on Show temperature will be near comfortable levels at a designated time Are these going to be true? What would you really show 6 Lecture 30: Verification I 11/24/2020
Example: Pinball Show that you don’t blow fuses No more than 2 solenoids on at once No more than 8 lights on at once Show that each element scores correctly Show ball logic is correct (number, free balls, …) 7 Lecture 30: Verification I 11/24/2020
Requirements: Goal Show the system meets its requirements That the system is safe That the system works correctly Have confidence in the system’s safety Show the system meets its requirements Under all possible circumstances 8 Lecture 30: Verification I 11/24/2020
Helpful Techniques: SE Good requirements analysis Understanding all the possible problems and solutions Understanding what it means to be safe Good specifications Accurate modeling Showing the models are correct Design for safety Defensive coding Sanity checks 9 Lecture 30: Verification I 11/24/2020
Helpful Techniques: Monitors Add monitor tasks to the code These check that the system is operating correctly Detect if something is going wrong Too many solenoids turned on Train moving onto a block containing another train Lights have been on for > 10 ms Temperature change more than expected Often easy to write If something is detected (unusual or wrong) Move the system to a “safe” state 10 Lecture 30: Verification I 11/24/2020
Helpful Techniques: Watchdog Timer Generate a time-out signal (interrupt) periodically System must actively reset this Often separate hardware for safety If the signal isn’t reset Move the system to a safe state 11 Lecture 30: Verification I 11/24/2020
Helpful Techniques: Testing Use lots of test cases Simulate the hardware and ensure system runs Simulate potential failures and see what the system does Multiple simultaneous failures as well as single ones Use regression testing Reuse test cases (don’t just throw them away) Test for unusual (error) cases, not just working ones Build test mode into the system 12 Lecture 30: Verification I 11/24/2020
Helpful Techniques: Validation Run time assertion checking Preconditions and postconditions Defensive coding Sanity checking But what do testing & these tell you How sure can you be? 13 Lecture 30: Verification I 11/24/2020
Verification: Basic Idea Mathematically prove the program is correct Show the program works for all possible runs Not just the ones covered by the test cases All possible (and impossible) inputs Show the timing works for all possible schedules Show that all (critical) requirements are met 14 Lecture 30: Verification I 11/24/2020
Verification: Issues What does it mean to be correct? If we are going to prove something, it has to be precise How can you state precisely what the program should do? Especially for a complex system How can you model all possible runs? There an infinite number of these (non-terminating) Or a very large finite number Runs are dependent on the outside world Runs are dependent on user inputs 15 Lecture 30: Verification I 11/24/2020
Verification: Issues Programs are very complex Proving anything about a program is undoable Almost anything you want to prove is undecidable Halting problem Actually not true since machines are finite, but effectively true Even restricting values to finite ranges, etc. Can involve huge numbers of program states and value sets 16 Lecture 30: Verification I 11/24/2020
Verification: Issues Proofs are very complex The mathematics involved is complex Would you believe a 1000 page proof of a program? Suppose it is done by a computer How long is the program doing the proof Has it been shown to be correct What about the compiler and the hardware? Yet the work has to be done And techniques have been developed to do it 17 Lecture 30: Verification I 11/24/2020
Verification: Basic Idea Steps Model the program Prove properties of the model Program Proof in Program Model Proof in Model Prove that the program satisfies the model 18 Lecture 30: Verification I 11/24/2020
Modeling the Program Goal: Map the program into a finite state automata Why finite state automata? It already is, but the number of states is very large We already have a finite state model (modeling) Reduce the number of states Restrict variable values to reduce the number of states Restrict the number of threads, tasks, etc. Show that the restriction doesn’t affect proof 19 Lecture 30: Verification I 11/24/2020
Proving the Model is Correct Provide a mapping from the code to the model Automatically generate the model from the code Prove the code implements the model Show that other aspects of the program don’t matter Consider all possible interpretations of those properties 20 Lecture 30: Verification I 11/24/2020
Model Checking 21 Lecture 30: Verification I Program Proof in Program Model Proof in Model 11/24/2020
Model Checking: Properties First we need to be able to state the property In a form that is understandable In a form that can be checked Represent the property to be proved in finite form As a finite automata As properties over a finite automata States in a finite automata are labeled with propositions Then you want to state properties over those propositions Taking time into account 22 Lecture 30: Verification I 11/24/2020
Finite Property Example Once an iterator is allocated, every call to next must be preceded by a call to has. Next 23 Lecture 30: Verification I 11/24/2020
Model Checking: Program Model the program as a generalized finite automata Need to generalize to handle infinite programs Automatically map code into model To ensure the mapping is correct With some help from the programmer For example, providing set of legal values for an integer The mapping has to be conservative Cover all possible executions Might allow “impossible” executions as well 24 Lecture 30: Verification I 11/24/2020
Modeled Program Example for (Iterator it = x. iterator(); it. has. Next(); ) { y = it. next(); } it = x. iterator / ALLOC it. has. Next() / HASNEXT y = it. next() / NEXT 25 Lecture 30: Verification I 11/24/2020
Model Checking: Proofs Show that all executions of the finite program Satisfy the desired properties How this can be done In terms of executions In terms of languages (symbolic math) 26 Lecture 30: Verification I 11/24/2020
Iterator Example A it = x. iterator / ALLOC B it. has. Next() / HASNEXT C D y = it. next() / NEXT E
Defining Properties Modularize correctness Rather than trying to show the whole program is correct Break correctness down into smaller pieces Define correctness in terms of specific program properties Prove each of these separately This is often easier than trying to show everything at once 28 Lecture 30: Verification I 11/24/2020
Example: HVAC System My heating system 3 floors Basement and 2 nd floor share common water unit Basement is radiant floor heating, 2 nd floor is water->air These have separate thermostats Want to ensure that not both heating and cooling the water How might you express this? 29 Lecture 30: Verification I 11/24/2020
Example: HVAC System Think about the HVAC controller as a FSA Transitions between states determined by events Including timer events and random events (outside temp) We can define different properties of interest Heating On (H), Air Conditioning On (AC) Build an FSA with states based on these properties This represents what we want to prove ~H, ~AC 30 Lecture 30: Verification I H, ~AC ~H, AC 11/24/2020
Example How would you model the program to check this Look only at elements that affect turning on heat and air conditioning and their interaction Assume the rest of the system can do whatever it wants Arbitrary setting of thermostats, temperatures, etc. Based on what program actually does 31 Lecture 30: Verification I 11/24/2020
Example: Microwave Oven
Example: Properties For all paths, if we hit start, then we eventually get heat Is this true? What would you actually prove? What else might you show
Next Time We’ll look at formalizing this process And later show it can be made practical 34 Lecture 30: Verification I 11/24/2020
Homework Read Chapter 14 Exercise 14. 7 35 Lecture 30: Verification I 11/24/2020
- Slides: 35