The day has finally arrived. The teams have been working nights and weekends for months and it is time to bring some closure and get to know our families again. The team leads are gathered in the war room for the “tapeout readiness review.”
The project manager starts around the room:
“How does timing look?”
“We fixed the last three slow paths and restarted STA last night. We have good confidence they are fixed and should have the final results this afternoon.”
“Good. Layout?”
“LVS is clean. We had 1500 DRC errors from the last rule changes and fixed them over the weekend. The run finished this morning and was clean.”
The Timing and Layout guys exchange a fist bump.
“Excellent! How does DFT look?”
“We have BIST in place for all the memory blocks and 95% stuck-at coverage on the random logic. We expect to hit 98% after we tweak some of the vector generation knobs.”
The DFT and layout guys exchange high-fives. Anticipation is building in the room but it is about to get awkward.
“Super! Is verification done?”
“No, we aren’t done. We can never really be done. You remember, the whole state-space larger than the number of atoms in the universe thing?”
The verification engineer traits of accuracy and compulsive honesty are starting to put a damper on the celebration.
“Oh yeah, we talked about that. OK then, how many bugs are left?”
“I could tell you that if we were done.”
The energy has been sucked out of the room. It is getting quiet.
“Right. I guess we just need to know if there are any fatal bugs left.”
“I could tell you if I knew which bugs were left.”
The other team leads are starting to wonder what is wrong with the verification team. Maybe verification really is hard (they can’t tell for sure) but how can you not know when you are done?
* * *
Why is it that “verification closure” is so elusive?
For a specific timing model, you can achieve timing closure by analyzing all of the paths in a circuit. A set of layout rules can be applied to all of the nodes in a design. For a given fault model, the exact fault coverage can be measured. All of these activities are bounded by the size of the actual implementation.
On the other hand, design verification is the comparison of a design implementation with a specification, comparing one really large state space with another.
If you have a mathematically precise (i.e., “formal”) specification, it is possible to compare it completely with the implementation using exhaustive simulation or symbolic means. This “complete formal verification” is the only real completion criteria.
In practice, formal specifications of entire designs are rare and the time and resources to formally verify an entire design are limited to products with extreme safety or security requirements.
Falling short of the gold standard of complete formal verification, we try to estimate how close we are coming to it. We don’t generally know the actual size of the state space we are trying to verify, but we use experience to try and identify the important parts of it with our verification plans and functional coverage.
The hot seat experience described above was real, but fortunately I had been working with the program managers ahead of time to help them understand that the current state of verification practice could not provide a crisp answer like the other team leads could provide. Instead, I detailed what we did for this tapeout above and beyond what we did last time: additional coverage types, higher coverage numbers, and focused efforts on the new features and design changes.
This ad hoc estimate of relative quality was not very satisfying, but was the best we could do in the real world and led to a successful tapeout.
More on quality measures in future posts.
© Ken Albin and System Semantics, 2011. All Rights Reserved.