Recently, it was the 10th annual Jasper User Group meeting (see my earlier post Jasper User Group 2017 for more background, and a summary of the two "best" papers presented). There were also two invited papers. To open the first day, the extensively named M V Achutha Kiran Kumar presented on Formal Verification Lead—An Interesting Career , and the second day was opened by Vigyan Singhal of Oski Technology presenting The Evolution of Formal Verification Sign-Off . Despite the very different titles, both presentations covered some of the same ground: a little history, some best practices, and how to sell formal verification inside your company to design teams and management. I will cover Vigyan's presentation in another post. An Interesting Career M V Achutha Kiran Kumar works at Intel, but he is well known in the formal verification community as one of the three authors of "the" book Formal Verification: An Essential Toolkit for Modern VLSI Design . Last year's conference had a keynote from Erik Seligman, another of the authors. Maybe Tom Schubert, the third author, will come next year. Kiran opened with a bit of history. He said that formal in Intel really kicked off with "a well-known bug that I'm not supposed to name". But Bob Bentley wasn't so reticent at 2012's Jasper User Group where he gave the keynote. As I said in my post about his keynote: Formal approaches suddenly gained a lot of traction after the 1994 Pentium floating-point divide bug. This caused Intel to take a $475M charge against earnings and management said "don't ever let this happen again". In 1996, they started proving properties of the Pentium processor FPU. Verification is a big challenge. Kiran showed Wally Rhines' slides from his DVCon keynote a couple of years ago that showed that there were already more verification engineers than design engineers, and that the CAGR for designers was 3.6% but for verification engineers it is 10.4%. I am sure most of them are mostly doing simulation. The growth rate for formal verification engineers must be higher still, but that largely reflects that it was starting from a small base. There has been a paradigm shift in formal verification, going from a set of PhDs working in silos, to different methodologies working together to achieve verification goals. This also meant that the concept of a formal verification lead came into existence, and one of the key tasks was to "sell the methodology" so that management could see how they could benefit. The rules for being a formal verification lead are: Sell the methodology Choose the right tool (that would be JasperGold, of course) Work with the architects to decide how things will be verified downstream, and with the designers Have a solid test plan, that is part of the overall (including simulation) test plan Speak the language management understands Constantly feed management progress, in metrics that they can understand such as coverage numbers, and articulate the ROI One thing that can help convince management is to take on an absurd challenge, one that would be impossible with simulation. He took a task that was budgeted for 10 weeks using simulation (and probably emulation too), and brought it into a two-week formal verification task. It took less than two weeks and the feature was in. Another bit advice came at last year's Jasper User Group (see What Is the ARM ARM? ) with Daryl Stewart's advice: The most important, in terms of keeping senior management on board, is that "this would have been a Cat A error not caught by simulation and you'd have been on a plane." Now that is language that management understands! Designs start with architecture and microarchitecture. Formal needs to get involved at that stage. The first reason is that some properties can be proven at the architectural level to validate the architecture itself. For example, nobody can design a cache-coherent interconnect at the RTL level without first verifying that the architecture itself is correct (and nobody seems to be able to get it correct first time). For more details on verifying cache-coherence, see the Arteris section of Decoding Formal Club: Arm and Arteris or CCIX Update: TSMC, Xilinx, Cadence, Arm...and Jasper from earlier this week. Some things like livelock and deadlock that are really hard to verify with simulation (did you really get every corner case?) are straightforward with formal. Also, how the architecture is assembled and described (executable code) can have a direct influence on verification possibilities. As usual, nothing succeeds like success, and solving a recurrent deadlock issue with formal means that at least one architect sells the formal methodology to anyone who will listen. Help management to understand by talking in terms they understand: Coverage Progress metrics against the test plan Regression pass rates Number of bugs found Don't forget to talk about the most interesting vector to them, dollars aka ROI Give management the tools they need to hold designers accountable Formal progress used to be hard to measure. It was converging (great), or it wasn't (and nobody knew when it might). Now progress can be tracked against the plan, and the test plans for sub-blocks can be tied into the test plans for higher level blocks. There are a number of ways to measure ROI, all of which have their place: K lines of code verified Millions of gates verified (or percentage of the chip area). Great to have a dashboard that shows all these basic metrics $ savings (need to weight since value depends on where the bug is found in the design flow. Curve is exponential with 1,5,10,25,100 for if the bug is found at unit level, cluster, emulation, full-chip, post-silicon) $ costs incurred Bugs found against $ costs incurred Coverage against $ costs incurred Engineering costs saved due to the shift left (bugs detected earlier) But...bugs found near the end ring a bigger bell (they were hard or dangerous by definition) Formal verification doesn't exist in isolation. The tools can be used together. The most obvious way is that anything that formal has proved does not require tests to be created for simulation, too. Extremely long cycles (timeouts, maximum length packets, etc) are hard for formal but straightforward for simulation. The tools can be used together, too, using formal to prove that certain code can never be executed and so doesn't require simulation. Semi formal is important as an alternative to constrained random simulation. This allows formal verification to be driven much deeper into the design than it would manage on its own to hit specific issues or code that is hard to verify. To go from proof of concept: Start small Take baby steps Establish the efficiency of the flow Work with the vendors and make sure the PoC succeeds Results of the PoC should translate into making the flow a PoR Constantly track usage models and re-invent better ways of attacking the problem (the technology is changing all the time) Start with simple stuff with high benefit: Take the latest bug found and find it quickly again. Establish clean RTL2RTL flows (with Conformal LEC). When changes should not change functionality, it is straightforward (clock gating, timing fixes, RTL optimizations, etc). If incremental changes do change the functionality, then you can constrain to disable the new functionality and check everything else, then check the new functionality separately. After a project winds up, it is good to analyze the hits and misses from formal. Find the big successes (fatal bug found at the last moment is always good), find genuine misses and constraint issues. Refine the methodology for next time with what is learned. In a nutshell, a formal verification lead is an evangelist for formal, a technology lead for the methodology, needs to work on the financial methodology to justify and evaluate in dollars, and an internal marketing guy for the technology. It is not just (or even mainly) a technical role, and because formal is not completely accepted it is above and beyond a normal lead (the simulation lead doesn't have to justify simulation being required). Of course, if you want to know more, then you should buy the book. Note that this is a book for verification engineers using formal verification, not a book for EDA software engineers on formal verification algorithms (which is what most books on formal seem to be). See my review . Just click on the book for an Amazon link: Sign up for Sunday Brunch, the weekly Breakfast Bytes email.
↧