Thinking beyond constrained random - VHDL Leapfrogs SystemVerilog

[JimLewis]

In Nitish's post, he asks the question, "Can we think beyond SystemVerilog ...?" While an interesting question, I think the more interesting question is, "Can we think beyond constrained random?"

Constrained random comes with a fundamental problem. Randomization is intended to be uniform over time. In testbenches, we generally want one each of a large set of test cases (transactions and/or sequences). Uniform randomization does not generate one each. Instead it has a significant amount of repetition. In general, uniform randomization takes O(N*LogN) randomizations to generate N unique test cases. As a result, it repeats Log N test cases. Even for small numbers such 64 test cases, constrained random will generate more than 4X more test cases than needed - actual results will vary with the randomization seed.

What we need is an approach that only requires O(N) randomizations to generate N unique test cases. Generally these approaches are referred to as being Intelligent Testbenches. Indeed there are some tools out there that handle this. However, when we use a tool based approach we end up with a vendor specific solution. This removes one of the major benefits of a programming language based approach - encounter a issue (pricing or functionality) with one vendor and you can easily switch to another.

What we really need is a methodology for Intelligent Testbenches that is based on a standard language and works on numerous vendor tools. We have one. VHDL's Open Source VHDL Verification Methodology (OSVVM). OSVVM's methodology leverages the functional coverage you must write when you are using any randomization based approach. Intelligent Coverage™, the main randomization methodology for OSVVM, randomly selects a hole in the coverage and passes this to the stimulus generation process. The stimulus generation process uses this information, perhaps refines it using any methodology (directed, algorithmic, constrained random or file based), and then generates one or more transactions to accomplish generate the item that needs covered.

OSVVM works with any VHDL testbench and is particularly effective when coupled with a transaction based testbench. For me, VHDL and OSVVM are the step beyond constrained random and SystemVerilog. Maybe it is time we update VHDL's acronym to mean Verification and Hardware Design Language.

To learn more about OSVVM, see:

• The OSVVM Community Forum
• SynthWorks' OSVVM Blog
• OSVVM Webinar on July 18th (US time zones)
• OSVVM Webinar on July 18th (European time zones)

OSVVM is an open source VHDL library that is free to use (no license fees) and works with any simulator that supports VHDL-2008 (or VHDL-2002 with a little work).

What is currently in the OSVVM library is only the beginning. Over time, I will be releasing our generic scoreboard package, memory modeling package, and others. None of this is rocket science and it is all feasible with VHDL-2008. All of these are provided in our VHDL Testbenches and Verification - OS-VVM Boot Camp class

 

[Daniel Payne]

Jim,

Thanks for the info on the Open Source VHDL Verification Methodology.

Is there no equivalent of this approach on the SystemVerilog side?

 

[JimLewis]

Daniel,
There are a couple of tools that do intelligent testbenches with SystemVerilog, but no language based approaches I have seen.

What the SystemVerilog community calls "Coverage Driven Verification" is an analysis step that says stop running this testbench when it is no longer increasing the functional coverage - for some this is an analysis step done after the test is run the first time. Unfortunately the there is no API that allows the testbench code to seek coverage closure on items it has not yet generated.

When randomizing using the Intelligent Coverage methodology, we rely on a high fidelity functional coverage model that potentially allows each bin in the model to have a different coverage goal. OSVVM supports an incremental method of capturing functional coverage in sequential code that offers a greater degree of flexibility and capability than SystemVerilog's declarative method. To learn more about this check out my blog post, VHDL Functional Coverage is more capable than SystemVerilog. In a later blog post, I will be explaining how OSVVM uses different coverage goals within a coverage model (coverageGroup) to do weighted randomization.

Jim

 

[Daniel Payne]

Jim,

Thanks for the update.

Would using a scripting language like Tcl work for your language-based verification approach? Most EDA simulators support Tcl scripting to control stimulus, make measurements, and continue simulating.

Or, what about using C code?

 

[JimLewis]

Hi Dan,
It is all programming. I am sure if you worked at it hard enough, you can do this in any language. If the language does not provide an API to access its own built-in constructs, you may have to abandon them in favor of ones you create. Any steps that are programmed (VHDL, Verilog, SystemVeriog, 'C', TCL, python, ...) would be ok.

The main point of an Intelligent Testbench is generating N test cases in O(N) steps. As a result, the coverage feedback step will need to be frequent, perhaps each stimulus generation step (ie: randomization), otherwise, you will be repeating test cases.

The main point of my post is that OSVVM is a vendor independent, open source (free), intelligent testbench methodology that is available today. I have not heard of anyone releasing any form of vendor independent, intelligent testbench methodology for Verilog, SystemVerilog, C, or TCL whether open source or not. Have you?

Hence, OSVVM gives VHDL for verification a significant benefit in over SystemVerilog and UVM (which focus on constrained random test generation).

Jim

 

[simguru]

I'm currently working with VHDL for the first time in years, and it's every bit as bad as I remember. Nobody in their right mind would use this language for anything serious. I don't even know how it got going in the first place given it has no support for bidirectional devices, and it's unnecessarily fussy about everything - no stray semicolons, no empty lists.

 

[Daniel Payne]

In 1987 when I first started hearing about VHDL the coders would call it a write-only language, or Verbose Hardware Description Language. Anything mandated by the US Government (or DOD) was destined to be large, verbose, and have something for everyone.

In stark contrast Verilog was designed to be simple and efficient, thank you Phil Moorby.

 

[simguru]

I think Verilog was an evolution of other simpler logic simulators, while VHDL was a cut down version of ADA.

As far as I can tell there have been no major advances in the modelling capabilities of commercial simulators other than some analog modelling since the 80s. In particular there have been no additions to handle power management.

I suggested that they support all VHDL features (warts and all) in SystemVerilog so we could deprecate it and just translate the legacy stuff into SV, but nobody went for it.

I was reconsidering this recently since the SystemVerilog guys are refusing to incorporate any analog support and that leaves VHDL as the only IEEE standard that supports both analog and digital (if dysfunctionally). So it may be easier to fix VHDL so that it works properly with C++ interfaces for new stuff such that it does actually handle everything sensibly. However, I'm thinking that it might be easier to do that with the open-source community first (e.g. Qucs) and donate something to the IEEE rather than slog through the committee process - which has been unproductive in the past.

If I was starting from scratch I would just do some extensions on C++ or ADA to support hardware modelling and skip the specialized HDLs, the design of the HDLs is quite bad and the lack of open-source support has killed development.

 

[JimLewis]

@Dan,
Really? Compare to VHDL-1987? That is silly!

VHDL-2008 + OSVVM compares favorably to SystemVerilog+UVM and/or Verilog in both capability and conciseness.

Check out my conference slides for VHDL-2008, The End of Verbosity! which is at Synthworks Expert VHDL Training: VHDL Papers
There is also more VHDL-2008 information at: VHDL Standards | SynthWorks Blog
The above reference includes links to my Verification Academy series on VHDL-2008.


@Simguru (Kevin right?)
You want bidirectional connections? You now have edit capability for VHDL-201X proposals. Here is the blank proposal for you to fill in: BidirectionalConnections < P1076 < TWiki
If you pass on doing the work, then you loose the right to complain.

";" at the end of an interface list is on the 201X list. Please email me your empty list issue.

Jim Lewis
VHDL Training Expert, Synthworks
IEEE 1076 Working Group Chair
OSVVM Chief Architect
jim at SynthWorks

 

[JimLewis]

I suggested that they support all VHDL features (warts and all) in SystemVerilog so we could deprecate it and just translate the legacy stuff into SV, but nobody went for it.

That is funny as I suggested that VHDL-2008 support Verilog gate-level libraries, because after all, Verilog makes a fine netlist language.

So it may be easier to fix VHDL so that it works properly with C++ interfaces for new stuff such that it does actually handle everything sensibly.

If the following link is not what you are looking for, you should put forward a proposal: DpiProposal < P1076 < TWiki
Also VHDL-201X is coordinating the standard effort with open source groups.

Jim

 

[Staf_Verhaegen]

I'm currently working with VHDL for the first time in years, and it's every bit as bad as I remember. Nobody in their right mind would use this language for anything serious.

Aren't the language, editor, etc. wars so eighties/nineties ?
BTW, front-end here prefers and thus most things in VHDL as it will make written code do what was intended. But then I am in Europe; likely considered a region without a 'right mind'.

 

[nbjessen]

strong-typed v. weak-typed

In 1987 when I first started hearing about VHDL the coders would call it a write-only language, or Verbose Hardware Description Language. Anything mandated by the US Government (or DOD) was destined to be large, verbose, and have something for everyone.

In stark contrast Verilog was designed to be simple and efficient, thank you Phil Moorby.

I think there's a higher-level concept here: strongly-typed languages are that way, because one of the fundamental ideas in verification is to be able catch problems as early as possible, and when they're caught, to be able to debug as easily as possible.

Thus:

• one performs unit-testing because it catches bugs early and has easiest debug, rather than waiting until system-integration.
• Verilog lacks strong-typing, so Verilog users have to use linters. Checking is free with strong-typing, but linters are expensive, error-prone and almost always incomplete (or vendor-specific).
• One creates pretested packages and explicit mappings/translation (between classes), to avoid the extremely difficult debug process if an implicit mapping/translation doesn't actually always do what you want: because not only do you need to worry about fixing THIS project, you have to worry about the bugs lurking in every other project that used this same implicit effect.

From a project schedule/loading standpoint, you always want to pick what's fastest/least cost.
If you're doing a 100-line project, Verilog/TCL/Perl/Python/C is great.
If you're building a 100M gate chip, use the language and methodology that will allow you to completely test sub-components before integration, because catching bugs at higher levels of integration is prohibitively expensive. That generally implies a language with strong-typing, packages, using revision-control and configuration-management, etc.: VHDL/C# for example.

Erik

 

[Daniel Payne]

JimLewis,

You are correct, with VHDL 2008 the improvements do approach the attractive brevity that Verilog has always had since the start.

Simguru,

The Open Source world can work only if there is a business model where development can be paid for. Look at the Open Source success of Red Hat Linux or WordPress as recent examples. I can put you in touch with an EDA start-up that will offer cloud-based tools that are Open Source.

 

[simguru]

Hi Jim,

Tried getting support for bidirectional signals into VHDL before with Rick Munden over a decade ago, the committee wouldn't even let us get it up to a vote.

At this point I feel offering solutions to problems to HDL committees is either pearls to swine or horse and water, and I'm only going to do it if I have actual voting power - i.e. if someone wants it fixed they can pay me to bang my head against the wall.

On the C++ front I was thinking more direct link-level compatibility, i.e. the code is compiled such that the objects are directly accessible by pointer/method as they would be if they were actual C++. Suggested that one a decade+ ago for SV but it got quashed, with the result that we are now replicating lots of standard C++ stuff in SV for testbench use. I don't mind wheels being reinvented if they are better, but SV is not a better version of anything.

(Kev)

 

[simguru]

Aren't the language, editor, etc. wars so eighties/nineties ?
BTW, front-end here prefers and thus most things in VHDL as it will make written code do what was intended. But then I am in Europe; likely considered a region without a 'right mind'.

The war ends when they're dead.

VHDL is pretty much useless for anything below RTL, and it doesn't have the A/D boundary handling of Verilog-AMS. It probably only persists because it's adequate for FPGA work. Can't see anyone rushing to do their power electronics design in VHDL.

Yep, the Europeans should know better.

I've just spent half a day recovering from discovering someone pegged the local VHDL code at '87 because they put a a call to "now" in something which makes it not work in '93 because it insists on that particular usage being "pure" - like that is in any way relevant to my SoC design.

VHDL has practically no merits as an HDL and next to none as program language.

[Trust me, I do know what I'm doing - at least when it comes to simulators]

 

[nbjessen]

I think you're missing an important point: strong-type-checking languages are intended for use in very large projects, because the more explicit one has to be, the fewer misinterpretations can be made (for example on how to cast from one variable-type to another). When doing SystemVerilog, one has to do all the same things as VHDL (create packages, create casting functions, etc.). It's just that one winds up doing more debug at runtime (which is slow) compared to compile time.

 

[nbjessen]

I prefer VHDL, because with SystemVerilog, I have to do a lot more unit-testing to make sure type-conversion goes as I think it will go.
And then there's having to deal with legacy code that has #0 delays and things like that - at least VHDL doesn't have that.

The bigger the project, the more reuse you need to do, and the more precisely you need to verify sub-components, AND be sure they'll work the same way (from simulator version to simulator version, with all possible option settings).

 

[simguru]

I think you're missing an important point: strong-type-checking languages are intended for use in very large projects, because the more explicit one has to be, the fewer misinterpretations can be made (for example on how to cast from one variable-type to another). When doing SystemVerilog, one has to do all the same things as VHDL (create packages, create casting functions, etc.). It's just that one winds up doing more debug at runtime (which is slow) compared to compile time.

Actually, the strong type checking is mostly for languages that are compiled in modules and will be linked together later - which would be true of ADA the language VHDL is based on. People use languages like Python for large projects too (though that might not be a good idea).

Given how hard it is to actually describe things in VHDL I'm not sure the debugging argument holds - C++ is much more efficient. SystemVerilog mostly adds functionality at testbench level, circuit-level it's pretty much the same as Verilog (which only has one type).

A problem with VHDL's type scheme is that wires/ports are typed when wires should be neutral/void and only drivers and receivers need types. Also, many type conversions which could be automatic are disallowed for arbitrary reasons or require explicit casting.

 

[jonbu]

Ancient language wars continued: Does anyone really think verilog is ok for RTL?

Just cannot resist diving in, having done ASIC design (not just verification) in both VHDL and SystemVerilog, and having had the misfortune to tweak some existing verilog. Sorry if it's off topic.

• I'm talking RTL design for synchronous digital (with clock domain crossings of course), not verification code.
• Verilog is the most verbose and appalling language for the above - just look at what it takes to define ports and instantiate modules
  • VHDL is much much terser.
  • System verilog is terser still, which is good.
  • But for these reasons alone I regard rtl coding using old verilog at nuts. I'd love to be in that market with so much typing time on your hands.
• Verilog and systemVerilog are enormously dangerous for ASIC design.
  • The quiet truncation of assignments with for example packed structs being able to be assigned to single logic bits is scary.
  • With $million mask costs, I want a strongly typed ASIC design language with no chance of that sort of silly error creeping through.
  • Why spend time and money on separate linters? Again, I wish I was in a market where design time and money were clearly so plentiful if you can afford linters and the time to configure them usefully.

In summary VHDL is the best of a bad lot for ASIC design. SystemVerilog is reasonably close, but has some terrible verilog baggage with it. Old verilog is just completely out of date, and astonishingly error prone. Nuff said. To get back on topic, properties with formal proof tools is the right way to go. TDD (test driven development) with assertions/covers, proving small pieces of code as you build it is the fastest way to high quality (aka no asic spins). The properties then also get run automatically by the constrained random environment when it eventually gets up and running.