Saturday, December 19, 2015

FuseSoC and VUnit

I recently improved the VHDL support in FuseSoC and since I've been using vunit a bit lately I thought it could be a fun experiment to see if I could combine the strengths of these projects.

For those not aware, FuseSoC is a package manager and build system for FPGA/ASIC systems. It handles dependencies between IP cores and has functionality to build FPGA images or run testbenches a single core or a complete SoC. Each core has a .core file to describe which source files it contains, which dependencies it has on other cores and lots of other things. The aim of FuseSoC is to make it easier to reuse components and create SoCs that can target different FPGA vendors and technologies. FuseSoC is Open Souce, written in Python and can be found at https://github.com/olofk/fusesoc

VUnit is an open source unit testing framework for VHDL released under the terms of Mozilla Public License, v. 2.0. It features the functionality needed to realize continuous and automated testing of your VHDL code. VUnit doesn't replace but rather complements traditional testing methodologies by supporting a "test early and often" approach through automation.

VUnit has a lot of great functionality for writing unit tests in VHDL, but requires the users to set up the source tree with all dependecies and their libraries themselves

FuseSoC on the other hand has knowledge of each cores files and dependencies, but very little convenience functions for writing unit tests. The ones that are existing are mainly targeting verilog users.

Given these preconditions, my idea was to let FuseSoC collect all source code and give it to VUnit to run unit tests on them.

Let's get started

VUnit requires the user to write a small Python script that sets up simulation settings, collects all source files, put them into libraries and starts an external simulator. This script is then launched from the command line with options to decide which unit tests to run, which simulator to use and where the output should go among other things. Here's the example script from VUnit's user guide
 
# file: run.py
from vunit import VUnit

# Create VUnit instance by parsing command line arguments
vu = VUnit.from_argv()

# Create library 'lib'
lib = vu.add_library("lib")

# Add all files ending in .vhd in current working directory
lib.add_source_files("*.vhd")

# Run vunit function
vu.main()

FuseSoC is also launched from the command line and expects to be instructed which core or system to use, if it should do synthesis+P&R or run a simulator and other options. FuseSoC however was always meant to be used as a library as well as a command-line tool, so to make these work together, we create a VUnit run script that imports the necessary functions from FuseSoC. Thankfully both tools are written in Python, or I would have given up at this point.

The first inconvenient difference between FuseSoC and VUnit is that the vunit run script need all source files that should be compiled for any testbench to run. FuseSoC on the other hand don't know which source files to use until we tell it which core to use as it's top-level core. To work around this I decided to look at the VUnit's -o option, which is used to tell VUnit which output directory to use. I simply peek at the output directory and use that as the FuseSoC top-level core name. We now got the first lines of the new script.

from vunit import VUnit
 
vu = VUnit.from_argv()
 
top_core = os.path.basename(vu._output_path)

Now we need to do some basic FuseSoC initialization. First we create a core manager, which is a singleton instance that is handling the database of cores and their dependencies.

from fusesoc.coremanager import CoreManager, DependencyError
cm = CoreManager()

Next step is to register a core library in the core manager. Normally FuseSoC picks up locations of core libraries from the fusesoc.conf file, which can be in the current directory or ~/.config/fusesoc, or from the --cores-root=/path/to/library command-line options.

We don't have any command-line options for this, but we can get fusesoc.conf by using the Config() singleton class.

from fusesoc.config import Config
config = Config()
cm.add_cores_root(config.cores_root)

We can also add any known directories directly with

cm.add_cores_root("/path/to/corelibrary")

The core manager will scan the added directories recursively and pick up any FuseSoC .core files it finds. (Note: If a .core file is found in a directory, its subdirectories will not be searched for other .core files).

It's now time to sort out the dependency chain of the top-level core we requested earlier

try:
    cores = cm.get_depends(top_core)
except DependencyError as e:
    print("'{}' or any of its dependencies requires '{}', but this core was not found".format(top_core, e.value))
    exit(1)

If a dependency is missing, we tell the user and exit. If all was well, we now have a sorted list of cores in the 'cores' variable. Each element is a FuseSoC Core class that contains all necessary information about the core.

With all the cores found, we can now start iterating over them in order to get all the source files and other information we need and hand it over to VUnit. Some notes to the code below:

1. 'usage' is a list of tags to look for in the filesets. Only look at filesets where any of these tags are present. FuseSoC itself looks for the names 'sim' and 'synth' to indicate if the files should be used for simulation and synthesis. We can also choose to only use a fileset with a certain tool by for example setting the tag 'modelsim' or 'icarus' instead of 'sim'
2. File sets in FuseSoC can be public or private. Default is public and indicates that other cores might find the files in there useful. This is for example the files for synthesis and testbench helper functions such as BFMs or bus monitors. Private filesets are used for things like core-specifc testbenches and target-specific top-level files.
3. Even though the most common way to use libraries is to have one library for each core, there's nothing stopping us from splitting a library into several FuseSoC cores, or let one core contain multiple libraries.

usage = ['sim']
libs = OrderedDict()
for core_name in cores:
    core = cm.get_core(core_name)
    core.setup()
    basepath = core.files_root
    for fs in core.file_sets:
        if (set(fs.usage) & set(usage)) and ((core_name == top_core) or not fs.private):
            for file in fs.file:
                if file.is_include_file:
                    #TODO: incdirs not used right now
                    incdirs.add(os.path.join(basepath, os.path.dirname(file.name)))
                else:
                    try:
                        vu.library(file.logical_name)
                    except KeyError:
                        vu.add_library(file.logical_name)
                    vu.add_source_file(os.path.join(basepath, file.name), file.logical_name)


With that we are done, and can safely leave it to VUnit to do the rest

vu.main()

It all works fine, and to make it more interactive, I set up a demo project at https://github.com/olofk/fusesoc_vunit_demo. This contains a packet generator for a made-up frame format together with a testbench. The packet generator has dependencies on two utility libraries (libstorage and libaxis) that I wrote a while ago to handle some common tasks that I got fed up with rewriting everytime I started a new project. The packet generator test bench uses some VUnit functions to make the example a bit more educational.


I hope this will be useful for all people using, or thinking of using, VUnit. For all you others, you can still use FuseSoC without VUnit for simulating and building systems and cores. The FuseSoC standard core library that can be downloaded as part of the FuseSoC installation contains about 60 cores, and there are several more core libraries on the internet that can be combined with this.

Happy hacking!

8 comments:

  1. Hi Olof,

    Looks interesting and I will have a closer look at fusesoc and what you can do with it. Just want to point out that a VUnit script doesn't have to be aware of all source files. You can have separate scripts for different cores and then the VUnit script for a core depending on the others can use the add_external_library(name, path) method. There is no dependency scanning for this approach though.

    /Lars

    ReplyDelete
  2. Hi Lars,

    Great to hear your input, and apologies in advance for the lack of documentation for FuseSoC.

    I probably should have said also that I am not very familiar with the VUnit code base yet, and I had not seen add_external_library.

    Regarding the dependency handling in FuseSoC, I haven't yet implemented my grand scheme yet. Currently, you can only depend on an exact version on a core, but the plan is to add support for version ranges and optional dependencies. Very much inspired by how gentoo does this.

    Keep up the good work on VUnit!

    ReplyDelete
  3. You do not have to piggy-back on the -o/--output-path flag to choose which core to use. We have a way for users to add custom command line options to their run.py scripts using the VUnitCLI class and the from_args method of the VUnit class.

    from vunit import VUnitCLI, VUnit
    cli = VUnitCLI()
    cli.parser.add_argument('--core', ...)
    args = cli.parse_args()
    vu = VUnit.from_args(args=args)

    Also with VUnit you could actually add all cores to the VUnit project.
    Tests for different cores can still be run individually by specifying the test pattern.

    The add_external_library method is mainly intended for adding pre-compiled libraries such as unisim or encrypted third party IP:s. Such libraries cannot be scanned for tests or dependencies.

    ReplyDelete
    Replies
    1. Adding an option to the argparse instance is a better idea. Thanks for that.

      Regarding adding all cores, we could let FuseSoC iterate over all cores in the libraries. For core libraries that use a lot of VUnit testbenches it would make sense, but for a library such as the FuseSoC standard library, there aren't a single VUnit-enabled testbench, so we probably only want to use a subset of the cores in those cases.

      It's great to have these comments. I'm already thinking of several alternative (better) ways to implement this :) and f

      Delete
  4. I tried your demo. Actually it runs into a dependency scanning problem since package VUnit does not scan package instantiations such as "package bfm is new libaxis_1.axis_bfm;" you would have had to add a "use libaxis_1.axis_bfm;" clause for it to have worked.
    I just added support for package instantiations to VUnit master today so now it should work.

    Also VUnit will run all testbenches in your other libraries but they will fail since they do not have the VUnit VHDL stuff in them:
    ==== Summary ==========================================================
    pass axis_packet_generator.tb_axis_packet_generator (1.0 seconds)
    fail libaxis_1.tb_axis_framer (0.2 seconds)
    fail libaxis_1.tb_axis_packet_fifo (0.2 seconds)
    fail libaxis_1.tb_axis_sync_fifo (0.2 seconds)
    =======================================================================

    ReplyDelete
    Replies
    1. Oops.. The package instantiation was a leftover from an old testbench. I fixed that now. You need to clear out libaxis-1.0 manually and run again.

      I also changed the libaxis-1.0 core file in the fusesoc_vunit_demo, so that the libaxis testbenches don't show up when you want to run the packet_generator tests

      Delete
  5. Hi Olof, is there any chance we can get an updated version of this demo that works with the latest versions?

    ReplyDelete
  6. vunit is mentioned in the fusesoc documentation here: https://fusesoc.readthedocs.io/en/stable/ref/capi2.html?highlight=vunit

    would have been nice to add an example for this functionality

    ReplyDelete