Consistent source location transfer with Patch

[widlarizer]

The greatest source of loss of source attributes is synthesis with abc, but not all users use abc and there’s a lot of gaps in Yosys passes. To address this, there’s two major features I think we need.

Patch

I have started prototyping a data structure called Patch that allows passes to prepare rewrites to the design and that will come with automatic source transfer. Potential future benefits is higher functional clarity, lower reliance on opt_clean to garbage collect design objects and cheaper equivalence checking of pass runs

Twines

To manage larger amounts of src locations, including “fused” source locations (some cells and wires are derived from multiple places in the source code) I’ll be adding new twine-tree data types which requires changes to the RTLIL textual format. This should improve performance and memory efficiency beyond the status quo, but extending the RTLIL textual format is something we should release only once we’re happy with it

Development will have to happen in PRs targeting the emil/src-location-story feature branch due to being fairly severe additions to infrastructure and publicly exposed interfaces. After it’s merged to main, I plan on keeping Patch an unstable feature, so that external users have access to it with appropriate expectations about it being likely to evolve for some time.

I’m putting up this thread with the sparse information I have right now just so I have something to point to, and major changes to the plan still might happen. I’ve also had to focus on other tasks for a couple of weeks so I haven’t touched it in a while

[rocallahan]

@widlarizer

I have started prototyping a data structure called Patch that allows passes to prepare rewrites to the design and that will come with automatic source transfer.

Would be really nice if we can prepare a Patch in off-main threads!

[widlarizer]

Here’s the minimum viable product of the Patch. So far so good! Twines are conceptually a simple implementation detail in comparison to designing a universal local design modification interface, so I’m not including them in the work so far

RTLIL::Patch attribute transfer and more (#5902)

[widlarizer]

I’ve been working on Twines, it’s tricky to integrate. Here’s a rough demo before I started weighing the interface down by integrating into Yosys:

main.cpp

That’s also before I switched TwineRef pointers to indexing with integers, which required me to fork the backing plf::colony allocator

GitHub - widlarizer/plf_colony at indexing

The TwineRef don’t live in a name field of named objects, but in a metadata block outside of the object. However a name field is provided for compat: [[no_unique_address]] RTLIL::CellNameMasq name;

This field should be able to materialize escaped or unescaped strings as needed. I can’t have unescape return a string_view since the whole point of twines is that you don’t hold the string in contigous memory. This is why it’s also a great time to throw out private/public name distinction from strings and instead push it into the metadata block. Prior discussion:

Cleaning up `log_id()` and `RTLIL::unescape_id()`

log_id() is ugly because it returns a const char * and to avoid potential lifetime issues it stores the string in log_id_cache which leaks memory until the end of each pass. Also, it does exactly the same thing as RTLIL::id2cstr (which just wraps log_id()) and apart from the return type, that’s the same thing as RTLIL::unescape_id(). The name log_idisn’t ideal because it doesn’t actually log anything. The name id2cstr isn’t ideal because it does something different from IdString::c_str() (the f…

So far, I also see this is as a parallelization win, as interning twines on per-design pools will be possible from multiple threads working on different designs. Also, Patch doesn’t intern strings into the design pool, it keeps twines uninterned in a local temporary pool, which interns its contents into the design pool only when the Patch is applied. This means it should be possible to prepare Patches in parallel, too!