Are we test yet?

[ghost]

Right now there are zero tests. We want to fix that so we can actually know what works. For test frameworks we have Pytest (which just looks for functions with "test" in the name), hypothesis for property testing (if it can be property tested please do.), coverage to see how far we've gotten in this journey, and mutmut to make sure we wrote the test right.

Our first (modest) goal is 5% coverage. This probably requires breaking the monolithic functions into smaller chunks.

[JulianKemmerer]

I added a little info to the 'how the compiler works'
about the flow through the tool, steps, architecture of everything

https://github.com/JulianKemmerer/PipelineC/wiki/How-does-the-compiler-work%3F#tool-architecture-overview

Could start making 'units' out of that functionality and making small test

It was brought up - is pycparser under test? Seems to make sense only if we are adding features to pycparser, etc

next level down might tests for parsing C AST nodes to internal ~intermediate things

[JulianKemmerer]

Also @DATAVENIA feel free to add info on style and tests to the new contrib file as they get into use (opening another PR still easy for this stuff?) idk what flow is easiest for you

Would love to see sample cmd lines for running tests for example

[JulianKemmerer]

And to be clear

I currently test by by hand

Running recent example code and checking that it works

Running some Verilator based tests for math / floating point stuff
https://github.com/JulianKemmerer/PipelineC/blob/master/examples/verilator/math_pkg/dut_main.cpp

Also could automate running tests for specific synthesis tools
https://github.com/JulianKemmerer/PipelineC/tree/master/examples/tool_tests

[JulianKemmerer]

rachitnigam was saying

Maybe it would be useful to have some continuous testing to keep the examples up to date! For example, we use runt to do this very lightweight but useful form of expectation testing. Maybe sometime like that would be useful? Happy to help figure out how to set it up if that'll be useful!

[JulianKemmerer]

Maybe we could setup tests that just run pipelinec on a bunch of the example .c files and see if it dies or not - produces a certain output idk