Quantized example.

[cgyurgyik]

We need LLVM to run this quantized example:

python3 frontends/relay/example.py quantized_net

Otherwise, we get:

...
TVMError: Check failed: bf != nullptr: target.build.llvm -keys=cpu is not enabled

Similar errors usually points to "Install LLVM", e.g. this and this as well.

1. Install LLVM. I had issues with this before, but it looks like everything works fine now with llvm-11.1.0 (on MacOS):

brew install llvm

2. Change the LLVM configuration in tvm/build/config.cmake:

set(USE_LLVM /usr/local/Cellar/llvm/11.1.0/bin/llvm-config)

3. Build TVM:
   Follow steps 3-7 in the TVM Relay frontend docs.

One thing I noticed was that with step 4,

 pip3 install --user dist/tvm-*.whl

I'm now getting the following error:

ERROR: tvm-0.7.dev1-cp37-cp37m-macosx_10_14_x86_64.whl is not a supported wheel on this platform.

I have no idea whether these are actually related. So instead, I followed the directions provided on the TVM Python Bindings page:

export MACOSX_DEPLOYMENT_TARGET=10.9  # This is required for mac to avoid symbol conflicts with libstdc++
cd python; python setup.py install --user; cd ..

This successfully installed the Python bindings.

Unfortunately, when I retry to run the quantized net example, I'm still getting the same error as above.

[sampsyo]

Wow! That seems annoying. IMO there are two ways to proceed:

• Is it really necessary to have LLVM to make this work? It's not clear to me why it would be necessary for this one but not others… I guess it's somewhere in the bowels of relay.quantize.quantize that's somehow invoking LLVM computation? Maybe there's some way to dig into that and to ask it to use a different execution engine to sidestep the problem.
• Install that dang wheel. The message "is not a supported wheel on this platform" seems to cover a lot of possibilities for what could be mismatched. Most prominently, it could the be Python version, if you happen (like most of us) to have multiple Pythons sitting around. cp37 means it's CPython 3.7—is that also what pip3 is referring to? Recall that you can always do python3 -m pip as an alternative to pip3 to be sure you're invoking the pip "for" a given Python. Upgrading pip itself with python3 -m pip install -U pip could also always help.

[cgyurgyik]

• Is it really necessary to have LLVM to make this work? It's not clear to me why it would be necessary for this one but not others… I guess it's somewhere in the bowels of relay.quantize.quantize that's somehow invoking LLVM computation? Maybe there's some way to dig into that and to ask it to use a different execution engine to sidestep the problem.

Hmm maybe not, I haven't actually dug into the code all that much. I'll spend some time peeking around, though frankly I'm not sure what screams "I need LLVM."

• Install that dang wheel. The message "is not a supported wheel on this platform" seems to cover a lot of possibilities for what could be mismatched. Most prominently, it could the be Python version, if you happen (like most of us) to have multiple Pythons sitting around. cp37 means it's CPython 3.7—is that also what pip3 is referring to? Recall that you can always do python3 -m pip as an alternative to pip3 to be sure you're invoking the pip "for" a given Python. Upgrading pip itself with python3 -m pip install -U pip could also always help.

Good point, thanks. This is probably a result of me using Python3.8 now. I'll have to downgrade to 3.7 again, and see if that works.

[cgyurgyik]

Edit: That was it! I had some conflicting Python issues. Starting a fresh build worked.

[sampsyo]

Yay; great news!

[rachitnigam]

Are there particular actions we need to take to merge this PR? Are we not planning to merge it?

[cgyurgyik]

Are there particular actions we need to take to merge this PR? Are we not planning to merge it?

Hmm good question. So without looking into it any further, running a quantized example requires LLVM. So some options are:

1. Leave this be until we actually get to quantization. I think a working VGG net ([Relay] Toward a working VGG Net (continued). #369) is a preliminary step.
2. Update the Relay frontend documentation in this PR to include LLVM installation if the user wants to run a quantized net example. This isolates this PR to simply adding the capability to run a quantized example.

[rachitnigam]

@sampsyo should we close this for now since there is no feasible path for progress towards a quantized example running?

[sampsyo]

OK yeah, sounds good. Figuring out serious quantization could be a component of a larger "next steps in TVM-to-Calyx" milestone.