volk 3.1.1 regression test failure

[chenrui333]

👋 trying to build the latest release, but run into some build issue. The error log is as below:

error build log

  ==> /opt/homebrew/Cellar/volk/3.1.1/bin/volk_profile --iter 10
  RUN_VOLK_TESTS: volk_64u_popcntpuppet_64u(131071,10)
  generic completed in 0.712 ms
  neon completed in 0.643 ms
  Best aligned arch: neon
  Best unaligned arch: neon
  RUN_VOLK_TESTS: volk_16u_byteswappuppet_16u(131071,10)
  generic completed in 0.619 ms
  neon completed in 0.273 ms
  neon_table completed in 0.136 ms
  Error: volk: failed
  An exception occurred within a child process:
    BuildError: Failed executing: /opt/homebrew/Cellar/volk/3.1.1/bin/volk_profile --iter 10

full build log, https://github.com/Homebrew/homebrew-core/actions/runs/7704559273/job/20997188097?pr=161268
relates to Homebrew/homebrew-core#161268

[chenrui333]

this is my local debug run

$ lldb /opt/homebrew/Cellar/volk/3.1.1/bin/volk_profile
(lldb) target create "/opt/homebrew/Cellar/volk/3.1.1/bin/volk_profile"
Current executable set to '/opt/homebrew/Cellar/volk/3.1.1/bin/volk_profile' (arm64).
(lldb) run --update
Process 74547 launched: '/opt/homebrew/Cellar/volk/3.1.1/bin/volk_profile' (arm64)
No prior test results found ...
RUN_VOLK_TESTS: volk_64u_popcntpuppet_64u(131071,1987)
generic completed in 74.997 ms
neon completed in 117.34 ms
Best aligned arch: generic
Best unaligned arch: generic
RUN_VOLK_TESTS: volk_16u_byteswappuppet_16u(131071,1987)
generic completed in 30.298 ms
neon completed in 51.535 ms
neon_table completed in 27.018 ms
Process 74547 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=2, address=0x100044000)
    frame #0: 0x0000000183b896f8 libsystem_platform.dylib`_platform_memmove + 168
libsystem_platform.dylib`:
->  0x183b896f8 <+168>: stp    q2, q3, [x0]
    0x183b896fc <+172>: subs   x2, x2, #0x40
    0x183b89700 <+176>: b.ls   0x183b8971c               ; <+204>
    0x183b89704 <+180>: stp    q0, q1, [x3]
Target 0: (volk_profile) stopped.

[argilo]

Thanks for the bug report.

From the log, it would appear the crash occurs in volk_16_byteswap_u_orc. That protokernel was previously inaccessible, and re-enabled in #719. It runs correctly in CI and locally on a Raspberry Pi, but it would seem there is trouble on Apple M1 for some reason.

[argilo]

@chenrui333 I opened #747 which I expect will fix this. Are you able to test it on your system?

[jdemel]

I added a test with FlyCI. It builds on their systems with an M2:
https://github.com/jdemel/volk/actions/runs/7717048987/job/21035255427

[argilo]

It looks like that built without ORC support:

ORC support not found, Overruled arch orc

[jdemel]

It looks like that built without ORC support:

ORC support not found, Overruled arch orc

It's time to figure out how we enable ORC on MacOS.

[argilo]

GitHub runners have Homebrew, so you could probably do brew install orc.

[chenrui333]

@chenrui333 I opened #747 which I expect will fix this. Are you able to test it on your system?

yeah, totally, testing it now.

[chenrui333]

trie to apply the patch, the build was not successful this time

[ 20%] Generating volk_32f_s32f_add_32f_a_orc_impl.c
cd /tmp/volk-20240131-22114-h6wzw/volk-3.1.1/build/lib && /opt/homebrew/bin/orcc --include math.h --implementation --include volk/volk_complex.h -o /tmp/volk-20240131-22114-h6wzw/volk-3.1.1/build/lib/volk_32f_s32f_add_32f_a_orc_impl.c /tmp/volk-20240131-22114-h6wzw/volk-3.1.1/kernels/volk/asm/orc/volk_32f_s32f_add_32f_a_orc_impl.orc
gmake[2]: *** [lib/CMakeFiles/volk_obj.dir/build.make:89: lib/volk_16u_byteswap_a_orc_impl.c] Error 1

[argilo]

You'll need to run cmake again (and maybe clear out the build directory if that doesn't work).

[chenrui333]

I realize I have to also drop orc dependency, the build works fine for me now. Thanks folks! Closing this issue. :)

[argilo]

I realize I have to also drop orc dependency

Why?

[chenrui333]

I realize I have to also drop orc dependency

Why?

HAVE_ORC detection?

[chenrui333]

You'll need to run cmake again (and maybe clear out the build directory if that doesn't work).

we always do the clean build, so that actually does not apply.

[argilo]

Hmm, if VOLK fails to build with ORC present that sounds like a bug. Could you open an issue for that?

[argilo]

Also, how did you build earlier when you encountered the error in volk_profile? Was ORC enabled at that time?

[chenrui333]

Hmm, if VOLK fails to build with ORC present that sounds like a bug. Could you open an issue for that?

totally. this is the success https://github.com/Homebrew/homebrew-core/actions/runs/7727239095/job/21065529883

[chenrui333]

Also, how did you build earlier when you encountered the error in volk_profile? Was ORC enabled at that time?

yes, that was when the orc present (do I need to explicitly enable any option to turn it on?) this is the build log, https://github.com/Homebrew/homebrew-core/actions/runs/7704559273/job/20997188097?pr=161268

[argilo]

do I need to explicitly enable any option to turn it on?

I don't think so. It should be automatically detected.

[argilo]

I'm not sure disabling ORC is a great idea.

Are you sure that you correctly applied the patch from #747 when you tested it? It builds fine for me, and also in our CI. Note that the patch deletes kernels/volk/asm/orc/volk_16u_byteswap_a_orc_impl.orc.

[chenrui333]

Are you sure that you correctly applied the patch from #747 when you tested it? It builds fine for me, and also in our CI. Note that the patch deletes kernels/volk/asm/orc/volk_16u_byteswap_a_orc_impl.orc.

yeah, I did, you can see this linked PR, Homebrew/homebrew-core#161268

[argilo]

Can you show me the build that failed with just the patch in place (and not the ORC removal)?

I found only this one, but it appears that it was cancelled: https://github.com/Homebrew/homebrew-core/actions/runs/7727191280

The PR #747 was only intended to locate the source of the problem, so I don't think it is a good idea for Homebrew to ship it. Since Homebrew has disabled ORC, the patch does nothing.