Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Python] Regression - Generated code gets overwritten with empty contents when included in another .fbs file #8490

Open
akb825 opened this issue Jan 19, 2025 · 5 comments · May be fixed by #8500
Open

[Python] Regression - Generated code gets overwritten with empty contents when included in another .fbs file #8490

akb825 opened this issue Jan 19, 2025 · 5 comments · May be fixed by #8500

Comments

@akb825
Copy link
Contributor

akb825 commented Jan 19, 2025

After updating to 24.12.23, I noticed that many of the generated Python files have contents such as this:

# automatically generated by the FlatBuffers compiler, do not modify

# namespace: DeepSeaPhysics

# NOTE Axis.py does not declare any structs or enums

After seeing that they are generated in some cases but not others, I have tracked down the issue to using #include with another .fbs file, where any included types will have its Python files overwritten.

For example, let's say you have foo.fbs with the Foo table, and bar.fbs that includes foo.fbs. If you generate foo.fbs, a Foo.py file will be generated with the proper types. However, as soon as you generate bar.fbs, it will overwrite Foo.py with an empty file with a comment noting that it doesn't declare any structs or enums, rendering the generated code unusable.
@bagring
Copy link

bagring commented Jan 20, 2025

I confirm that. I think it is caused by the #8292.

@bagring
Copy link

bagring commented Jan 22, 2025

I have just found that you can avoid this using the --gen-all switch. You can just call
flatc -p --gen-all bar.fbs
and that's it. You do not have to generate the code from the foo.fbs anymore. The switch will ensure that the code will be generated also for all the files included in the bar.fbs.

@akb825
Copy link
Contributor Author

akb825 commented Jan 22, 2025

Using --gen-all may help in some situations, but can break in others.

One big issue is that if you need to generate multiple .fbs files in a build system that runs multiple processes in parallel, the same file could be written to multiple times simultaneously and corrupt the file or give filesystem locking issues on Windows. Another is if foo.fbs is in another project with a different namespace, you may end up outputting .py files in a separate directory you didn't want.

For some using --gen-all may be a way to unblock themselves without downgrading, but both of these issues are dealbreakers for me.

@bagring
Copy link

bagring commented Jan 22, 2025

Yeah, I am experiencing an issue similar to the ones you just mentioned :-(. I have the same table imported in two other .fbs files - once as a root_type and once as a regular table nested inside a different root_type. I have managed to fix the generator (somehow) so that it generates GetRootAs() and BufferHasIdentifier() only when the table is the root one. But now it depends on the order of generation :-(. The same file may get later overwritten by its alternative without the GetRootAs() and BufferHasIdentifier() functions :-(.

akb825 added a commit to akb825/flatbuffers that referenced this issue Jan 22, 2025
@akb825
Avoid outputting Python files for already generated types
d20de06 
This may overwrite types that have already been generated and can create
unwanted empty files. Fixes google#8490
@akb825 akb825 linked a pull request Jan 22, 2025 that will close this issue
Open
@akb825
Copy link
Contributor Author

akb825 commented Jan 22, 2025

I created #8500 to revert the portions of #8292 that caused the empty files to be output.

akb825 added a commit to akb825/flatbuffers that referenced this issue Jan 24, 2025
@akb825
Avoid outputting Python files for already generated types
fae7554 
This may overwrite types that have already been generated and can create
unwanted empty files. Fixes google#8490
akb825 added a commit to akb825/flatbuffers that referenced this issue Jan 25, 2025
@akb825
Avoid outputting Python files for already generated types
54f5806 
This may overwrite types that have already been generated and can create
unwanted empty files. Fixes google#8490
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Avoid outputting Python files for already generated types akb825/flatbuffers
2 participants
@akb825 @bagring