Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parallelism for secret search #832

Open
KirillOsenkov opened this issue Nov 4, 2024 · 4 comments
Open

Parallelism for secret search #832

KirillOsenkov opened this issue Nov 4, 2024 · 4 comments

Comments

@KirillOsenkov
Copy link
Owner

One todo would be to investigate adding parallelism to secret detection because right now it's really slow on real-life binlogs, even moderately sized. Takes over two minutes on a 5 MB binlog I use often. I think a judicious use of Task.Run() will really make things better here. You can maybe spawn one task per string, and one task per file, then just foreach the tasks and do task.Wait() on each of them. No need for async as only one thread will be blocked (the one where the computation happens)

@JanKrivanek
Copy link
Collaborator

Once improved - what would you think about spinning it on background for all opens and then showing some form of info (node in the tree, goldbar, ...) if sensitive data found (except probably username)? It might feel too heavy - but preventing spilling secrets unintentionaly feels worth it.

@KirillOsenkov
Copy link
Owner Author

definitely too heavy, can't have it burn CPU and battery and interfere with the investigation. Needs to be an explicit gesture from user.

@JanKrivanek
Copy link
Collaborator

I'm sad about it - but it makes perfect sense

@KirillOsenkov
Copy link
Owner Author

also from looking at the current perf this seems like it will be very expensive computationally, we should get some data for larger binlogs (and I can share some large binlogs if you need them)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants