Interested in creating epoll patch?

[mspanc]

Hello,

I am building a network app that strongly relies on GLib/GStreamer stack. However, I would like to give it a boost by switching to epoll. I've noticed abandoned epoll branch in your repo.

Would you have been interested in finalizing this project and making working, stable version of GLib with epoll? I can sponsor this somehow.

[mzabaluev]

@mspanc
The conflicting changes do not seem to be much divergent from the last time I was doing this.
Do I understand correctly that you need that branch ported to the latest stable branch of GLib?
What kind of poll size is expected in your application, is it large enough for the changes to have noticeable effect?

[mspanc]

Hello @mzabaluev

I want to run streaming service based on gst-rtsp-server and it can easily hit 10-50k simultaneous connections in a peak so I think that's already range in which regular poll will be a bottleneck (correct me if I am wrong).

So yes, I am basically interested in having a fork of GLib that has enabled epoll.

Have you tried to merge this into main GLib code base? If yes what was the reason it is not a part of GLib? Are there any reasons why GLib should stick to poll? (i don't see any)

[mzabaluev]

Have you tried to merge this into main GLib code base? If yes what was the reason it is not a part of GLib?

It has been submitted as GNOME bug 699132, but nobody has seen it necessary to review the patch. I guess there is no mainstream need for it, as GLib is mostly used in application stacks.

Are there any reasons why GLib should stick to poll?

If it ain't broken, don't fix it :) poll is portable across POSIX systems, and it's been working well enough for most usages of GLib.

[mzabaluev]

After rebasing to recent master, a few regression tests fail. Will investigate.

[mzabaluev]

A regression test added since the branch was last submitted exposes an API break: if multiple polls were added on one descriptor with different priorities and different event masks, the priority in g_main_context_query decides which event mask the corresponding output GPollFD will get. The code on the pollers branch does not preserve the original poll entries' priorities.

Fixing this would require more work than I'm ready to do on my own time currently, I'm afraid. Pity that such a corner case gets in the way, because g_main_context_query in that branch is no longer used in GLib's implementation of the poll loop, and is only needed for integration with foreign event loop implementations (which the branch aims to solve in a different way).

[mspanc]

Well I don't mind if this fork won't be 100% compatible.

[arteymix]

I work on Valum, a web micro-framework based on libsoup-2.4 and gio-2.0 and I would really like to see epoll eventually powering GLib.

@mzabaluev What kind of issues have you encountered when you tried to rewrite GMainContext upon epoll? is maintaining priorities per file descriptor such a pain?

In the meantime, it is still possible to plug a polling function into GMainContext, so I though I could wrap epoll with a file descriptor cache. How well do you think it would perform?

[mzabaluev]

I have pushed branches:

• pollers, rebased against recent master;
• pollers-2-46, rebased against glib-2-46.

@arteymix

is maintaining priorities per file descriptor such a pain?

It should be doable: the members of pollfd lists in GPollBin could include the original priorities, to be considered when composing the output array of g_main_context_query.

In the meantime, it is still possible to plug a polling function into GMainContext, so I though I could wrap epoll with a file descriptor cache. How well do you think it would perform?

There's expectation that a GPollFunc callback is able to poll an arbitrary set of file descriptors passed into the function. Any implementation would have to scan the entire input array, which would negate part of the overhead reduction provided by epoll_wait.

[Akagi201]

@mzabaluev Does this glib use epoll by default? I have successfully built the glib source modified here, and my glib application runs successfully, not modifying any code. Does my glib application use epoll already?

[arteymix]

@Akagi201 use strace <the binary> if you want to make sure

[Akagi201]

@arteymix It is poll ....... :(, so how can I switch to epoll.

[arteymix]

The easiest way is to point the LD_LIBRARY_PATH variable where the shared library is built:

LD_LIBRARY_PATH=<glib source dir>/.libs <the binary>

[Akagi201]

@arteymix I already complile and build my source code with this glib, but I want to know how can I modified the application code to use epoll.

[arteymix]

I've just compiled the whole thing and it's effectively not using epoll at this moment. Maybe ping @mzabaluev ?

[Akagi201]

@arteymix I found the problem. You must run LD_LIBRARY_PATH=<glib source dir> <the binary> will use the dynamic glib you build, or it will the system default glib which use poll.

[Akagi201]

How can I do some benchmark tests between poll and epoll implementations? Does anyone have some ideas?

[arteymix]

@Akagi201 You could use wrk and measure the latency as the number of connection increases along with libsoup-2.4? Or implement a basic web server for that purpose, it's pretty easy with GSocketService.

[Akagi201]

I made a PR to merge this patch to the latest glib.

When I run make check, I got some errors:

Akagi201/glib-epoll#1

[yin-zhang]

Would this version work with recent version of glib? Am playing with janus-gateway, a glib-based WebRTC gateway. Profiling shows a lot of overhead lies in glib. Am hoping to see if glib + epoll can help reduce CPU consumption.

[xunlujiehuo]

Would this version work with recent version of glib? Am playing with janus-gateway, a glib-based WebRTC gateway. Profiling shows a lot of overhead lies in glib. Am hoping to see if glib + epoll can help reduce CPU consumption.

Hello, seniors, I have encountered the same problem as you now. I am using janus-gateway, and the cpu overhead of glib is too high. I would like to ask you how to solve it in the end.

[mzabaluev]

@xunlujiehuo I suggest you report your observations to the GLib project.

The ideas on how to support epoll in GLib have probably evolved, but I have not followed up on the development.
The original issue is now tracked at https://gitlab.gnome.org/GNOME/glib/-/issues/693 and is still open.