Server-side window shadows

[YaLTeR]

This is more important now that you can float windows, to make them stand out when overlapping.

Need to check what customization other compositors (swayfx, Hyprland) provide and how they handle irregular window shapes.

[VaclavC]

It would be nice to have semi-transparent mask over whole desktop in case of modal window alongside the actual shadow.

[YaLTeR]

There aren't really any modal windows or dialogs in niri at the moment. Either way, it's a separate issue from this.

[timgott]

As I have made https://github.com/timgott/wayfire-shadows but now often use niri, I can give you some tips:

• Read this: https://madebyevan.com/shaders/fast-rounded-rectangle-shadows/. I would consider it the standard (and optimal) method to draw gaussian shadows. You essentially only need to draw a rectangle with this shader and you have shadows.
• I have tried other implementations for fun (e.g. I did the math such that the light source is circular and not gaussian) but it is not worth it, gaussian approximates all other distributions very well
• I believe swayfx later implemented shadows the same way and referenced exactly the same article, and hyprland has some ad-hoc gradient as shadow that is not worth cloning (a gradient cannot have the right light distribution in the corners and looks a bit to dark there, though you probably don't notice that as much unless you look for it).
• To improve the contrast, you can make the box of the shadow a bit bigger than the window, e.g. grow the borders by 10 pixels. (reasons for doing this: The shadow is at only ~0.5 opacity at the borders of the rectangle because only half the light is blocked. Also a light source can have a bigger shadow than the object if it is close enough.)

[YaLTeR]

Thanks, that will be very helpful!