How Kunquat achieves consistent UI look across display configurations

Most programmers dealing with user interfaces understand that it is not a good idea to use pixels as measurement units when specifying sizes of UI elements. Still, I believe most of us targeting desktop platforms have grown not to care about it too much. I also believe that, for most of the history of desktop applications, the increase in common display resolution has been so gradual that we have afforded not to care, and mobile devices have been different enough to warrant an entire design overhaul when developing for those platforms.

In the desktop world, however, things changed dramatically at the introduction of displays with pixel densities similar to mobile device displays. The increase in resolution variety has been fast enough that those of us who have gone for the higher resolutions could easily encounter usability problems with existing software. Most of the issues can be worked around with compatibility modes, but these often come with a loss of drawing quality and only provide coarse-grained scaling.

After I got my first UHD display (with approx. 284 pixels per inch) in the autumn of 2018, this is what I saw when I first launched Kunquat:

Ouch.

After some investigation I soon discovered some compatibility modes in Qt that helped make the interface look a bit more user-friendly, but it was clearly just a temporary solution at best:

(Click the image for a better view of the drawing bugs.)

All my custom widgets were rendered in lower resolution and were then scaled up, and even then some of the scales were way off. Also, the compatibility modes have very coarse-grained scaling support: usually you can only choose to scale the interface by an integer multiple. This makes sense when consistency in drawing is a concern, but it can easily lead to situations where one scale is too small and the next one available is too large.

I was also surprised that I couldn't find proper support for large variety of screen resolutions in Qt yet. Based on what I could find, for instance, the icon system still assumes that the interface uses a rather limited set of pixel sizes for icons, and most default dialogs and windows have obvious scaling bugs even if I set the font size correctly for my display. I am also not very fond of the approach taken by Qt in supporting "high DPI" in some of their UI elements, because it oversimplifies the problem. To be fair, these may very well be intended for easier migration of existing applications, not to be used as definitive solutions for properly scaled interfaces.

It is entirely possible that I simply don't know how to configure my settings properly to make Kunquat (and all the other desktop applications) look right. However, I did spend several days investigating these problems, and I figured that if it's this complicated to set things up correctly, I can't assume that other potential Kunquat users will go through the trouble. So, my goal was to make Kunquat look good on a desktop system out of the box without relying on a number of obscure configuration parameters.

Since I was going to rely on my own implementations more, I also had more responsibility. Previously, I had relied somewhat on the Qt style system so that users who needed to adjust the appearance could do so at the Qt level. Now I had to provide at least enough flexibility to allow the user to scale the interface in case the defaults I provide are not suitable for their needs.

I decided to give the user a single setting for controlling the scaling of the UI: the default font size. Almost every size and margin in the UI is scaled based on this setting, including non-text elements like icons, envelope nodes and the peak level meters. The only exceptions are the default window sizes, which are relative to screen dimensions. The whole tracker interface also reacts to changes in font size instantly — no restarting required!

Main screen of Kunquat Tracker with various font size settings on a UHD desktop. Notice how everything is scaled according to the font size, including icons and line widths. The largest window also demonstrates some minor drawing bugs that haven't been ironed out yet.

My initial plan was to set a fixed DPI value for default font size and let users adjust it manually if their system has an incorrect DPI setting. However, it turned out that I can be a bit more helpful, as Qt provides two different DPI values: the "physical" DPI, which is based on actual physical dimensions of the screen and the screen pixel size, and "logical" DPI, which is based on the DPI setting in the system. Therefore, I could calculate how much I need to scale the initial default font size so that it matches the actual physical font size I want. So, in the end, I was happy to discover that I could achieve a fairly consistent initial look of Kunquat even when the system DPI setting does not match reality.

Eventually I realised that I need to take variations in viewing distance into consideration when choosing the default interface size. I decided to assume that larger screens are probably viewed from greater distance, so I scale the default font size to a certain extent based on physical screen dimensions.

Performance benefits of rendering audio in larger chunks

I recently ran into performance issues in Kunquat as I was working on some new music (hopefully not too many months away from release). Large bursts of near-simultaneous notes caused load peaks high enough to cause audio drop-outs even on my high-performance laptop. I am going to explain the performance bottleneck and the way I was able to alleviate the problem.

Understanding the performance issue requires some background knowledge of the Kunquat audio rendering system. The playback logic is divided into two main systems: the sequencer and the signal processing system. The sequencer is responsible for reading event triggers (i.e. notation) from pattern data and processing the events at the right times. The signal processing system keeps track of internal states of all the processors that create or modify signal data (these states include information such as current waveform phase, sample position, filter states, etc.). The signal processing system also produces the final audio output by combining all the signal data.

In principle, the sequencer of Kunquat splits the pattern data into slices that may only contain event triggers at the very beginning:

An example of splitting notation data into four slices at locations where event triggers are found. Observe how the notes F5, B♭2 and F3 span multiple slices.

The basic process of converting each slice into audio is fairly straightforward: first process all the event triggers at the beginning of the slice, and then use the signal processing system to produce audio for the duration of the slice.

A key performance characteristic of the Kunquat audio rendering system (and just about any complex audio synthesiser) is the significant amount of overhead associated with each call of the signal processing system. Whenever we request more audio data, the system needs to run a lot of set-up code in order to calculate new signal inputs, traverse the internal data structures, and find out how to continue audio rendering after the previous call. Therefore, it takes less work to render a single large chunk of audio in one call than render the same amount of audio in several small chunks.

Now, consider what happens when we render the following section which might represent a guitar strum:

A zoomed-in representation of a guitar strum, and the way Kunquat 0.9.2 and earlier versions divide signal processing work.

In this case, rendering of each note is interrupted at locations where the sequencer needs to process the next note, causing additional overhead. Furthermore, each of these events would also interrupt processing of other notes played at the same time with different instruments, slowing audio processing down even further.

However, we can process this section much more efficiently. In Kunquat, a note playing in one channel cannot alter the state of notes playing in other channels. Therefore, it is possible to process the audio associated with each channel separately, and we can generate the audio with fewer splits:

The new approach in dividing signal processing work in Kunquat.

In practice, modifying the sequencer of Kunquat to support this more efficient slicing strategy was not straightforward. First, many event types in Kunquat notation affect more than the state of a single channel (e.g. tempo adjustments and instrument-specific commands), in which case we still need to introduce a global breakpoint in signal processing. Second, the event system of Kunquat can generate almost any event on the fly based on current environment state, which means we cannot easily preprocess the pattern data into a more convenient form. Finally, audio rendering of Kunquat is designed to operate without memory allocations, which causes a number of issues in resource management.

In the end, I believe the performance gains justify the increased complexity in the sequencer. Here is a comparison of time usage with the most demanding use case that I currently have:

Performance comparison between old and new slicing strategies (single-threaded). Around the -50 second mark is the peak utilisation of the signal processing system (over 800 calls to signal processing units during a single audio data request!).

While the comparison certainly looks impressive, the graphs are also misleading. This optimisation specifically targets the highest peaks in computation time while leaving the rest of the performance characteristics mostly unaffected. In any case, the benefits for real-time playback are clear.

In conclusion, maximising the amount of audio data you produce in a single pass is one of the most important optimisations you can make in your audio system. The impact may not be obvious in profiler output, but it is significant where performance matters the most.