A Summit on the Summit
A handful of brave real-time developers decided to climb to the Glungezer summit in the Austrian Alps. Besides enjoying the hike and alpine scenery, the goal was to discuss current challenges and solutions to mainline the remaining -rt patches.
Christoph Hellwig and Thomas Gleixner had always though a conference in the alps was a great idea. After many years not materializing the idea, it became a running gag. This changed when I suggested to Christoph it would nice to have a conference somewhere else rather than a freezing chamber in a hotel bunker.
So Christoph and I teamed up to organise a small conference on a mountain. With Christoph’s familiarity with the Innsbruck mountains, the Glungezer lodge was chosen. The lodge is located almost on the top of the Glungezer summit, which is reachable by no road or cable car. John Kacur voiced the suspicion that Christoph had a secret plan to promote physical fitness in real-time developers by making them hike through the Austrian Alps. As it turns out, he was right.
On Friday, the 3rd of June, Steven Rostedt, Denise Rostedt, Jan Kizka, Christoph Hellwig, John Kacur, Michael Haberler, Jan Blunk, Ralf Ramsauer and I, met at the base of the mountain and took a taxi up to the Tulfeinalm where our hike began. Normally, the trails are mostly free of snow this time of year but the snow came late during this year’s spring. So, we had to find our way through snow patches, fog and misty scenery. Christoph excellently guided us to the lodge and we enjoyed the hospitality of our innkeeper who is an excellent cook.
The next day during breakfast, technical discussions began. Everyone seemed eager to get things done. Thomas arrived at noon, and our many questions to him increased the intensity of the discussions.
- Ralf presented the patch history of the real-time tree. Ralf developed the PaSta tool, that allows him to track individual patches and their progression through the trees. For example, he was able to track down that the swait (simple wait) patch lived for over 1500 days in the -rt tree before it recently made into mainline. Thomas quickly pointed out that the sleeping spinlock patch will beat that record easily. Overall, there was great interest in these statistics. Ralf also created a tree with all -rt releases https://github.com/lfd/PreemptRT
- Michael Haberler reported about a significant improvement to the machinekit project (). A long standing design issue was finally addressed, which improved the acceleration of a stepping motor by a factor of 2.5 in a real-time benchmark setup.
- I summed up my recent testing results of the swait-completion patch. The patch replaces the (complex) wait with the swait in the completion code. To our surprise, the numbers did not indicate any significant performance improvements. So, we investigated completion and swait and Christoph immediately pointed out two possibilities for optimization:
- First, almost all users of completion have only one waiter. The few that have more than one waiter are not performance critical, e.g., the suspend & resume code base. The idea is to support only one waiter and use (cmp)xchg only. This avoids enable/disable interrupts anymore via the spin_lock API. Obviously, the real complete_all() users need to maintain the waiter list themselves.
- Second, the complete_done() function is most of the time misused. So we aim to get rid of it altogether. We start with reviewing all complete_all() and complete_done() users and drop/rewrite the users.
- Christoph reported that xfs implements their own lock primitives to solve a fundamental problem: A lock protecting a ioblk transaction can be released in interrupt context. However in interrupt context, the owner of the lock is unknown, which foils mutex owner ship tracking by lockdep.
- On the one hand if an ioblk is prepared to be sent, the hardware takes over the control flow. Boosting a lock has no influence on the transaction speed. On the other hand if an ioblk is received from the hardware, the boosting of a lock makes perfectly sense.
The homegrown locking primitives from xfs are missing lockdep and priority boosting and Christoph would like to have generic support for unknown owner lock API with lockdep support. This would allow consolidation of a bunch of local implementations in various filesystems and in the blocklayer.
- Steven brought up cpu_chill() which is implemented as msleep(). The try_lock_*() for -rt depends on cpu_chill(). Without it, some code like the dentry() could starvate because under certain conditions try_lock_*() makes no progress.
mseep() is a clumsy workaround. Steven had proposed two different approaches to fix the cpu_chill() problem but none of them were really convincing Thomas. So another approach was discussed involving the above mentioned unknown owner locking API.
Soon it was dinner time and we were looking forward to an excellent meal. Overall it was extremely productive mini conference and everyone agreed we should do this again.
I would like to thank BMW Car IT for supporting the event and Christoph for supporting me in organizing it.