Constantine Murenin offered a history of the OpenBSD hardware sensors framework during his talk at BSDCan 2008, describing how it was originally based on a port from NetBSD, then evolved and was eventually ported to all the BSDs. He also discussed his own involvement with the framework, having ported it from OpenBSD to FreeBSD as a Summer of Code project, and how his port was merged into DragonFly BSD. At the end of the talk, there were some interesting ecxhanges between Constantine and Poul-Henning Kamp, the latter explaining why he'd had the code backed out of FreeBSD and why he continues to oppose it being merged back in.
Derived from notes taken at a one-hour BSDCan talk by Constantine Murenin, titled, OpenBSD Hardware Sensors Framework, Past and present history of the OpenBSD's hardware sensors framework.
Background
Constantine began by explaining that a sensor is any device with sensor-like data, measuring things like temperature and voltage. He noted many places where sensors can be found, then went on to examine the need for a sensors framework. He suggested that sensors are useful for preducting, detecting and troubleshooting sysytem failures, and noted that the unified interface provided by OpenBSD works out of the box with no configuration required. Finally, he commented that sensors are simply fun, offering as an example that it's interesting to compile code and see how it changes the CPU tempurarture.
He also referred to one of the latest sensors added, the sdtemp(4) SO-DIMM temperature sensor. Sensor support for this chip was added by OpenBSD founder Theo de Raadt before DIMMs using the chips were available, possible because Theo took the chip and manually soldered it onto one of his existing DIMMs.
Design Decisions
The overall design goals included keeping the framework simple, secure and usable. Another goal was to make things work by default, without complicated configurations. He noted that over-engineering would be usless, as many devices have incomplete specifications. And finally, he reiterated that the framework should have "No Buttons", so things just work without needing to be tweaked.
Constantine provided some technical details as to how voltage sensors work, and how they're read, noting that for many there is no documentation available. He then dropped into the framework API, pointing to /sys/sys/sensors.h and reviewing the sensor struct, and offering a code snippet showing how to add sensors in attach().
Sensor Tools
Constantine listed the following sensor tools, available in OpenBSD 4.3:
Many of the above tools were quickly discussed, along with code examples. It was noted that sensorsd has been improved since OpenBSD 4.2, no longer requiring manual configuration for "smart" sensors -- those that keep state. Other sensors still require minimal configuration, for example setting "temp:low=15C:high=65C" to specify a normal temperature range, along with possibly specifying a command to be executed when state changes.
Sensor Drivers in OpenBSD
Constantine went on to discuss the many sensor drivers found in OpenBSD 4.3, offering the following list:
He then offered a historical look at the sensor drivers in OpenBSD. He explained that the first 3 drivers, lm, it, and viaenv, were added in OpenBSD 3.4. The nslpcsio driver was added in OpenBSD 3.5, and the lmtempt driver was added in OpenBSD 3.6. Four more found their way into OpenBSD 3.8, aps, viasio, afte, and ses. The total number of drivers jumped up to 33 in OpenBSD 3.9, with a large number of i2c sensors added, along with IPMI and a drive status sensor. OpenBSD 4.0 introduced the first timedelta sensors, bringing the total to 42. In 4.1 there were 46 driver, 51 in 4.2, and 61 in the latest OpenBSD 4.3 release.
Detecting Chips
It was explained that when polling chips, many lack distinct signatures making it difficult to probe them and know which sensors you're talking to. He noted that in some cases the device can be looked up by matching an Open Firmware string, however not in all cases. When there's no Open Firmware string, /sys/dev/i2c/i2c_scan.c walks through all the values reported by sensors and creates a signature which is compared against values in i2c_scan.c. Constantine explained that in OpenBSD this scan functionality and all drivers are enabled by default, and thus the code is tested on all machines that have I^2C and very stable.
Unidentified messages are reported into dmesg. The OpenBSD project collects dmesg output from users voluntarily and anonymously mailing in their dmesg, with archives going back as far as 1998. He suggested that this archive made it possible for OpenBSD to support more sensors than even FreeBSD with a larger user base, because it's easy to find unsupported sensors and add support based on the register dumps collected by i2c_sna.c.
Sensor Drivers in NetBSD
Constantine reported that there are 31 drivers in NetBSD's -current tree compared to OpenBSD's 67 drivers. Even though the OpenBSD sensor framework was originally based on the NetBSD sensor framework, he suggested that the latter has a more complicated API and utilizes non-standard tools for the various sensors.
Sensor Framework Timeline
Envsys/sysmon were first introduced into NetBSD with the lm(4) and viaenv(4) drivers in 1999 or 2000. These drivers were ported to OpenBSD and simplified by Alexander Yurchenko (grange) for OpenBSD 3.4 on April 5th, 2003. The framework was improved by grange, dlg, kettenis, and deraadt over the next couple of years. On December 23rd, 2006 Constantine's patches were committed, converting the device drivers from one-level addressing to two-level addressing (e.g. hw.sensors.II to hw.sensors.lmo.temp2). Finally, in September of 2007 his Google Summer of Code project was completed and a patch was release for FreeBSD 7.0-CURRENT.
Porting the Framework to FreeBSD
Constantine ported the drivers framework to FreeBSD as part of a Google Summer of Code project in 2007. During the project, he ported the sensors API and docs, and ported two drivers from OpenBSD (lm and it) and the coretemp FreeBSD driver. He also ported the sysctl, sensorsd and systat userland apps. Throughout the effort, he fixed many smaller bugs that he found in the process, as well as a 10-year old OpenBSD bug and a 12-year old FreeBSD bug.
The patch was completed on September 13th, 2007, but the FreeBSD HEAD branch was still frozen. On September 25th, Constantine received an unexpected email from Hasso Tepper saying that his patch would be merged into DragonFly BSD. It was committed to DragonFly BSD on October 2nd, less than a month after his patch had been completed.
The patch was finally merged into FreeBSD 8.0 -current by Alexander Leidinger on October 14th the same week that the RELENG_7 branch was created and the FreeBSD code freeze ended. However, it was backed out the next day on the request of Poul-Henning Kamp, who complimented the efforts of the SoC project but questioned whether the framework was needed in FreeBSD, and whether the code had a FreeBSD feel.
A lengthy discussion followed, with many people on both sides of the argument. Constantine explained that the framework may not be necessary in FreeBSD due to some "sysctl magic" introduced by Poul-Henning Kamp. In OpenBSD sysctl's are static and you can't create random sysctl nodes on the fly, whereas in FreeBSD you can. He added that there were also questions about whether all of this should be done in userland rather than in the kernel.
In the end, Constantine still felt like he gained quite a bit of experience with the drivers, and was glad to see the code make it into DragonFly BSD. He noted that the patch is still available and should still apply.
Questions and Answers
Constantine opened the floor to questions. Poul-Henning Kamp got up and tried to better explain why he was opposed to the sensor framework being merged into FreeBSD. First, he suggest that the "framework" isn't really a framework, and can't be extended on the fly, instead requiring changes to sensorsd to add support for additional sensors. Second, he noted that time sensors don't belong with the others, that instead they should be handled by ntpd, "that they're used here is a prevailing architecture issue that I object to." Finally, his third complaint was that there hasn't been enough thought as to what a sensor actually is, and because of this much ends up in the kernel that would normally be handled in userland. He concluded by saying that the code is a very good prototype, but that he'd like to work with people to get it done right.
Constantine asked why coretemp(4) and k8temp(4) are in the kernel then, to which Poul-Henning Kamp acknowledged that they probably shouldn't be. He added that in his experiences with open source he's found that something adequate that is already merged is capable of preventing something that is decent or even good from ever being merged. "That why we didn't want to merge your patch, to increase the pressure to do it right. What you've done is adequate enough that it would prevent it from getting done right."
Constantine then pointed out that the framework was first written in 1999 or 2000 in NetBSD, "if no-one has done it right until now, do you really expect it will be done right any time soon?" Poul replied, "not doing it right is not an excuse for not doing it right." Constantine countered, "a lot of people use it and like it," to which Poul quickly retorted, "look at Windows."
Poul then went on to stress that calling it a framework is wrong because it does not have designed in extensibility, the application has to have intimate knowledge of every sensor it supports. He suggested calling it a "communication method" instead of calling it a "framework", at which point he'd be more willing to consider merging it. "It started out as sort of a hack, things were added, and one day you have Windows Vista architecture. No, this needs to be stopped, thought out carefully, and done right. Or you'll end up with Linux procfs. A bucket of random junk that nobody can find things in."
Poul continued, suggesting that there meaning needs to be given to the sensor values, "what do the numbers mean? It's 50 degrees hot, but what's the context? He also suggested that the values should be recorded in a machine readable format, and that a mechanism should be provided for determining what the values actually mean. He concluded, "I didn't say it would be easy".
product-fest?
I think Poul said "Or you'll end up with something like linux procfs." referring to the /proc mess on linux, not "product-fest"
Nice review btw,
that argument at the end of the talk was worth the whole trip to ottawa ;-)
thanks
Thanks for the correction, I've updated the article. It certainly makes more sense... and yes, the end of that exchange was certainly a highlight! :)
Adequate enough
s/would/could. OpenBSD (and probably others too) have more than once rewritten code simply because they didn't like the existing solution, even though it was adequate enough. ntpd and softraid comes to mind.
I'd rather have an imperfect solution that works, than not have it at all (unless the solution is a really horrible and ugly hack).
Re: Adequate enough
Which is the Linux approach, dressed up in the "release early" argument. However, as PHK pointed out, this leads to a situation where adequate but inelegant code persists in the kernel, because it's "just about good enough". And the net result is a morass of hacks and kludges, and a culture where one sub-par implementation is just replaced with another without pausing to design a decent solution.
Leads to ...
I agree, except that I excluded the worst cases by saying "unless the solution is a really horrible and ugly hack". There's always something in a system that could be improved.
I've never gotten sensors to work properly under Linux. I've read man pages, loaded modules, edited configuration files, and what not. In the end I just gave up.
Under OpenBSD I only have to type "sysctl hw.sensors". No configuration required. Perhaps the implementation isn't perfect, but from a users' point of view it couldn't be much simpler.
Can lead to ... The argument is a bit speculative, IMO. More than once have developers rewritten existing code not because it lacks functionality, but because they don't like it.
I wonder if this exchange
I wonder if this exchange means that PHK will get involved.