"It hasn't been a week, I know, and this is a pretty small set of changes since -rc7, but I'm going to be mostly incommunicado for the next week or so, so I just released what will hopefully be the last -rc," began Linux creator Linus Torvalds, announcing the 2.6.26-rc8 kernel. He added, "or maybe not. It depends on how good you all are while I'm not looking." Regarding the latest release candidate, Linus explained:
"Most of the bulk of the changes here are to Xen and to KVM in particular, which shows up as a rather unusual dirstat: 65% is in arch/x86 (counting the asm-x86 changes too). The rest is mostly random stuff, the appended ShortLog gives a reasonable idea. Several bugzilla entries are hopefully now closed."
"HP has released AdvFS, a file system that was developed by Digital Equipment Corp and continues to be part of HP's Tru64 operating system," announced Xose Vazquez Perez, offering a link to the re-licensed source code. 2.4 maintainer Willy Tarreau replied favorably, "wow! That's awesome. I discovered it in 1999 and 9 years later, it probably remains the most advanced FS I encountered." HP's Linda Knippers explained:
"In case its not clear, this is a GPLv2 technology release, not an actual port to Linux. We're hoping that the code and documentation will be helpful in the development of new file systems for Linux that will provide similar capabilities, and perhaps used to make tweaks to existing file systems."
Interesting features found in AdvFS include, "simplified file system and storage management; flexible multi-device storage pools shared by multiple file systems, with or without a volume manager; exceptional file system availability (no need to take file systems off-line to expand, shrink or reconfigure; snapshots for consistent backups while applications are on-line; ability to recover deleted files); wide range of performance management tools (fine grain control over file system and file placement within the storage pool; on-line rebalancing of files and free space across the storage pool; on-demand or background file and file system defragmentation); and transaction log management, allowing choices for logging metadata and data asynchronously or synchronously."
"As part of the Linux Foundation Technical board, we confront the issue of closed source Linux kernel modules all the time, and we wanted to do something that could be seen as a general 'public statement' about them that is easy to understand and point to when people have questions," began Greg KH, explaining, "so, after working on this for a while, and asking some of the other major contributors and maintainers of the kernel, what we have is below." a FAQ on the Linux Foundation website provides more background on the statement, which was undersigned by nearly 140 Linux kernel hackers. The statement reads:
"We, the undersigned Linux kernel developers, consider any closed-source Linux kernel module or driver to be harmful and undesirable. We have repeatedly found them to be detrimental to Linux users, businesses, and the greater Linux ecosystem. Such modules negate the openness, stability, flexibility, and maintainability of the Linux development model and shut their users off from the expertise of the Linux community. Vendors that provide closed-source kernel modules force their customers to give up key Linux advantages or choose new vendors. Therefore, in order to take full advantage of the cost savings and shared support benefits open source has to offer, we urge vendors to adopt a policy of supporting their customers on Linux with open-source kernel code."
"Another week, another -rc," began Linux creator Linus Torvalds, announcing the 2.6.26-rc7 Linux kernel, "and as usual, it's mainly drivers and arch updates - over 90% of changes are in one or the other." He continued:
"A big part of it (about two thirds of the driver update, in fact) is a late-dropping AGP/DRM update that adds support for some new Intel and ATI graphics cards. And a big part of the arch update is the inevitable def_config updates, of course. I'm not all that happy about the timing of the support for the new cards, but at the same time I also hate delaying new drivers. Obviously the hope is that it can't cause any regressions, since the added code is almost entirely for stuff that simply wasn't supported at all before."
Linus concluded, "if you ignore the driver and arch updates, the rest is pretty minor. About half is in networking, and half of the remaining is filesystems updates (mainly ocfs2). And random smatterings elsewhere, like some scheduler updates."
"In the kerneloops.org stats, a new oops is rapidly climbing the charts, began Arjan van de Ven, referring to his website where he automatically collects kernel oops and warning reports from mailing lists, bugzillas, and a special client. Regarding the latest oops, he noted, "the oops is a page fault in the ext3 'do_slit' function, and the first report of it was with 2.6.26-rc6-git3." Linux creator Linus Torvalds took a quick interest in the issue, observing that all the oopses seemed to be on the i686 architecture, suggesting, "could this perhaps be an indication that it is specific to i686 some way (eg a compiler issue?)"
Shortly before Linus sent out his emails, Dave Airlie confirmed that this was indeed a known compiler bug affecting GCC 4.3.1. The bug report notes, "any ext* filesystem which enables the dir_index feature is likely susceptible". Linus caught up on his email and retorted, "gaah. I should read all my email instead of wasting my time trying to match up the code with what I can reproduce.." The reason the Red Hat bug report wasn't automatically picked up by the kerneloops website was because the oops was reported in a jpeg image, leading Arjan to quip, "maybe one day if I'm really bored I'll implement OCR into [kerneloops.org] ;)".
"As promised, this cycle was short and the release is with only relatively small impact changes," said Git maintainer Junio Hamano, announcing the release of Git v1.5.6. He noted that both gitk and git-gui have been updated. To improve portability, when running "
git init", git now autodetects whether or not a filesystem is case insensitive, and updates a new configuration variable accordingly. Dependencies on the '
cpio' and '
curl' binaries have also been removed. Among the changes improving performance, the "
git clone" command has been rewritten in C. Other changes include:
git bisect help' gives longer and more helpful usage information; '
git branch' (and '
git checkout -b') can be told to set up branch..rebase automatically, so that later you can say '
git pull' and magically cause '
git pull --rebase' to happen; '
git cherry-pick' and '
git revert' can add a sign-off; '
git commit' mentions the author identity when you are committing somebody else's changes; '
git log' and friends learned the '
--graph' option to show the ancestry graph at the left margin of the output; '
git send-email' now can send out messages outside a git repository; '
git svn' learned --add-author-from option to propagate the authorship by munging the commit log message; new object creation and looking up in '
git svn; has been optimized."
"I regularly run and post various benchmarks comparing POHMELFS, NFS, XFS and Ext4, [the] main goal of POHMELFS at this stage is to be essentially as fast as [the] underlying local filesystem. And it is..." explained Evgeniy Polyakov, suggesting that the POHMELFS networking filesystem performs 10% to 300% faster than NFS, depending on the file operation. In particular, he noted that it still suffers from random reads, an area that he's currently focused on fixing. He summarized the new features found in the latest release:
"Read request (data read, directory listing, lookup requests) balancing between multiple servers; write requests are sent to multiple servers and completed only when all of them send an ack; [the] ability to add and/or remove servers from [the] working set at run-time from userspace; documentation (overall view and protocol commands); rename command; several new mount options to control client behaviour instead of hard coded numbers."
Looking forward, Evgeniy noted that this was likely the last non-bugfix release of the kernel client side implementation, suggesting that the next release would focus on adding server side features, "needed for distributed parallel data processing (like the ability to add new servers via network commands from another server), so most of the work will be devoted to server code."
"Oh great, not yet-another-kernel-tree, just what the world needs..." began Greg KH, continuing, "yes, this is an announcement of a new kernel tree, linux-staging." He explained:
"In a long and meandering thread with some of the other kernel developers a week or so ago, it came up that there is no single place for companies and developers to put their code for testing while it gets cleaned up for submission into the kernel tree. All of the different subsystems have trees, but they generally only want code that is about to go into this release, or the next one. For stuff that is farther off, there is no place to go. So, here's the tree for it."
In a readme created for the new tree, Greg adds, "the linux-staging tree was created to hold drivers and filesystems and other semi-major additions to the Linux kernel that are not ready to be merged at this point in time." He also requested that the new tree be included in Linux -next, leading Theodore Ts'o to ask, "does this mean that the nature of linux-next is changing? I thought the whole point of linux-next was only to have what would be pushed to Linus in the near future, so we could check for patch compatibility issues." Greg explained that he was hoping for an exception for his new -staging tree as it only includes whole new drivers and filesystems, not changes to existng features, "there is stuff that users can use to get hardware to work that currently is not supported on kernel.org kernels at all." As an example he noted, "there are 2 big network drivers in there that support a wide range of devices that some people would like to see working :)"
"I'd like to say that the diffs are shrinking and things are calming down, but I'd be lying," began Linux creator Linus Torvalds, announcing the 2.6.26-rc6 kernel. He noted, "another week, another -rc, and another 350 commits. Yes, the diff is smaller than the one from rc4 to rc5 (despite having more commits), so we're on the right trajectory, but I was hoping for less churn at this stage." Linus continued:
"As usual, most of the changes are to drivers (with arch updates a strong second). The DVB updates are the biggest chunk of that, but on the whole it's quite spread out. As mentioned, the diffs are smaller and there are more commits, and yes, most of the commits are really rather small and trivial fixes.
"Give it a try, we should have a few less regressions once more,"
"These patches allow data integrity information (checksum and more) to be attached to I/Os at the block/filesystem layers and transferred through the entire I/O stack all the way to the physical storage device," began Martin Petersen. He went on to explain, "the integrity metadata can be generated in close proximity to the original data. Capable host adapters, RAID arrays and physical disks can verify the data integrity and abort I/Os in case of a mismatch." He noted that support currently only exists for SCSI disks, but that work is underway to also add support for SATA drives and SCSI tapes, "with a few minor nits due to protocol limitations the proposed SATA format is identical to the SCSI". Explaining how this works, Martin continued:
"SCSI drives can usually be reformatted to 520-byte sectors, yielding 8 extra bytes per sector. These 8 bytes have traditionally been used by RAID controllers to store internal protection information. DIF (Data Integrity Field) is an extension to the SCSI Block Commands that standardizes the format of the 8 extra bytes and defines ways to interact with the contents at the protocol level. [...] When writing, the HBA (Host Bus Adapter) will DMA 512-byte sectors from host memory, generate the matching integrity metadata and send out 520-byte sectors on the wire. The disk will verify the integrity of the data before committing it to stable storage. When reading, the drive will send 520-byte sectors to the HBA. The HBA will verify the data integrity and DMA 512-byte sectors to host memory."
In a series of seven patches, Arnd Bergmann proposed adding in-memory write support to mounted cramfs file systems. He explained, "the intention is to use it for instance on read-only root file systems like CD-ROM, or on compressed initrd images. In either case, no data is written back to the medium, but remains in the page/inode/dentry cache, like ramfs does." Reactions were mixed. When Arnd suggested this as an alternative to using the more complex unionfs to overlay a temporary filesystem over a read-only file system, and that similar support could be added to other file systems, it was pointed out that there was ultimately more gained by focusing on a single solution that worked with all filesystems. David Newall stressed, "multiple implementations is a recipe for bugs and feature mismatch." Erez Zadok suggested, "I favor a more generic approach, one that will work with the vast majority of file systems that people use w/ unioning, preferably all of them." He went on to add that more gains would be had from modifying the union destination filesystem rather than multiple source filesystems. Arnd agreed in principle, but noted it would add complexity. He indicated that he'd explore the idea further, then explained:
"My idea was to have it in cramfs, squashfs and iso9660 at most, I agree that doing it in even a single writable file system would add far too much complexity. I did not mean to start a fundamental discussion about how to do it the right way, just noticed that there are half a dozen implementations that have been around for years without getting close to inclusion in the mainline kernel, while a much simpler approach gives you sane semantics for a subset of users."
Jakub Jelinek announced the availability of GCC 4.3.1 saying, "GCC 4.3.1 is a bug-fix release, containing fixes for regressions in GCC 4.3.0 relative to previous GCC releases." He adds the standard tag, "as always, a vast number of people contributed to this GCC release -- far too many to thank individually!"
Constantine Murenin offered a history of the OpenBSD hardware sensors framework during his talk at BSDCan 2008, describing how it was originally based on a port from NetBSD, then evolved and was eventually ported to all the BSDs. He also discussed his own involvement with the framework, having ported it from OpenBSD to FreeBSD as a Summer of Code project, and how his port was merged into DragonFly BSD. At the end of the talk, there were some interesting ecxhanges between Constantine and Poul-Henning Kamp, the latter explaining why he'd had the code backed out of FreeBSD and why he continues to oppose it being merged back in.
"Another week, another batch of mostly pretty small fixes. Hopefully the regression list is shrinking, and we've fixed at least a couple of the oopses on Arjan's list," said Linux creator Linus Torvalds, announcing the 2.6.26-rc5 kernel. He added, "as usual, the bulk of the changes are in drivers and arch code - together they are about 70% of the diffstat. And the arch stats are bloated by some new/updated SH and avr defconfig files, which is also fairly common at this stage." Linus concluded:
"Perhaps unusually, 13% is in kernel/, almost all of it fixing up some scheduler issues - with the bulk of it by far being a couple of reverts due to performance regressions. But there's a few other fixes too. And then there is networking and some ocfs2 updates. Along with various one-liners sprinkled all around.
"The shortlog (appended) gives a reasonable view of it all. Nothing hugely exciting sticks to my mind, but then I don't think we've had any really hugely exciting problem spots either.."
Tony Luck offered some statistics focused on the frequency of developers that only contribute to the Linux kernel one time, "I skimmed through looking for drive-by contributors (defined as someone who contributes to just one release and is then never heard from again)." Starting with the 2.6.11 kernel, he suggested the following numbers: "63 [developers contributed patches] in version 2.6.11 [and then were] never seen again, 148 in version 2.6.12, 128 in version 2.6.13, 92 in version 2.6.14, 96 in version 2.6.15, 122 in version 2.6.16, 137 in version 2.6.17, 140 in version 2.6.18, 135 in version 2.6.19, 95 in version 2.6.20, 136 in version 2.6.21, 153 in version 2.6.22, 179 in version 2.6.23, 179 in version 2.6.24, and 304 in version 2.6.25".
It was pointed out that the statistics were artificially inflated due to name misspellings and other variations, and that many of the listed people are actually still working with the Linux kernel. Greg KH added, "well, you do know that the distribution of all of our users are: 50% only contributed 1 patch; 25% contributed 2; 12% contributed 3; 6% contributed 4 and so on?" He went on to point out:
"Our curve is leveling out much better now though. For the whole 2.5 release, the top 30 people did over 80% of the work. Now, the top 30 people are doing 30% of the work. So it is getting much better, as long as we still continue to keep our massive rate of change that we have going, and huge number of developers, we should be fine. So this list doesn't necessarily mean anything is wrong, only that 50% are one-time contributors. And I think that shows we are easy to get a change into our tree from just about anyone, not that we are driving people away."