Planet Linux Plumbers Conf

This puzzle came up at a recent ISO SC22 WG14 standards–committee meeting (for the C language).

Suppose you have an array of three RCU-protected structures. At any given time, one of them is the current structure that will be used by RCU readers. This means that there is a global pointer that will be pointing to one of the elements of this array, thus designating it as the current element. (And yes, a grace period must elapse before a given element is reused.)

But suppose that there is another global integer whose value must be kept consistent with the contents of the current element — to keep things trivial, let's assume that the value of this global integer must be twice that of an integer within the current structure.

The data structures might be set up as follows:

struct rcu_protected {
  int a;
};
struct rcu_protected elements[3];
struct rcu_protected *current = &elements[0];
int consistent;  /* must be 2 * current->a */

How can this be accomplished?

The Inquirer certainly thinks so.

However, I'm not so sure it makes sense to compare Infiniband to an as-yet-unannounced optical external QPI. QPI is currently a processor interconnect. CPUs, RAM, and devices connected by it are conceptually part of the same machine -- they run a single OS, for example. They are both "networks" or "fabrics" but they have very different design trade-offs.

Another widely-used bus in the system is closer to Infiniband than QPI -- PCI Express. Isn't it more likely that PCIe could take on IB? There are companies already who have solutions that use external PCI Express for cluster interconnect, but these have not gained significant market share. Why would QPI, a technology whose sweet spot is even further from Infiniband's than PCIe, be able to challenge Infiniband? It's hard to speculate without much information, but right now it doesn't seem likely to me.

The other prediction made in the article is that Intel's 10GbE iWARP card could squeeze IB on the low end, due to its greater compatibility and lower cost.

It's definitely never a good idea to bet against Ethernet when it comes to mass-market, commodity networking. Ethernet will win. 10GbE will win. But, there are now two competing ways to implement the low-latency RDMA Verbs interface on top of Ethernet. iWARP is essentially RDMA over TCP/IP over Ethernet. The new alternative is IBoE (Infiniband over Ethernet, aka RoCEE, aka "Rocky"). This encapsulates the IB packet protocol directly in the Ethernet frame. It loses the layer 3 routability of iWARP, but better maintains software compatibility with existing apps that use IB, and is simpler to implement in both software and hardware. iWARP has a substantial head start, but I believe that IBoE silicon will eventually be cheaper, and more likely to be implemented in commodity Ethernet hardware.

I think IBoE is going to take low-end market share from traditional IB, but I think this is a situation IB hardware vendors have no problem accepting. Commoditized IBoE NICs invite greater use of RDMA features, and when higher performance is needed, customers can upgrade to "real" IB, maintaining IB's justification for higher prices. (IB max interconnect speeds have historically been 2-4x higher than Ethernet, and I don't see that changing.)

(ObDisclosure: My current employer now sells IB hardware. I previously also worked at Intel. My opinions are my own, duh.)

If you use git on a daily basis like I do, git-completion.bash is a great way to make your life a little easier. While I guess it does add tab-completion for git commands, the most useful feature for me is the ability to put the current branch into the cmdline prompt. Now that I am comfortable working with multiple git branches and remotes, a little reminder where I am prevents time-consuming mistakes. git-completion.bash lives in git's git tree.

1. git clone git://git.kernel.org/pub/scm/git/git.git
2. copy git/contrib/completion/git-completion.bash to ~/.git-completion.sh
3. Follow the instructions in the file to set up, and enable showing branch in $PS1

I also use this alias in my ~/.gitconfig, which is convenient:
[alias]
        log1 = log --pretty=oneline --abbrev-commit

Have fun!

I have been a happy vi user for almost three decades. My initial choice of vi over emacs was quite simple: the machine that was available at the time could run about ten vi sessions, but only one emacs session. Given that this was a shared machine, use of emacs was therefore socially irresponsible. Given a more capable machine, perhaps I would instead be a happy emacs user. But by now, vi is ingrained firmly in my fingers. Or rather, those features of vi that have been around for some decades are ingrained — I still haven't figured out what the g command does, aside from trip me up when I don't quite hit the shift key firmly enough. I am sure that there is some comprehensive vi documentation out there somewhere, but when I search for it, I only find web pages that helpfully offer tutorials on vi features that I have known about for more than a quarter century.

As a result, these days I mostly learn vi by making mistakes. For example, yesterday I downloaded an ASCII-text web page and edited it with vi. The startup message from vi looked very strange, but life was moving especially quickly that morning, and vi did show me the desired page, so I moved on.

This morning, I saw the same strange startup message. I got done with my vi session, and looked at the file I had downloaded to see if there was anything strange about its permissions or content. There was in fact something very strange, namely that it didn't exist at all.

The mistake I had made was to give the URL to vi instead of to wget. Nevertheless, vi happily downloaded the web page to /tmp and let me edit it. So, to see one of my recent patches, instead of using a browser, I can type:

vi http://www.rdrop.com/users/paulmk/patches/2.6.34-rc3-rcu-5.304d8da6.patch

This can be quite nice when testing scripts that automatically download information from the web!

http://tinyurl.com/mrwuzy Program – Linux Plumbers Conf 2009

A while back I posted a Netconsole tutorial for how to capture Linux kernel debugging messages from a crashing machine. I've refined the instructions down to three scripts and three commands, which are after the break.

Read more »

I like seeing LWN writers pick up small patches and explain what they are why they are important. As a developer, often the impact of a change is not obvious and without further explanation significant changes go unnoticed. The recent story about Generalized TTL Security Measures in lwn.net is one such example.
But, when a story comes out, the writer should do research on the background. First, it is nice to give some credit to the author :-) and Vyatta, as well as also some history. I did this patch based on an enhancement request for the current Vyatta version. The starting point was a (unaccepted) patch to Quagga, and existing implementation for FreeBSD systems. It was one of those patches where the kernel change took less time than writing the test programs.

Also, the initial patch wasn't perfect since (nothing ever is), since it broke time wait sockets, and missed the case of ICMP messages. Both should be fixed by the time 2.6.34-rc2 comes out. Also, the necessary support has not been integrated into upstream Quagga (yet).

I appreciate the review and feedback from Eric, Andi, David, and Pekka for making this work.

Linux Plumbers Conf 2010 will take place in Cambridge, MA on November 3-5,
2010. The event will be co-located with the Linux
Kernel Summit taking place on November 1-2, 2010.

To get announcements about LPC 2010 you can
subscribe to our announce list. Additional details will be made available
as the conference date nears at this URL. See you in Cambridge!

If you have any questions or suggestions for this year’s conference, please do
send in your comments to us at contact@linuxplumbersconf.org.

As the Android kernel code is now gone from the Linux kernel, as of the 2.6.33 kernel release, I'm starting to get a lot of questions about what happened, and what to do next with regards to Android. So here's my opinion on the whole matter...

See more ...

KeynotesのDavid S. Millerのスライド見ればだいたい分かった気になれます！ http://linuxplumbersconf.org/2009/program/

My Linux Conference AU slides are now posted on my server. The USB 3.0 talk went really well, and I look forward to sharing it when the LCA videos go up in the next couple weeks.

LCA was a total blast! The speakers were wonderful, and I really felt integrated into the conference and social events by the techie women of LCA. Thank you to Sara, Jo, Jacinta, Liz, and all the other Haecksen of LCA2010!

The only downside is the weather. It's really quite rainy here, although it's 10-15 degrees warmer than Portland. I think Jamey and I are going to skip the Tongariro Crossing and go straight to the glow worm caves at Te Kuiti/Waitmo. At least we'll be in a cave while it's raining!

Here's the state of the -stable kernel trees, as of January 18, 2010.

2.6.27-stable

The 2.6.27-stable kernel tree is still living on, as a "long-term" stable release. But, I do have to warn users of this tree, the older it gets, the less viable it becomes. Not all bugfixes are being backported to this kernel version due to massive code changes in the over 2 years since this kernel has been released. I am doing my best to backport fixes that I become aware of, and I encourage anyone who does fix any types of bugs in the main kernel tree to let me know if the change should be applied to this older kernel version.

I'll probably keep maintaining it for at least 6-8 more months, but after that, I can not guarantee it's viability. Note, one other developer has volunteered to pick up the tree after I am finished with it, but I can not speak for him at this time.

2.6.31-stable

Today the last 2.6.31-stable kernel was released, all users of this kernel series are strongly encouraged to switch to the 2.6.32 kernel series, as there will not be any more updates for this branch in the future.

2.6.32-stable

I'd like to announce that the 2.6.32-stable tree is also going to be maintained as a "long-term" stable release, living for 2-3 years, like the 2.6.27 kernel is. This is because a number (i.e. more than 2) Linux distributions are basing their "enterprise" releases on this kernel version, and it will make their lives easier if I keep it alive.

Note, the viability of me keeping this tree alive for such a length of time relies on the developers working for those distros to keep me informed of patches that need to be backported and applied to it. Without their help, I will have no problem in stopping the maintenance of the tree.

Submitting patches for stable trees

Again, the easiest way to get your patch into a -stable tree is to merely add the line:

Cc: stable <stable@kernel.org>

to the Signed-off-by: area of your patch. When the patch goes into Linus's tree, it will be automatically sent to the stable address, and I will know to apply it to the trees. If I have any problem applying it at that time, I will email the author and reviewers of the patch about it.

If you forgot to add this line to the patch, or you have found a patch written by someone else that you wish to have applied to the stable trees, email the git commit id of the patch as it shows up in Linus's tree to the stable@kernel.org email address. Any stable correspondence sent to my personal accounts has the chance of being lost in the shuffle, so please try to not do that.

If a patch needs to be backported to one of the stable trees because it does not apply directly, please send the backported patch, along with the git commit id of the original patch, to the stable@kernel.org address, with a description of which kernel tree it should be applied to.

If anyone has any other questions about stable releases, please let me know.

Anyone in the Portland area interested in a fun and creative event is invited to the 1st Timbertalkers Powerpoint® Karoke contest on Tuesday 11/24 at noon.

Meeting location is: 9403-B SW Nimbus Ave., Beaverton, Oregon

If you have never done PPTK, here are the rules:

• Topic is draw from set of 30 topics. Probably 10 to 15 slides
  
• Speaker will have 2 to 3 minutes
  
• Prizes awarded
  

In spirit of open source, it will really be a OpenOffice Impress contest, and the slides will be drawn from Creative Commons licensed decks.

My LiveJournal blog name - valhenson - was the last major holdover from my old name, Val Henson. I got a new Social Security card, passport, and driver's license with my new name several months ago, but migrating my blog? That's hard! Or something. I finally got around to moving to a brand-spanking-new blog at WordPress:

Valerie Aurora's blog

Update your RSS reader with the above if you still want to read my blog - I won't be republishing my posts to my new blog on this LiveJournal blog.

If you're aware of any other current instances of "Val Henson" or "Valerie Henson," let me know! I obviously can't change my name on historical documents, like research papers or interviews, but if it's vaguely real-time-ish, I'd like to update it.

One web page I'm going to keep as Val Henson for historical reasons is my Val Henson is a Man joke. Several of the pages on my web site were created after the fact as vehicles for amusing pictures or graphics I had lying around. In this case, my friend Dana Sibera created a pretty damn cool picture of me with a full beard and I had to do something with it.



It's doubly wild now that I have such short hair.

ZFS now has data deduplication - with the right configuration options for safety and performance in a compare-by-hash based storage system. From Jeff Bonwick's ZFS deduplication blog entry:

Given the ability to detect hash collisions as described above, it is possible to use much weaker (but faster) hash functions in combination with the 'verify' option to provide faster dedup. ZFS offers this option for the fletcher4 checksum, which is quite fast:

zfs set dedup=fletcher4,verify tank

The tradeoff is that unlike SHA256, fletcher4 is not a pseudo-random hash function, and therefore cannot be trusted not to collide. It is therefore only suitable for dedup when combined with the 'verify' option, which detects and resolves hash collisions. On systems with a very high data ingest rate of largely duplicate data, this may provide better overall performance than a secure hash without collision verification.

What I like is (1) the user chooses the hash function based on their security and performance needs, (2) the system can optionally check for hash collisions, and (3) the ZFS storage pool design makes it easy to migrate data to a new hash function if necessary. ZFS is the first deduplicating storage system I know of with these features. (Do let me know if there are others out there!)

We’ve started populating the Program page with available videos, including Linus’s git tutorial and some of the security, networking and x  sessions. We’d like to thank the Linux Foundation video site which is hosting these for us.  Not all of the sessions were taped, unfortunately – only the sessions in one of the tracks/rooms all day long were taped – something we hope that next year’s organizing committee will be able to improve upon!

A few broken URLs and slide pointers have been cleaned up. If you don’t see your slides here please do send/ resend them to us at contact@linuxplumbersconf.org. Speakers who want to add their updates, summaries or notes on the sessions can do so by editing their proposal pages as well (linked to on the Program page).

If you have any questions or suggestions for next year’s conference, please do send in your comments to us at contact@linuxplumbersconf.org.

Now that you have VMX working, you might notice that kvm/qemu's PS/2 mouse emulation sucks. The VM's mouse cursor totally does not follow the host's mouse cursor at all! That's a horrid mess; how would we clean that up? The ugly hack is to disable PS/2 mouse emulation and present a USB tablet instead:

$ kvm -usb -usbdevice tablet

Now, why is PS/2 mouse emulation messed up in the first place? It turns out that PS/2 mice do not report absolute coordinates; they only tell the OS about relative changes in position. "Two steps up" "Seventeen steps to the right" etc. This is fine with a physical mouse because the physical mouse moves in an arbitrarily defined rectangle that is separate from the screen. The mouse is not placed directly on the screen, so there is no need to know the absolute position of the physical mouse on the screen. In the case of a VM, however, we know the absolute position of the (host) mouse on the screen. Tablets provide a fixed size rectangle with absolute coordinates, and that's why they (should) work better.

Obvious caveat: The VM's graphical windowing system must know about USB and tablets. Allegedly X.org is not good at that, though Windows is.

Update: Recent Ubuntu X.org seems to work....

If you’re ever in the state of Oregon, take the time to visit the  Rice Mineral Museum. I took a trip there today, and it was eye-opening, staggering, and simply wonderful. This is a world-class museum, a Smithsonian-level collection hiding out in the middle of nowhere, also known as the north end of Shute Road in Hillsboro. Their website and photo gallery simply do not do them justice.

Most of the pieces were so staggeringly beautiful that they far outdid commercial art that’s sold for megabucks. Amongst the very cool things, a slice of the collection comes from Pashan, Pune, one of the several places on this planet I call home.

I wanted to express my best wishes and hope for the safety of those
who are in danger.

Especially to the GNOME Farsi community in Iran, we are all hoping for your
continued safety during this upheaval. Please be safe.

re:Twitter -
The wonderful thing about twitter is that they can communicate their
stories and experience in such a profound way. I find myself glued
to the screen to see how things are unfolding in Iran. I’ve been
following the news on Andrew Sullivan’s blog site which gives up
to date information of what is happening in Iran.

edit: fixed broken link.

So I’ve been meaning to put a post on this but I haven’t had a chance thanks to school and what not.  Having finished my exams and projects, I thought I would put in a word or two regarding the other thing I’ve been busy with and that is the Linux Plumbers Conference.  This is my second year in a row in being part of the organizing committee of Linux Plumbers Conference!

Linux Plumbers Conference is one of those small conferences organized by developers for developers and thus have a highly technical focus.  As a bonus, it’s held the same week as Linuxcon so you can get your taste of the O’Reilly type conference as well if laundry is an issue.   The goal is to improve the Linux platform, targetting the weaknesses and improving them.  Last year we had the fast boot by Intel which started a distro war over boot time. and we had a piece of X put into the kernel, so things happen at this conference.

We are currently looking for paper submussions.  Have an idea for the desktop but you need kernel developer support?  This is your chance!  Thanks to Jim Gettys to being our runner for the UI track.  So if you have ideas that will help advance the Linux platform/ecosystem.  Please submit your ideas! The conference is in Portland, OR Sept 23-25, 2009. I hope to see a lot of you there. As many can attest, Portland is an awesome city with some of the coolest people on the planet living here.

/me rounded up a bunch of (old) panoramas and put them into the high-definition panorama viewer. Be sure to check out the (huge spike in memory cache when you load the) panorama previewer (click the "See All" button).

So here’s the news – we’ve invited Vivek Kundra, Federal CIO to keynote at LPC, and we hope he can make it! This should be really interesting. If he makes it, it will be via a remote, interactive, real time video session.  If you have any suggestions or questions for Vivek Kundra, or how the LPC team should set this up, feel free to post comments here or on our LPC blog!

Nisha and I started seriously looking at getting a house recently. It being the 21st Century means large amounts of our house hunting time will be spent on the Internet; specifically rmlsweb.com a regional real estate listing service.

A huge annoyance with the site is that it uses a pretty lame mapping service called “GeoJet” which generates horrible looking maps.

Naturally, I started Googling for Greasemonkey scripts to find out if someone had fixed this already and sure enough my friend Kees Cook had written a user script in 2006!

With a little modification I got it working great with the new RMLS site. You can find it at git://ifup.org/philips/rmls-mapper.git or install the latest version now!

The Linux Plumbers Conference focuses on a collection 2.5 hour microconferences that gather experts together to discuss and develop solutions to emerging problems that affect the Kernel, libraries and supporting system daemons that make Linux Linux. Last year there were six of these focused microconference on topics including: audio infrastructure, power management, and boot.

Many of the most successful microconference topics were suggested by community members not directly involved in planning the conference.

So, I need your help for 2009!

Head over to the Call for Topics and suggest an emerging topic in the Linux plumbing that we should discuss in 2009. Topics ideas can be submitted in the comments section of the LWN.net article.

for real.

I couldn’t have been happier with how the Linux Plumbers Conference went last week. I went back and looked at the original proposal that we had Arjan, Greg, and Randy present to the Linux Foundation, and we seem to have hit all our original goals. From conception we wanted this to be a “working” conference – and from the conversations in the hallways that I overheard, to the discussions in the microconfs that went on, I could see that people were indeed getting together, discussing issues and solving problems. Conferences require a lot of time, effort, and money to do right, and it’s gratifying to feel that something useful will come out of this.

I think that now I can go back to blogging about duck poo and vegetables.

After spending a few days manning the Plumbers booth at OSCON, I thought I’d post the answer to the question that everyone seemed to want to know – What is a Linux Plumbers Conference? We came up with the word “Plumbing” to describe the low level infrastructure of a Linux System. This includes the Kernel, desktop infrastructure like X and graphics libraries, system utilities like udev and hal, as well as essential libraries like glibc and friends. These components interface with each other at times – some better than others. We hope to provide a forum for people from these types of projects to get together and try to solve problems that are system wide or cross multiple project boundaries.

In addition to the topics to be discussed in the microconfs and the general talks (see http://linuxplumbersconf.org/program/schedule/), we will have “unconference” style talks. We have several smaller rooms available for people to get together and work out specifics, talk about something they didn’t get on the schedule, or have a group hug. These rooms can be reserved at the start of the conference.

August 18th the registration fee for Plumbers will increase to $300. If you haven’t already registered, what are you waiting for?