Planet Linux Plumbers Conf

Parallel Programming: Administrators as Architects

2010-03-01T01:43:44Z

Although IT professionals should take care to avoid engineering envy, it is often useful to learn from the experiences of other engineering disciplines. In this posting, I will compare and contrast construction of a building to implementation of a large software project.

Leaving aside financial engineering, building construction starts with an architect, who lays out the general shape and look of the building. A structural engineer creates a detailed design, with an eye towards ensuring that the building will remain standing despite the best efforts of wind, gravity, and plate tectonics. A construction engineer works out the details of the construction process — for example, it is good if the building can support itself while being built as opposed to doing so only when completed. Other engineering specialties may be required as well, for example, HVAC (heating, ventilating, and air conditioning).

Once the building is built, different skills are needed, including operating engineers, maintenance personnel, and janitors.

A very similar sequence of events can play out for a large software application. Software architects (for better or worse) lay out the general shape of the project, developers design and code it, and others ensure that it is built, tested, and safely ensconced in some source-code management system.

However, once the application is completed, it is likely that its care and feeding will be taken over by application, database, and system administrators. The architects and developers will switch to other projects (possibly version N+1 of this same application), and perhaps even retire or otherwise move on. Of course, if the application runs at multiple sites, there might well be a separate set of administrators for each site. But for simplicity, let's assume that this application runs at only one site.

Now suppose that it is necessary to parallelize this application.

This is tantamount to major structural change to the building, such as adding several new floors. A structural change of this nature is clearly not a job that you would normally entrust to operating engineers, maintenance personnel, or janitors.

But what else can you do if the original architects and developers are gone?

Parallel Programming: Selective Struggling

2010-02-12T23:33:32Z

An Eminent Reader privately indicated some distaste for the non-technical nature of recent parallel programming posts. Given that many of the obstacles to successful development of parallel software are non-technical, there will be future non-technical posts, but there is no reason not to take a technical break from these issues. And so, just for you, Eminent Reader, I present this parallel programming puzzle.

This puzzle stems from some researchers’ very selective struggles with parallel algorithms. Of course, it should be no surprise that many people, researchers and developers included, will struggle quite happily with their “baby”, but will even more happily bad-mouth competing approaches, even when (or perhaps especially when) those approaches requiring much less struggling. And yes, some might accuse me of favoring RCU in just this manner, but this is my answer to the likes of them.

Such selective struggling seems to have given rise to an interesting urban legend within the concurrency research community, namely that allowing concurrent access to both ends of a double-ended queue is difficult when using locking.

Can you come up with a lock-based solution that permits the two ends of a double-ended queue to be manipulated concurrently?

LPC 2010 November 3-5 2010 in Cambridge, MA

2010-02-06T21:11:56Z

Linux Plumbers Conf 2010 will take place in Cambridge, MA on November 3-5,
2010. The event will be co-located with the Linux
Kernel Summit taking place on November 1-2, 2010.

To get announcements about LPC 2010 you can
subscribe to our announce list. Additional details will be made available
as the conference date nears at this URL. See you in Cambridge!

If you have any questions or suggestions for this year’s conference, please do
send in your comments to us at contact@linuxplumbersconf.org.

Android and the Linux kernel community

2010-02-03T00:18:00Z

As the Android kernel code is now gone from the Linux kernel, as of the 2.6.33 kernel release, I'm starting to get a lot of questions about what happened, and what to do next with regards to Android. So here's my opinion on the whole matter...

KeynotesのDavid S. Millerのスライド見ればだいたい分かった気になれます！ http://linuxplumbersconf.org/2009/program/

2010-01-29T11:45:58Z

KeynotesのDavid S. Millerのスライド見ればだいたい分かった気になれます！ http://linuxplumbersconf.org/2009/program/

Yup, Exadata uses OFED

2010-01-26T02:41:52Z

I checked out this video today, with OF's Executive Director, Bill Boas, speaking to the Lustre Users Group 2009.

He mentions RDS favorably a number of times. One thing that he would probably also like me to mention is that yes, Oracle's Exadata product (you know, the one with all the ads in The Economist, The Week, and The WSJ) is based on InfiniBand and OpenFabrics' software, OFED. Perhaps OFED has succeeded in its mission so well, and become so ubiquitous, that it may have become one of those parts of your system that "just works", and gets no love.

I think it would be cool for Larry Ellison to name-check OFED, IB, and RDS as ingredients in the Exadata special sauce... but I don't know if he or the ads even mention it runs Linux? Or GNU software? Or OEL's lineage from code by a certain North American Enterprise Linux vendor? I'm pretty sure he does mention the database software, though. :)

So, what can you do? People working on HPC and high-speed networking already know who OpenFabrics is, or at least about OFED. Even as converged Ethernet and IBoE bring RDMA to a wider audience, I have my doubts whether OFED will become a well-known name. It may simply have to count being relied-upon by a growing, happy userbase as its success.

LCA 2010

2010-01-22T23:26:19Z

My Linux Conference AU slides are now posted on my server. The USB 3.0 talk went really well, and I look forward to sharing it when the LCA videos go up in the next couple weeks.

LCA was a total blast! The speakers were wonderful, and I really felt integrated into the conference and social events by the techie women of LCA. Thank you to Sara, Jo, Jacinta, Liz, and all the other Haecksen of LCA2010!

The only downside is the weather. It's really quite rainy here, although it's 10-15 degrees warmer than Portland. I think Jamey and I are going to skip the Tongariro Crossing and go straight to the glow worm caves at Te Kuiti/Waitmo. At least we'll be in a cave while it's raining!

Stable kernel tree status, January 18, 2010

2010-01-18T19:11:00Z

Here's the state of the -stable kernel trees, as of January 18, 2010.

2.6.27-stable

The 2.6.27-stable kernel tree is still living on, as a "long-term" stable release. But, I do have to warn users of this tree, the older it gets, the less viable it becomes. Not all bugfixes are being backported to this kernel version due to massive code changes in the over 2 years since this kernel has been released. I am doing my best to backport fixes that I become aware of, and I encourage anyone who does fix any types of bugs in the main kernel tree to let me know if the change should be applied to this older kernel version.

I'll probably keep maintaining it for at least 6-8 more months, but after that, I can not guarantee it's viability. Note, one other developer has volunteered to pick up the tree after I am finished with it, but I can not speak for him at this time.

2.6.31-stable

Today the last 2.6.31-stable kernel was released, all users of this kernel series are strongly encouraged to switch to the 2.6.32 kernel series, as there will not be any more updates for this branch in the future.

2.6.32-stable

I'd like to announce that the 2.6.32-stable tree is also going to be maintained as a "long-term" stable release, living for 2-3 years, like the 2.6.27 kernel is. This is because a number (i.e. more than 2) Linux distributions are basing their "enterprise" releases on this kernel version, and it will make their lives easier if I keep it alive.

Note, the viability of me keeping this tree alive for such a length of time relies on the developers working for those distros to keep me informed of patches that need to be backported and applied to it. Without their help, I will have no problem in stopping the maintenance of the tree.

Submitting patches for stable trees

Again, the easiest way to get your patch into a -stable tree is to merely add the line:

Cc: stable

to the Signed-off-by: area of your patch. When the patch goes into Linus's tree, it will be automatically sent to the stable address, and I will know to apply it to the trees. If I have any problem applying it at that time, I will email the author and reviewers of the patch about it.

If you forgot to add this line to the patch, or you have found a patch written by someone else that you wish to have applied to the stable trees, email the git commit id of the patch as it shows up in Linus's tree to the stable@kernel.org email address. Any stable correspondence sent to my personal accounts has the chance of being lost in the shuffle, so please try to not do that.

If a patch needs to be backported to one of the stable trees because it does not apply directly, please send the backported patch, along with the git commit id of the original patch, to the stable@kernel.org address, with a description of which kernel tree it should be applied to.

If anyone has any other questions about stable releases, please let me know.

A New Years Update

2010-01-13T23:13:38Z

tap tap Is this thing on?

Choose your excuse:

I haven't posted in about six months, because

a. I bought a house,

b. I acquired two cute kittens (Derek and Elisa),

c. The acquired house needed a ton of work, including taking down a 30 foot "hedge" in the backyard,

d. I've been experimenting with all kinds of cooking,

e. I'm lazy.

What's next for me?

I'm presenting about Linux USB 3.0 support at Linux Conf AU 2010. If you're going to LCA next week, you can check out my talk at Friday at 10:30am in the Ilott Theatre. I'll have a USB 3.0 demo for the brave souls that make it to morning talks. ^_^ If you miss it due to being hung over or not being able to attend LCA, I believe they'll be video taping it.

LCA is being held in Wellington, New Zealand, this year. Jamey and I are going bike camping for two weeks afterwards. As Martin Short put it, "I planned a spontaneous adventure." Let's hope Captain Ron doesn't screw it up. ;)

Any suggestions of cool things to see on the North Island? We're probably going to National Park, Te Kuite, Auckland, and the Coromandel Peninsula. Anything else in that general area?

Linux checkpoint/restart: http://linuxplumbersconf.org/2009/slides/Sukadev-Bhattiprolu-Plumbers.pdf http://bit.ly/8jNEPI

2009-11-20T16:50:38Z

Linux checkpoint/restart: http://linuxplumbersconf.org/2009/slides/Sukadev-Bhattiprolu-Plumbers.pdf http://bit.ly/8jNEPI

Powerpoint® Karoke contest

2009-11-12T00:58:39Z

Anyone in the Portland area interested in a fun and creative event is invited to the 1st Timbertalkers Powerpoint® Karoke contest on Tuesday 11/24 at noon.

Meeting location is: 9403-B SW Nimbus Ave., Beaverton, Oregon

If you have never done PPTK, here are the rules:

Topic is draw from set of 30 topics. Probably 10 to 15 slides
Speaker will have 2 to 3 minutes
Prizes awarded

In spirit of open source, it will really be a OpenOffice Impress contest, and the slides will be drawn from Creative Commons licensed decks.

Migrated to WordPress

2009-11-08T23:36:10Z

My LiveJournal blog name - valhenson - was the last major holdover from my old name, Val Henson. I got a new Social Security card, passport, and driver's license with my new name several months ago, but migrating my blog? That's hard! Or something. I finally got around to moving to a brand-spanking-new blog at WordPress:

Valerie Aurora's blog

Update your RSS reader with the above if you still want to read my blog - I won't be republishing my posts to my new blog on this LiveJournal blog.

If you're aware of any other current instances of "Val Henson" or "Valerie Henson," let me know! I obviously can't change my name on historical documents, like research papers or interviews, but if it's vaguely real-time-ish, I'd like to update it.

One web page I'm going to keep as Val Henson for historical reasons is my Val Henson is a Man joke. Several of the pages on my web site were created after the fact as vehicles for amusing pictures or graphics I had lying around. In this case, my friend Dana Sibera created a pretty damn cool picture of me with a full beard and I had to do something with it.

It's doubly wild now that I have such short hair.

ZFS gets deduplication - the right way

2009-11-03T01:23:47Z

ZFS now has data deduplication - with the right configuration options for safety and performance in a compare-by-hash based storage system. From Jeff Bonwick's ZFS deduplication blog entry:

Given the ability to detect hash collisions as described above, it is possible to use much weaker (but faster) hash functions in combination with the 'verify' option to provide faster dedup. ZFS offers this option for the fletcher4 checksum, which is quite fast:
zfs set dedup=fletcher4,verify tank
The tradeoff is that unlike SHA256, fletcher4 is not a pseudo-random hash function, and therefore cannot be trusted not to collide. It is therefore only suitable for dedup when combined with the 'verify' option, which detects and resolves hash collisions. On systems with a very high data ingest rate of largely duplicate data, this may provide better overall performance than a secure hash without collision verification.

What I like is (1) the user chooses the hash function based on their security and performance needs, (2) the system can optionally check for hash collisions, and (3) the ZFS storage pool design makes it easy to migrate data to a new hash function if necessary. ZFS is the first deduplicating storage system I know of with these features. (Do let me know if there are others out there!)

Ubuntu 9.10 hates kernel developers?

2009-10-28T04:02:01Z

Ubuntu has never been the easiest distribution to do kernel development, but it looks like with 9.10 it has made things too painful. I need to build and install kernels all the time, and usually just update grub menu manually. But now with grub 2 in Ubuntu 9.10 they have wrapped the grub menu in grub-mkconfig. Why?

It would be great if the system was setup so just doing 'make install' in the kernel source put in the kernel and updated the grub.cfg, but no that would make too much sense.

P.s: they managed to break the sky2 driver somehow, the connection won't come up and negotiates the wrong speed. It turned out not to be a kernel problem; wiring issue (speed), combined with some Network Manager changes

LPC videos are going up

2009-10-09T17:58:16Z

We’ve started populating the Program page with available videos, including Linus’s git tutorial and some of the security, networking and x sessions. We’d like to thank the Linux Foundation video site which is hosting these for us. Not all of the sessions were taped, unfortunately – only the sessions in one of the tracks/rooms all day long were taped – something we hope that next year’s organizing committee will be able to improve upon!

A few broken URLs and slide pointers have been cleaned up. If you don’t see your slides here please do send/ resend them to us at contact@linuxplumbersconf.org. Speakers who want to add their updates, summaries or notes on the sessions can do so by editing their proposal pages as well (linked to on the Program page).

If you have any questions or suggestions for next year’s conference, please do send in your comments to us at contact@linuxplumbersconf.org.

Fix Guest Mouse Sync Problems in kvm/qemu

2009-09-20T17:56:37Z

Now that you have VMX working, you might notice that kvm/qemu's PS/2 mouse emulation sucks. The VM's mouse cursor totally does not follow the host's mouse cursor at all! That's a horrid mess; how would we clean that up? The ugly hack is to disable PS/2 mouse emulation and present a USB tablet instead:

$ kvm -usb -usbdevice tablet

Now, why is PS/2 mouse emulation messed up in the first place? It turns out that PS/2 mice do not report absolute coordinates; they only tell the OS about relative changes in position. "Two steps up" "Seventeen steps to the right" etc. This is fine with a physical mouse because the physical mouse moves in an arbitrarily defined rectangle that is separate from the screen. The mouse is not placed directly on the screen, so there is no need to know the absolute position of the physical mouse on the screen. In the case of a VM, however, we know the absolute position of the (host) mouse on the screen. Tablets provide a fixed size rectangle with absolute coordinates, and that's why they (should) work better.

Obvious caveat: The VM's graphical windowing system must know about USB and tablets. Allegedly X.org is not good at that, though Windows is.

Update: Recent Ubuntu X.org seems to work....

GSOC Mentoree interviewed

2009-08-28T19:05:57Z

My GSOC 2009 mentoree, Kanika Vats, shares some thoughts on her experiences, and tech careers for women.

I think her take on open source technology and its community is incredibly positive. I have no doubt at some point the bloom will come off the rose a bit, but this is certainly a good first impression for a newcomer to have, and hopefully will make it easier to put future negativity in perspective.

I was the sole non-female in the Systers development effort to customize Mailman to their community's requirements. (I was recruited by my SO, Jen, who was also a mentor as well as overall organizer for Systers' GSOC effort.) As a first-time mentor, this was an extremely rewarding experience.

First, it made clear that being a mentor/teacher/manager-type was work, even though someone else was doing the heavy lifting. It really helps to know what you're talking about, which I didn't always, such as giving advice on how to use the Storm ORM, when my experience was with SQLObject. (An ORM's an ORM, right?)

Second, working with a new contributor was a chance to give exposure to the things that school doesn't seem to teach but that are very important in the real world, and especially around open-source development: Coding standards; Understanding and enhancing other people's code, and writing code so that other people can understand it; Using source control and branches to help one's productivity; bug tracking; and online collaboration, especially across time zones.

Finally, I will have the opportunity to attend the annual Grace Hopper Women in Computing conference, where I am led to believe people of my gender will be in the extreme minority. Will it be weird? Too early to say -- although I have heard the conference schwag may include soaps and candles instead of 2XL-sized t-shirts. Should be interesting!

A Diamond in the Rough

2009-08-23T23:31:52Z

If you’re ever in the state of Oregon, take the time to visit the Rice Mineral Museum. I took a trip there today, and it was eye-opening, staggering, and simply wonderful. This is a world-class museum, a Smithsonian-level collection hiding out in the middle of nowhere, also known as the north end of Shute Road in Hillsboro. Their website and photo gallery simply do not do them justice.

Most of the pieces were so staggeringly beautiful that they far outdid commercial art that’s sold for megabucks. Amongst the very cool things, a slice of the collection comes from Pashan, Pune, one of the several places on this planet I call home.

Iran and the elections aftermath

2009-06-15T03:49:25Z

I wanted to express my best wishes and hope for the safety of those
who are in danger.

Especially to the GNOME Farsi community in Iran, we are all hoping for your
continued safety during this upheaval. Please be safe.

re:Twitter -
The wonderful thing about twitter is that they can communicate their
stories and experience in such a profound way. I find myself glued
to the screen to see how things are unfolding in Iran. I’ve been
following the news on Andrew Sullivan’s blog site which gives up
to date information of what is happening in Iran.

edit: fixed broken link.

Linux Plumbers Conference…

2009-06-12T00:35:58Z

So I’ve been meaning to put a post on this but I haven’t had a chance thanks to school and what not. Having finished my exams and projects, I thought I would put in a word or two regarding the other thing I’ve been busy with and that is the Linux Plumbers Conference. This is my second year in a row in being part of the organizing committee of Linux Plumbers Conference!

Linux Plumbers Conference is one of those small conferences organized by developers for developers and thus have a highly technical focus. As a bonus, it’s held the same week as Linuxcon so you can get your taste of the O’Reilly type conference as well if laundry is an issue. The goal is to improve the Linux platform, targetting the weaknesses and improving them. Last year we had the fast boot by Intel which started a distro war over boot time. and we had a piece of X put into the kernel, so things happen at this conference.

We are currently looking for paper submussions. Have an idea for the desktop but you need kernel developer support? This is your chance! Thanks to Jim Gettys to being our runner for the UI track. So if you have ideas that will help advance the Linux platform/ecosystem. Please submit your ideas! The conference is in Portland, OR Sept 23-25, 2009. I hope to see a lot of you there. As many can attest, Portland is an awesome city with some of the coolest people on the planet living here.

Picspam!

2009-06-03T02:27:57Z

/me rounded up a bunch of (old) panoramas and put them into the high-definition panorama viewer. Be sure to check out the (huge spike in memory cache when you load the) panorama previewer (click the "See All" button).

LPC keynotes announced!

2009-05-15T05:40:48Z

So here’s the news – we’ve invited Vivek Kundra, Federal CIO to keynote at LPC, and we hope he can make it! This should be really interesting. If he makes it, it will be via a remote, interactive, real time video session. If you have any suggestions or questions for Vivek Kundra, or how the LPC team should set this up, feel free to post comments here or on our LPC blog!

Greasemonkey makes real estate fun!

2009-05-13T07:00:00Z

Nisha and I started seriously looking at getting a house recently. It being the 21st Century means large amounts of our house hunting time will be spent on the Internet; specifically rmlsweb.com a regional real estate listing service.

A huge annoyance with the site is that it uses a pretty lame mapping service called “GeoJet” which generates horrible looking maps.

Naturally, I started Googling for Greasemonkey scripts to find out if someone had fixed this already and sure enough my friend Kees Cook had written a user script in 2006!

With a little modification I got it working great with the new RMLS site. You can find it at git://ifup.org/philips/rmls-mapper.git or install the latest version now!

Linux Plumbers Conference Call For Topics

2009-02-17T08:00:00Z

The Linux Plumbers Conference focuses on a collection 2.5 hour microconferences that gather experts together to discuss and develop solutions to emerging problems that affect the Kernel, libraries and supporting system daemons that make Linux Linux. Last year there were six of these focused microconference on topics including: audio infrastructure, power management, and boot.

Many of the most successful microconference topics were suggested by community members not directly involved in planning the conference.

So, I need your help for 2009!

Head over to the Call for Topics and suggest an emerging topic in the Linux plumbing that we should discuss in 2009. Topics ideas can be submitted in the comments section of the LWN.net article.

Mission Accomplished

2008-09-22T21:50:09Z

for real.

I couldn’t have been happier with how the Linux Plumbers Conference went last week. I went back and looked at the original proposal that we had Arjan, Greg, and Randy present to the Linux Foundation, and we seem to have hit all our original goals. From conception we wanted this to be a “working” conference – and from the conversations in the hallways that I overheard, to the discussions in the microconfs that went on, I could see that people were indeed getting together, discussing issues and solving problems. Conferences require a lot of time, effort, and money to do right, and it’s gratifying to feel that something useful will come out of this.

I think that now I can go back to blogging about duck poo and vegetables.

What is a Plumbers Conference?

2008-07-28T17:12:03Z

After spending a few days manning the Plumbers booth at OSCON, I thought I’d post the answer to the question that everyone seemed to want to know – What is a Linux Plumbers Conference? We came up with the word “Plumbing” to describe the low level infrastructure of a Linux System. This includes the Kernel, desktop infrastructure like X and graphics libraries, system utilities like udev and hal, as well as essential libraries like glibc and friends. These components interface with each other at times – some better than others. We hope to provide a forum for people from these types of projects to get together and try to solve problems that are system wide or cross multiple project boundaries.

In addition to the topics to be discussed in the microconfs and the general talks (see http://linuxplumbersconf.org/program/schedule/), we will have “unconference” style talks. We have several smaller rooms available for people to get together and work out specifics, talk about something they didn’t get on the schedule, or have a group hug. These rooms can be reserved at the start of the conference.

August 18th the registration fee for Plumbers will increase to $300. If you haven’t already registered, what are you waiting for?