35,000-Core Ubuntu Farm Renders Avatar

I just attended Paul Gunn's talk at LCA2010, entitled:

• Challenges in Data Centre Growth (or, "You need how many processors to finish the movie???")

Paul is a Systems Administrator at Weta Digital, a Wellywood digital effects studio here in Wellington, New Zealand. Check out some of the feature films that Weta Digital has worked on, and I think you'll recognize a few. District9, Day the Earth Stood Still, Jumper, King Kong, Lord of the Rings, Fantastic Four, Eragon, X-Men, i-Robot. Wow!

It was a great talk, about the type of data center needed to render special effects in today's blockbuster movies. They have a 2 Petabyte disk array, 10gbps networking, and 35,000 cores (4,000+ HP blades) in their data center, and still it takes 48 hours to render some of their graphic sequences.

According to Paul, Ubuntu is at the core of all of this, running on all of the rendering nodes, and 90% of the desktops at Weta Digital. He notes that his farm (he calls it a "render wall") is in fact an Ubuntu Server farm, and not RHEL as he has seen reported in the media.

Here's a couple of articles on Weta Digital's data center and their work on Avatar:

• http://www.information-management.com/newsletters/avatar_data_processing-10016774-1.html
• http://www.itworld.com/hardware/93127/data-center-plays-supporting-role-avatar

:-Dustin