Archive for October, 2010

Scanning today.

Window, 2010

New work

New work coming soon. Stay tuned!

Building a RAID array for ProRes Video Editing

Well. It’s been quite a technical hayride the last four days. It all started with OS X Snow Leopard. I had built a raid system on a Mac Pro. I have a system HD and a basic 3 hard-drive raid running 1.5TB Barracudas (Not enterprise level. We don’t got the money.) I ran it RAID zero and let everyone know that it was for temp projects only. Then I time-machined it and scheduled the backup to run at 4am each morning just so most recent projects are saved (maybe a week’s worth or so.)

In the back we have 2 LACP 1Gb ethernet ports for 200MB of duplexed bandwidth. I’m going to add another 2 ports soon and another hard-drive to the RAID array. Needless to say, the thing is pretty peppy. It takes about 3 seconds to open a 300MB file off this server onto an iMac with Photoshop. Opening the same file right off the “Artbox” server takes less than a second.

But then we ran smack into a problem. A few students incorrectly converted their Canon 7D footage to ProRes 444. This is a way big file format and for some weird reason it totally freaked out AFP file sharing (the protocol we were using to host the server.) Whenever this 444 project was opened, all the connections on all the other client computers dropped. The wheel of death came next and students started loosing work and getting frustrated.

So I stayed up late last night and rebuilt the server using OS X Leopard Server. This lets me add processor threads to to Apple File Server. I brought this to 602 threads and brought my TCP ACK delay to 2 on all the machines. This helped matters but I was still sending out way too much bandwidth. A simple ProRes 422 LT stream was taking up 60 MB/s of bandwidth. Totally stupid.

Eventually I got it into my thick noggin that the problem was not AFP settings but it was the protocol itself!! I’ve been a diehard AFP believer for over a decade, but it was finally letting me down. OS X just kept dying whenever the Artbox share was canceled.

So in the end I turned to NFS as our prefered protocol. I also turned on Flow Control and Jumbo frames on all the client computers. The result? Right now, as I type this, we have 10 people editing live with 1080i resolution files directly off the server using only 3 raided hard-drives with zero problems. For the cost of three HDs (less than 500 bucks) we have a system that runs fast enough for an entire production studio.  Thank you Mac Pro. Thank you NFS. Fuck you AFP.

Now you know what I do every day.

ACF loading animated gif