HD Video – A Primer
This weekend I spent an exorbitant amount of time trying to deal with HD video in the same way I manage my digitized DVD collection. By that I mean, attempting to figure out how to get ripped HD movies into a format that I can play over my network, with the XBox 360, the Mac in our room and a PC as the target players. Things have not gone well, and while some people may have worked out a system they can live with, I've not found a solution that can be applied efficiently enough to make it worth the effort. Nonetheless, I'll document what I've learned, in case its useful to anyone. There is no new information here -- all of it is on the web somewhere -- I'm just hoping to pull it into a coherent form...
MPEG4
For starters, high def content is invariably distributed in MPEG4-10 format. The different ways it can be bundled into a file will be discussed later on, but its important to first understand that not all MPEG4 is created equal.
MPEG4-2, or Part 2, is very common, and not really suitable for HD content. DivX and XVid are both implementations of MPEG4-2. DivX is MPEG4-2 with a couple extra features added. XVid is an independent re-implementation of DivX that, while usually compatible, is definitely a different beast. A player with DivX stamped on it can probably play any MPEG4-2 content, and probably play any XVid content. In cannot, however play MPEG4-10 content.
MPEG4-10, or Part 10, is relatively new and most commonly known as H.264. Its based on the same technology as its older cousin, but includes improvements to color handling, and allows higher quality content at lower bit rates.
Bit Rate: Loosely defined, the Bit Rate is the number of bits of data that go into showing 1 second of video. The higher the bit rate, the more information making up the image you see on the screen. Higher bit rates = better quality video.
H.264 can do better picture, with less artifacting, and less "chunkiness" (if you don't know what I mean by chunkiness, watch a dark scene in any DivX movie) at a lower bit rate. A 2GB MPEG4-10 video will look better than a 4GB MPEG4-2 video.
Containers
All video, regardless of the codec (be it MPEG2, MPEG4-2, H.264, etc...) is typically distributed inside a container, bundling up the video track with an audio track, and possibly other features such as alternate audio tracks (containing a different language) or subtitles. Wikipedia has a good entry comparing the different container formats, but here are the common ones you'll see for HD Video:
Quicktime (.MOV)
Quicktime is an incredibly flexible container that can hold virtually any collection of files, by using plug-in codec support. A .MOV container can hold an H.264 video track plus a DTS 5.1 surround sound audio track without any problem.
Limitations: There's not many devices with full Quicktime support. Basically you're stuck with something from Apple. The XBox does have basic support for .MOV containers, but doesn't have the ability to plug-in codecs to extend its limited offering.
Windows Media File (.WMV)
Windows Media is actually very fully featured, with support for containing high bit-rate HD video and audio tracks with 5.1 surround sound.
Limitations: Its a fantastic format, despite being from Microsoft, but there aren't many devices that support it, and I frankly hate Windows Media Player. You'll rarely find any content distributed in a WMV container.
MP4 (.MP4)
By far the most popular, and infuriatingly limited container for H.264 (MPEG4-10). Do not confuse "MPEG4" for "MP4" -- one is the video track, one is the container it travels in. MP4 supports H.264 video at all bit rates (but not all bit rates can be played on all devices, read about levels here) and can hold an AAC-LC audio track (probably more than one.) But herein lies the biggest problem with MP4: AAC-LC means "Low Complexity" or in other words "no digital surround sound." You can encode Dolby Pro Logic II into the stereo track and thus get surround sound that way, but it won't be DTS and it'll basically suck.
Matroska (.MKV)
The Matroska container is the most common one you'll see -- both from ripped optical media, such as Bluray or HD-DVD discs, and online. Its very flexible, and can hold super high bit-rate H.264 video tracks as well as multiple audio tracks, including DTS (Digital Theater Surround).
Limitations: Apparently its a poorly defined standard and there are lots of different interpretations on how to implement it. No OS or media player software supports Matroska out-of-the-box. Few set-top devices can support Matroska. The XBox 360 does not.
Useage
None of these containers formats are really mature enough to be considered ready for consumer use. Different devices can handle different containers -- but each with restrictions on what's inside the container. Here's a list of what the XBox 360 can handle.
The best standardized container, MP4, is missing a pretty critical piece of the home theater puzzle: digital surround sound. This is not a new feature here, folks. It needs to be supported for MP4 to be useful. The other containers, which do support surround sound, are either too proprietary or too disorganized to be useful yet.
If you were to obtain a MKV file with HD video inside (H.264) and a surround sound audio track, and you wanted to play it on your XBox, you'd basically have two choices on how to proceed:
Extraction and Re-bundling
Using an array of poorly-made tools and hacks available online, which I won't cover here out of disdain for them and the hours of time I wasted on them, you can extract both the H264 video stream and the AC3 (digital surround sound) audio track into separate files. Then you can modify them to be playable on your target device, and re-contain them in a supported container.
For example, you could drop the bit rate on the H264 stream down to at least level 4.1, outputting it to a new, slightly more compressed, but certainly still "Hi-Def" file. Then you could pare the audio track down to Stereo, with Dolby PL2 intact. Finally you could combine the two files together into an .MP4 container and play them on your XBox 360, PS3, etc...
A friend claims to have this process down to a few clicks done in less than 10 minutes (he says 6 minutes or less, but I'll chalk that up to bragging.) Using a Mac with Windows running in a VM, then later, after much frustration and waiting, using a well-equipped physical Windows PC after sneaker-netting the files, I was unable to complete all the steps of the process after 5 hours. The "software" is buggy, has poorly documented dependencies and 0 troubleshooting information if anything goes wrong.
My friend calls this operator error and says I'm technologically inept. I call it really bad software being used to do something it wasn't really intended for. I finally gave up.
Down-sampling
The less attractive, but much easier option, is to simply down-sample the whole contained file into a MPEG4-2/DivX file. If you have a Matroska file, you might still want to do some conversion first -- Quicktime Pro can 'Save As' a MKV file into a MOV in a couple seconds.
From there, many mature applications, like Handbrake or FFMPEGx on the Mac, will be able to convert the whole thing into something playable for you. It may take some time, and you'll want to compress at a high bit-rate to keep as much of the video quality as possible (and don't fool yourself -- you will loose quality), but it'll be a usable, decent-looking file that can play on a wide-range of devices, AND it can contain DTS 5.1 audio.
Conclusions
HD Video is the biggest screw-up in consumer technology history. We've been waiting for this for years... Waiting while they sort out 720p, 1080i, or 1080p as the resolution we're supposed to buy. Waiting while the cable companies struggle to get a trickle of 720p content to our sets (then screw their customers around trying to save money on it.) Waiting while big companies publically fight over which optical format we're supposed to go out and re-buy all our movies on...
In fact, "HD" video (eg: 1080p) describes only the resolution. You can have a 268kbps bit-rate video in 1920x1080 and call it "HD" if you want. It'll look like crap, but it'll be big!
Learning about the different container formats and how they're still a fractured mess of incompatible technologies only makes me angrier. Here I stand, ready to jump on the Hi-Def band wagon, with my brand-new TV, my brand-new XBox 360, cash in my hand... and the only reasonable and affordable way to get HD content is the now-defunct $50 HD-DVD add-on player I bought on clearance.
Bluray is ridiculously expensive. No one offers 1080p downloads or subscription content. Even the pirate world can't figure out how to distribute content playable on any common device...
Its a disaster. I spent the better part of a day trying to make it useful, and I've decided that, for now, I'll be storing my content in high bit-rate DivX with my surround sound intact, thank you very much. I'll re-visit this again sometime in 2009, and hope they've got their crap together...
Anyone have any corrections to this, or suggestions to make this space more understandable and usable?
- Update: Gizmodo has a great article with more details on this topic.
Leopard doesn’t quite stack up
I'm a Mac fan boy, and I'm very tolerant of change. When the Mac using world screamed when they replaced OS 9 with OS X, killing a lot of the nice functionality in OS 9 in favor of a brand new OS, I drank the kool-aid and made the change as soon as I could.
And I've been a big fan of each major revision of X -- and have run all of them right out of the gate, warts and all. But I'm not sold on Leopard.
The loudest touted features are either useless, or already existed in 3rd party apps for 10.4.
- Spaces is just VirtuDesktops -- and a concept I never had any use for. I'd much rather have multiple monitors then multiple "virtual" screens.
- TimeMachine is just a back-up app -- granted its well integrated and looks pretty. But the method I have in place now is much less intrusive (not to mention more fire-proof) and doesn't require me to be tethered to a local external hard drive.
But nothing bugs me more than Stacks...
Stacks kills a feature I've used and loved in OS X since they killed the Apple menu's hierarchical app-launching capabilities from OS 9.
It used to be, in OS X 10.2-10.4, that you could drop any folder onto the right side of the Dock and turn it into a menu by right clicking on it. In this way I could add my very organized Applications folder to my Dock and use it to find applications in a very simple manor (I'll admit, its kind of like a Start Menu, but come on -- its a good way to organize your apps).
In 10.5 that great feature was replaced with either a silly, and surprisingly static, curved list, or a box full of icons. Now that box can contain subfolders -- but if you click on them, it opens a window. That's a window you have to close after you find your application. Unacceptably stupid.
Know what else is stupid? Apparently there's a massive data loss bug in the new Finder. For years -- literally, we've been complaining about how awful the Finder is (even compared to its OS 9 grandparent) and this is the fix they offer us?
Damage to the Dock and a bad upgrade for the Finder aside (I do like the new sidebar, btw) Stacks are not what they were supposed to be. A "Stack" was designed as a replacement for (or addition to) the folder concept. It was an organizational item that existed in the Finder, had a single icon that could be dragged around like a file, but expanded when clicked to show a collection of items. There are plenty of UI prototype videos on the web showing what Stacks meant to us before Apple polluted the term.
I'm ditching Leopard for the time being. The only feature I really wanted was the improved Front Row... maybe there's a way I can run it in 10.4. If not, I'll just hold out hope that Apple gets their crap together with upcoming updates.
Update: Yay! It looks like they fixed stacks... a little bit!
Experiments in Virtualization
Virtualization is so hot right now... (although it can't be that hot, because my spell-checker doesn't even recognize it as a word.)
The concept is that as hardware gets faster and faster, we've actually reached the point of diminishing returns on all but the most grueling of tasks (HD Video rendering is the only other thing I've seen that really taxes a modern PC.) Now we've got hardware that can support not just one environment without breaking a sweat, but 2 or 3.
At work we have a server running VMWare's ESX Server that's currently running 16 "virtual" computers at once. None of this is that new -- what is new is the way processors are being built with this kind of thing in mind. Intel's Core2Duo is made for virtualization, and nothing proved this to me faster than Parallels Workstation.
I have a fairly recent Dell Laptop, based on a the Centrino chipset -- essentially a P4, at 1.3GHz with 2GB of RAM. It runs Windows and VisualStudio adequately. For awhile, I was given the use of a MacBook Pro, running a CoreDuo. The thing screamed running its native OS -- OS X 10.4, but when I booted up Windows XP inside a Parallels virtual machine, I was astonished to find that Windows actually ran faster virtualized on my MacBook then it did on my Dell. I used it for my day-to-day development, rendering video in one machine, while compiling code in another. And I was sold.
A colleague of mine has taken this concept to its logical extreme, and I won't pretend to have his knowledge on the subject. Rather, if you're interested, you should read his posts.
- Build Machine Virtualization discusses how he eliminated the need for frequent hardware replacement and streamlined our company's build process using VMWare.
- Development Machine Virtualization discusses how he's created specific environments for different development tasks, while still adhering to our I.T. departments onerous rules.
The latter article is the one I'd like to build on with my own thoughts here. My set-up, briefly...
I'm using a desktop computer -- not a laptop, as my afore-mentioned Dell left a bad taste in my mouth. The desktop is a Dell as well, since I.T. gives us no other choices, but despite being as ugly as sin cosmetically, its a very nice machine:
Processor: Intel Core2Duo @ 2.4Ghz with a 1Ghz FSB
RAM: 4GB @ 800Mhz (although Windows XP only recognizes 3GB)
Hard Drive: 250GB SATA II 7200RPM
Operating System: Windows XP SP2 - I.T. Core Load
The real beauty of this set-up is that the OS referred to above, which I consider to be less than satisfactory (at the least, they should have used Windows XP 64-Bit), really doesn't matter -- it could even be Ubuntu. Its only the host OS. As Shawn recommends, I'm running a very light-weight collection of apps, that I'm happy to let I.T. manage for me. I've got Windows and Office, plus a messaging client and FireFox, and various Remote Desktop/VNC clients. That's it. Add VMWare, and from here, I can have the world...
Currently I'm running two "virtual" computers running Windows XP, one running Windows Server 2003, and one running Mac OS X...
My Development VM gets 1GB of RAM, and has a 50GB pre-allocated virtual hard drive. I took a snap-shot when I started, but I realised later I could have achieved the same thing, just by copying the VM directory off to an external archive hard drive. Either way, if anything gets messed up in Windows (which, lets admit, happens fairly frequently) I can just revert the whole system to a working state with a couple mouse clicks.
My Test VM gets 512MB of RAM, more if it needs it, and a 15GB growable hard drive. This is a "clean" install of Windows XP, updated with nothing installed on it. I run the latest version of the software I'm developing on this.
Whenever I want to test how a specific build of my software behaves when installed on a customer's computer, I can snapshot back to its "clean" state, and run the Setup program.
My Server VM is a machine I took from our Green Bay office while there. They had a virtual server configured with a replica of a customer's configuration, and all I had to do was copy it to a portable hard drive and carry it with me. It has a pre-allocated hard drive, and gets 512MB of RAM -- more if needed.
My OS X VM lets me run Mail.app (my preferred mail client), PhotoShop and other nice programs that only Mac has to offer. It should be noted that performance on this machine is not stellar -- but that's only due to a lack of driver support, since Virtualizing Mac OS X is not supported.
The coolest of these virtual machines is the Development VM -- it was made from an actual physical computer. A little app called VMWare Converter can run on a machine, and create a VM out of it. So doing, I was able to switch from my old computer to my new one in a matter of hours, instead of the day or two it normally takes to get Windows, VisualStudio + Orcas, SQL Client Tools, .NET 1.0-3.0, and TortoiseSVN up and running on a fresh box.
The performance on any 3 of these machines running at the same time is equal-to, or better-than a physical computer only a year or two old. With two monitors, and an RDP connection to another headless server I have running behind me, I can juggle 5 computers at once, (6 if you count the host) with more than adequate performance.
I'll point you again to Shawn's articles if the benefits of this set-up aren't self-evident. I was initially a little unsure that the VMs would be responsive enough for my daily use, but VMWare and the Core2Duo have proven to be a stellar team...