Will Windows, OS X and Android go 3D?

Pros and cons of 3D operating systems

Will Windows  OS X and Android go 3D

3D is everywhere. It's in the movies. It's in TVs. It's in gaming PCs. It's even rumoured to be coming to the iPad 3, although we're taking that particular rumour with a huge pinch of salt.

One place 3D doesn't appear, though, is in our desktop operating systems - or at least, it doesn't appear in operating systems yet. Could Windows 9 be Windows 3D?

Three kinds of 3D

In user interfaces, there are three kinds of 3D. There's 3D input, where sensors track an object - a mouse, or your hand - in three-dimensional space. There's simulated 3D output, where a two-dimensional display portrays a 3D environment: think Apple's Cover Flow, or Windows' Flip 3D, or Microsoft Photosynth. And there's real 3D output, where interface elements are displayed in three dimensions.

3D input is already here. 3D mice are used by architects, modellers and other professional types, and you'll find a 3D interface in millions of front rooms: Microsoft's Kinect, which can tell not only where you are and what you're wiggling, but how far from the sensor you're wiggling it.

3D connexion

ABOVE: 3D mice already exist: firms such as 3Dconnexion offer 3D mice for architects and other 3D specialists

It's clever and great for gaming, but it's also imprecise, tiring and needs an enormous amount of space to work effectively. Nevertheless, the newly published software development kit means Kinect is coming to Windows - although from what we've seen so far, it'll be a while before we're controlling Excel charts by waggling our fingers.

The 3D PC

3D PCs already exist - that's what Microsoft and Nvidia call their 3D technology for PC gaming - but will we ever see a 3D Windows? Phil Eisler is Nvidia's general manager of 3D Vision. "We believe that a 3D UI for Windows could work," he says.

"Windows already has different UI options including Basic and Aero Glass mode, which already uses some 3D effects and would be much better if the desktop was actually rendered in stereoscopic 3D. Icons could be organised in a volumetric way. I've seen designs based on an office, or blocks of a city, for example... the volumetric space of a 3D viewing system provides more space for organising icons, applications and other desktop elements to make it easier to work with. 3D has been used for years in data visualisation and so it can help with the understanding and retention of 3D graphs."

Kinect paint

BASIC: Painting with Kinect is cute, but it's hardly Minority Report. It'll be a while before useful Kinect apps appear

One issue that may be preventing widespread use of 3D is the requirement for special glasses, and as Eisler points out, it'll be a while before that particular issue is addressed.

"The problem with glasses-free is that is has trade-offs in resolution and viewing angle that are generally not acceptable," he says."The resolution is typically half or less, and the viewing angle can be as narrow as 10 degrees.Therefore, glasses will be required for the foreseeable future to get full resolution and a wide viewing angle that is required for an application like an operating system UI."

Glasses are becoming more comfortable, but some people can't use them for very long. Manufacturers of 3DSes and 3D TVs alike warn us not to let under-sevens view 3D displays, and some adults may have issues too.

As Larry Benjamin, Chair of the Education Committee for the Royal College of Ophthalmologists, explains: "Approximately 2-3% of the population can't see 3D because of early onset squint, or an eye problem that develops later in life - and you need good vision in both eyes to be able to see 3D."

UK charity The Eyecare Trust reckons that as many as 12% of people may find 3D uncomfortable because of uncorrected vision problems.

3D that isn't 3D

One solution that should work for everyone is pseudo-3D. The forthcoming Android 4.0 Ice Cream Sandwich uses head-tracking to monitor the movement of your face, adjusting the on-screen image accordingly to give the illusion of looking around a three-dimensional object.

That's not the only kind of 3D Google's looking into. In 2010, it acquired BumpTop, whose software turned a traditional PC desktop into something more like a three-dimensional desk. It makes navigation and organisation much more natural, and it isn't hard to imagine it working on an Android tablet or Surface system - anything with a touch screen or the ability to recognise hand movements.

Apple is clearly thinking in similar ways. MacRumors uncovered an Apple patent, filed in 2007, for a "multidimensional desktop" where your desktop extends to the horizon - think Time Machine, but for the entire OS.

It's quite possible that Apple merely patented an idea it has no intention of implementing, but the level of detail in the supplied images suggests otherwise.

Apple patent

OS X 3D: Could OS X 10.8 be in 3D? Probably not, but Apple's patents show a 3D OS that looks awfully familiar

The Sun Microsystems-sponsored Project Looking Glass takes a similar approach on platforms including Linux and Windows.

Rather than display objects that look 3D, future displays could actually change their shape. We already have prototype screens that you can feel with your fingers, and Microsoft is taking that idea even further.

Late last year, Microsoft filed a patent application for what it calls a "light-induced shape-memory polymer display screen" that uses a light-sensitive polymer to change its physical shape. Your screen could "grow" a keyboard, or a gaming D-pad, or Braille lettering, or anything else developers fancy making. Don't get too excited, though: the patent is an idea rather than a working piece of technology, so - assuming it's even feasible - such screens are several years away.

Project looking glass

3D PC: Sun Microsystems sponsors Project Looking Glass, a 3D overlay for Windows and Linux

To 3D or not 3D

You could argue that Microsoft's ill-fated Microsoft Bob (1995) was an attempt to make Windows more three-dimensional, and in the 1990s VRML promised a 3D internet. As you might have noticed, we aren't using Bob and the web is still 2D. Are things different now?

The hardware certainly is - 3D acceleration and technologies such as WebGL can deliver Doom in a browser - but 3D isn't just about technology: for it to become part of our operating systems it needs to address a specific need, and that means it needs to deliver something genuinely useful rather than the same old stuff with a 3D sheen.

So far, it hasn't done that. Gord Kurtenbach is Autodesk's director of research. "Many people believe that because in the real world we navigate 3D physical spaces with ease, 3D navigation should make navigation of 3D virtual spaces easy as well," he says.

"However, everything changes when the space is behind the piece of glass of the display monitor. Suddenly it becomes hard, like learning to fly an airplane using controls. So the old idea that your desktop will be easier to use if it's a 3D virtual space that you can navigate around hasn't really made things easier."

Will future operating systems embrace both 3D input and output? Nvidia's Eisler thinks so. "Gesture recognition systems such as Kinect are a natural extension of computer vision and seem like a very promising input mechanism for the future of computing," he says.

"They work very well with a stereoscopic 3D display system to create a more immersive and easier to use interface with your computer.Add voice recognition and a nice looking avatar and the computer may be able to replace the dog as man's best friend."