Published On: Thu, Dec 20th, 2012

Seeing Ghosts: merging the real world with the virtual world

Share This

Science fiction provides an endless parade of humans living in a virtual reality world.  Tron, the classic 1982 Disney science fiction film where the hero enters a virtual world is perhaps the best known original version of this virtual world.   The Matrix, which came out in 1999, took the “virtual reality” level of the film to the next level, whereby humans did not even realise that they were living in a virtual world.

The problem with living in a virtual world is not that computer animation is not good enough to imitate real life (the graphics of films like Avatar should be available on your phone within 10 years) but that most people are quite happy living in the “real” world and would not want an existence that is completely virtual.

History Future Now looks at five of technologies that already exist and argues that by integrating their attributes you could create a merged virtual / real world which would have the best features of each of them and would be perfectly liveable.


1.  Augmented Reality

First up, augmented reality.  There are dozens and dozens of augmented reality apps available for the iPhone.   Many of them are glorified map applications: point your phone in a particular direction and the phone will use your location and the direction you are pointing your phone in to provide information about what you are seeing.

Another app, for example, allows you to look through the camera function of your phone into your room and to superimpose furniture in it, to see what the furniture would look like in situ, before you buy.

The most interesting ones, however, are games that allow you to mix up the virtual world and the real world.  ARDefender, for example, requires you to print off a small piece of paper with a particular pattern on it.  The phone sees the patterned paper through the camera lens and then uses it as a reference point for the game.  As you move your phone around the paterened paper you can see “virtual” buildings, tanks and cannons set against the backdrop of what are are seeing in the real world.

2. Green Screens

Green screens are the mainstay of modern special effects.  Films like The Hobbit are stuffed full of scenes that are played out with green screen backdrops.  Actors move around on a set that is covered in green materials.  Once the shot is finished, the computer animated scenery is added into where the green screens were previously.

Click on the video link below to see how it is done.

3. Motion Capture animation

Films like Planet of the Apes, Avatar and The Hobbit use actors to provide all of the movements of characters such as Cesar (the Chimpanzee), Sagar and Gollum.  That is why the performances look so realistic.  Tiny dots on their faces and bodies are picked up by a camera which then translate the dots into a digital wire frame which is then used to move the computer animated character.


4. Headset computers

Microsoft, Apple and Google are all developing computers that can be embedded into your glasses.  The most widely known of these concepts is Google Glass, which was shown off by Google co-founder Sergey Brin a few months ago.  Google Glass contains a video camera, to capture everything that you see, with a small screen that allows you to get information about what you are seeing – like the Augmented Reality Apps described above.


5. Transparent screens

The next technology that exists are transparent screens.  This combines a real world background and a virtual world foreground.  Samsung demonstrated one in January 2012 at CES, to which it added a touch sensitive display.  If you incorporated this technology into headset computers you would be able to see both the real and virtual worlds simultaneously, without looking too ridiculous.

Putting this all together

Imagine you are at home.  Along the entirety of the corridor going from your dining room (do people have these nowadays?)  to the kitchen are green screens – the floor, the ceiling and the walls.  In your dining room, which has green screen over one of the walls with a balcony in front of it, there are a number of places set at the dining room table.  At one end of the table is a patterned area.  In the centre of the table is another patterned area.  You are wearing clothes with small dots and have applied green dots on your face.

You put on a pair of glasses and your whole world changes around you.

What was once a green coloured corridor is a corridor with an expensive, mosaic floor.  On the walls are Roman frescos, that would not be out of place in a Roman palace.  In the dining room, on the other side of the open balcony is a magnificent view overlooking the Bay of Naples, circa 60AD.  The centrepiece on the table has a gold and silver bowl, covered in intricate scenes of wild animals.  At the end of the table, looking out at you is your dinner guest, who is looking back at you seeing the identical scene from their house 200 miles away.  Small protruding cameras from  your glasses look at the green dots on your face, allowing your digital image to be presented to your dinner companion through their glasses.

Unlike many future technologies, which require huge leaps of imagination to make them happen, all of the technologies described above exist today.  They also don’t require huge amounts of expensive equipment to make them viable.  Green screens are literally just green coloured cloth, or green paint. To create the patterns on a table which allow for the gold and silver bowl you need some masking tape.  The glasses themselves, should be relatively inexpensive.  The screens will be tiny OLEDs similar to what is used in some DSLR cameras, the cameras are already small and cheap. iPhones are only as big as they are because of the big screen, which requires a big battery.  Within a few years the glasses should be as cheap as a smartphone is today.

How would this change the world?

Humans are primarily visual animals.  We like nice things to see on our walls (paintings, sculptures for example) and nice views from our windows. Green screen walls could allow us to have the best views in the world and the finest art.  All for the price of a smartphone and a lick of green paint.

For some people, as these virtual scenes became almost indistinguishable from real life, this would be sufficient.  They would no longer be willing to pay a significant premium for the real thing. This would have an impact on house pricing and art, amongst other things.  It would also have an impact on the physical things that people buy – if you can see anything you want to see, how much of it needs to be actually real?

It would also change the concept of meeting. If someone can join you “physically” in your room and appear to sit in the chair next to you how would that impact on relationships? You could potentially have dozens of people meeting all at the same time, at a real table, with patterned marks showing where they are sitting.  You could swap seats with people instantly – allowing you to chat to the people that you want to.

This merger of the real and virtual world combines the best of both.  With some glasses on you can augment your reality – take them off and the virtual world falls away, like shadowy ghosts.

How do you see this changing the world?




History Future Now, ebook edition, is now available from the Apple iBookstore!  So if you have a iPad or iPhone click on this link to download it.  It is currently on at a special offer of 99c.   The Kindle version has been submitted to Amazon and should be available shortly.

HFN on Twitter