Kenedict Innovation Analytics

Who’s Doing What in Virtual Reality? A Network View on Technological Activity

Virtual Reality has recently received widespread renewed attention through the initial success of the Oculus Rift and its subsequent acquisition by Facebook in 2014. Ever since, other market players announced upcoming products or early teasers, including Sony, HTC, Microsoft and Google-backed startup Magic Leap. Apple recently also entered the market through the acquisition of German augmented reality company Metaio. What do the technology networks of these key players look like, and how can we relate these to their upcoming product introductions? Read on to find out more.

Written by André Vermeij, Kenedict Innovation Analytics

Virtual Reality (VR) has been around for quite a while actually: one of the first patent applications was filed in 1961 by cinematographer Morton Heilig, describing the ‘Sensorama simulator’. The Sensorama showed 3D movies, had stereo sound, and was even able to produce wind and smells to accompany the virtual experience. Unfortunately, Heilig was unable to secure the necessary financial backing for the project, and ultimately had to abandon it. In the following decades, many attempts were made to commercialize VR products, though with limited success. Successful corporations such as Nintendo also jumped the bandwagon, but the company’s ‘Virtual Boy’ turned out to be a commercial failure. Consumer prices for other VR products were usually too steep, while the then available technology lacked in terms of display resolutions, viewing angles, etcetera. For more on the history of VR, check out this excellent feature on The Verge: The Rise and Fall and Rise of Virtual Reality.

Sensorama

The Sensorama

This all changed in 2012 when Oculus suddenly raised $2.5 million on Kickstarter to further develop the Oculus Rift. Things have gone quickly since then: Facebook acquired Oculus for $2 billion in 2014, and many other leading consumer technology companies announced Virtual Reality or Augmented Reality products at around the same time. Sony has PlayStation VR (previously Project Morpheus), Microsoft announced the HoloLens, HTC and Valve teamed up to create the Vive, Samsung worked with Oculus to create the Gear VR, and startup Magic Leap received $542 million from an investment consortium led by Google. On its own, Google also introduced the Cardboard: a low-cost, do-it-yourself VR solution made of (surprisingly) cardboard, which can hold any modern smartphone and provide a basic VR experience. Most of these products (except for the Gear VR and Cardboard, which are available already) are set to be released during the first half of 2016, so we have an exciting time ahead of us.

Of course, all these technological developments are backed with significant Research & Development investments, often leading to various patent applications to protect the underlying technologies. Oculus VR, the company behind the Oculus Rift, currently has at least 8 published patent applications, including the design of the headset (see the picture below) and technologies relating to predictive tracking and system calibration.

The design of the Oculus Rift, as depicted in patent USD738374S1 (https://patents.google.com/patent/USD738374S1/en)

The design of the Oculus Rift, as depicted in patent USD738374S1 (https://patents.google.com/patent/USD738374S1/en)

What have the other main players in VR technology been focusing on so far? Let’s take a look at all their published patent applications over the past three years to find out.

Data & Search Strategy

All of the companies above have built up significant patent portfolios over the years, covering most of their technology areas and providing a glimpse into the interests which they’ve had recently. Even Magic Leap, despite being a relative newcomer, currently has over 175 published patent applications attached to its name since its first application in May 2012. For each of the companies mentioned above, we thus obtained the patent applications published between January 2012 and now (October 2015) from the databases of the European Patent Office. We then filtered the data by focusing on keywords relating to Virtual Reality or Augmented Reality technology appearing in titles or abstracts, including for instance head-mounted displays, immersive entertainment, or more exotic related concepts such as ‘hybrid reality’ or ‘mediated reality’.

Virtual Reality Patent Networks

Each patent comes with a wealth of useful metadata on its frontpage, including applicants, inventors, technology classifications and citations. All these bits of metadata can be used to connect entities together in a network, to ultimately be able to get a view on the actual relationships residing within a dataset. Since we’re investigating the VR-related patents of specific companies here, it would be worthwhile to map the interrelatedness between patents through a common characteristic. Assuming that each of these companies have dedicated Research & Development teams working on VR, let’s connect patents in the networks when they share at least one inventor – when doing this, we’re actually connecting patents through the knowledge sharing which has taken place among inventors and across technologies within an organization. Each patent was subsequently algorithmically assigned to a cluster, resulting in clearly identifiable related technologies through the colors in the network. Patents with the same colors are thus part of the same technology group and are related through the knowledge sharing which took place between the inventors listed on these patents.

The interactive visuals below allow you to go through each of the networks and gain your own insights. Clicking a patent shows more information. Patents close to each other with the same title are part of the same patent family. A full-screen version of the visuals can be found here.

What The Networks Tell Us

The visuals show that most key players’ networks show clearly identifiable technology clusters. Sony’s network shows various large clusters related to head-mounted displays, which suggests that there seem to be multiple separate R&D teams working on VR/AR-related technology. The largest group of connected patents, however, relates to activity of Sony’s recent acquisition Softkinetic – a company specialized in 3D sensors and gesture tracking.

Microsoft’s network is more integrated – it shows one highly interconnected cluster holding technology related to holographics, head-mountable displays and augmented reality environments – all of which can be related to the upcoming HoloLens. Samsung’s largest group of interconnected patents focuses mostly on virtual and augmented reality by making use of a mobile phone – which is quite in line with what the Gear VR accomplishes.

The bulk of startup Magic Leap’s 178 patent applications resides in one large cluster. This is mainly due to the activity of Brian Schowengerdt, who appears on a wide variety of applications and thus serves as the connector between them. Founder Rony Abovitz is also listed as an inventor on patents at the left-hand side of the main cluster. Two other inventors are responsible for their own, separate clusters: John Macnamara (Senior Creative Scientist at Magic Leap according to LinkedIn) and Chunyu Gao (currently president at Augmented Vision Inc.).

Google’s VR/AR-related patents reside in a number of interconnected clusters. Some of the patents found here can actually also be related to the Google Glass (such as ‘Proximity sensing for wink detection’). Most activity seems to be in eyepieces for head-mounted displays.

Interestingly, when running a search for patents applied for by HTC and including the VR-related keywords mentioned above, we received 0 results. Upon further inspection of HTC’s full technology network, it became clear that there do not seem to be any currently published patent applications which relate directly to the Vive – most probably, these applications (if any) are still to be published (the usual delay between application and subsequent publication is 18 months). On the other hand, HTC’s partner Valve did apply for a number of patents which can most probably be related directly to the Vive, including head-mountable displays and wearable input devices.

The overall patent activity by Facebook and Oculus is relatively limited as well, especially when compared to Sony and Microsoft – Facebook holds 5 VR/AR-related patent applications in multiple jurisdictions (all relating to the same invention), while Oculus holds 8. Including the patents of Oculus’ recent acquisitions (such as Pebbles and 13th Lab), the total comes down to 23 patent applications. Two of the Oculus patents relate to the design of the headset, as depicted  in the picture above – the Facebook patents describe a ‘Method relating to presence granularity with augmented reality’.

Apple recently entered the Augmented Reality space by acquiring Metaio, a German company which has been at the forefront of AR for years. Metaio filed over 180 applications over the years, adding to the 14 patent applications which Apple filed on its own. As is visible in the visuals, Metaio’s portfolio consists of two connected large clusters and various smaller ones around them. On its own, Apple filed a number of applications related to head-mounted displays, including the much discussed application which describes a device which sounds very similar to Samsung’s Gear VR, but was applied for as early as 2008 (see here: Head-mounted display apparatus for retaining a portable electronic device with display).

The Way Ahead

The significant attention for the Oculus Rift back in 2012 paved the way for a renewed spur of innovation in virtual and augmented reality technology. Many large consumer technology companies have jumped the bandwagon and are set to release products next year. The patent networks in this article provide a glimpse into the internal workings of these companies.

Since VR has a long history, it would also be interesting to get a further view on the evolution of the underlying technologies through an analysis of patent citations. Today’s VR patents cite earlier patents in the field – when diving into multiple ‘citation generations’, it’s possible to obtain a visual overview of the key patents and clusters of related technology around them. In the same vein, there are numerous scientific publications which deal with Virtual Reality/Augmented Reality. A quick search on the IEEE databases shows that there are almost 4,000 documents which list VR or AR in their titles, which can serve as a great source of more information on the latest developments in this specific technology field.

Based on the current market buzz and technological activity of many leading companies, the future for VR/AR looks promising – let’s hope that we’ll soon finally be able to fully reap the benefits of the technology.

1 Comment
Leave a Reply

Kenedict’s mission

To significantly improve Research & Development organisations’ decision-making by ensuring day-to-day usage of a social network perspective in the areas of Open Innovation, Competitive Intelligence, Human Resource Management and R&D Policy.

Contact Kenedict

Phone: +31 645 956 845

E-mail: info@kenedict.com