Science inspiration networks for Google, Microsoft and Apple
By André Vermeij, Kenedict Innovation Analytics
The inventions by large companies such as Google, Apple, and Microsoft are often seen as the direct result of significant investments in Research & Development. What’s less visible, and perhaps too often overlooked, is how some of the world’s largest corporations are actually inspired and influenced by the scientific activity that preceded their inventions. Which papers have they cited while working on their inventions? How do these papers relate to some of these companies’ key products or services? An answer to these questions can provide us with a unique view on the impact of science on industry.
Text continues after the interactive visualization below
Viewing this on mobile? Click here to view the interactive visualization full-screen
Connecting patents and science
The patent activity of large tech companies such as Google, Apple and Microsoft has soared over the past decade, with each filing thousands of patents each year. Just like scientific publications cite other publications, patents also contain citations to previous work (or, prior art). The most commonly found citations on patent documents are references to other patents in the same field. Next to this, a significant number of patents also cite what is called Non-Patent Literature (or NPL), which includes a wide variety of sources including web pages, blog articles, device manuals, conference proceedings and journal publications. The last two are especially interesting: these documents are the output of scientific activity, and could thus be used to connect scientific knowledge to subsequent inventions by businesses.
A key challenge when working with the NPL citations on patent documents is that they have no unified formatting and unique identifiers attached to them. Each citation is simply listed as a line of text delivered by the patent applicant or examiner, resulting in many different ways of referring to the same citation across a dataset (for example, journal title abbreviations vs. full journal titles, inclusions/exclusions of page numbers and volumes, different ways of writing author names, etc.). This makes it hard to accurately connect unique scientific publications to the patent documents which cite them.
Luckily, open patent search platform The Lens have recently released PatCite [1] (paper here), a tool which allows one to explore the linkages between patents and cited scientific publications based on matchings of the raw NPL text to unique identifiers such as DOIs (Digital Object Identifiers). Based on this excellent initiative, it is now finally possible to link patents to related scientific activity and vice-versa on a large scale.
With the help of this data, let’s take a look at the patents published by Google, Apple and Microsoft over a 3-year time period (2014 to 2016 – full data for 2017 was not available yet in PatCite at the time of writing) to find out in how far they are inspired by science.
Science Inspiration Networks
For each of the three companies, maps were created showing the key areas in which science plays an important role based on the citations to scientific publications in these areas. The maps are presented as networks, where patents (circles) are connected to scientific publications (squares) whenever there is a citation relationship between them. To focus on the most important sources of inspiration, the maps were filtered to only show scientific publications which were cited more than 4 times. Colors were assigned to groups of densely interconnected patents and publications by using a community detection algorithm, and documents are sized based on the number of connections (degree in network parlance) they have.
Labels were assigned to a selection of areas based on an analysis of their contents. Of course, there is much more to be found here. You can search and explore the maps yourself as well using the interactive tool at the top of the page. Clicking documents show some more information in the panel on the right-hand side, including links to the Lens for patent documents and to Google Scholar for scientific publications.
Key areas in which Google cites scientific activity based on patents published in 2014, 2015 and 2016
Google is inspired by science in many of its key technology areas. The red cluster on the left-hand side relates to all kinds of video and image processing, encoding and compression topics, and may relate to the techniques used by YouTube to encode and save videos on its platform. The Document Digitization / Character Recognition cluster may be related to Google Books, and includes citations to articles such as “Extraction of Text Areas in Printed Document Images”, part of the proceedings of a conference held in 2001. In the Search Technology cluster, we find citations to papers like “Learning user interaction models for predicting web search result preferences”
The scientific publications which are cited most frequently by Google relate to social networks, located at the top right of the map. A total of 108 patents cited an article titled “Social Translucence: Using Minimalist Visualisations of Social Activity to Support Collective Interaction”, which may have been a key source of inspiration for techniques used by Google on its social platforms to further stimulate user interaction.
Key areas in which Microsoft cites scientific activity based on patents published in 2014, 2015 and 2016
Of the three companies analyzed here, Microsoft takes the number one position in terms of the number of scientific publications it has cited. It takes inspiration from scientific output in many areas. The Motion Tracking / Gesture recognition cluster may be directly related to the HoloLens, and includes references to papers such as “Focus of attention for face and hand gesture recognition using multiple cameras”, which was cited by a total of 104 patent documents. The Speech Recognition cluster seems related to Microsoft’s Cortana digital assistant, with an example citation being “Context-Dependent Pre-Trained Deep Neural Networks for Large-Vocabulary Speech Recognition”, a paper from 2012.
The Virtual Machines / Remote Computing cluster at the top right seems related to the company’s Azure cloud platform. There’s a lot more to be seen in this map, so don’t forget to check out the interactive versions above to generate your own insights.
Key areas in which Apple cites scientific activity based on patents published in 2014, 2015 and 2016
Compared with Google and Microsoft, Apple’s citation patterns are focused on a smaller number of key technology areas. In line with expectations, touch screens and tablets take up a significant portion of the activity here. Particularly interesting are the many references to a paper from 1985 titled “A multi-touch three dimensional touch-sensitive tablet”, published no less than 25 years before the introduction of the first iPad in 2010. This paper was co-written by Bill Buxton, who currently works at Microsoft Research.
Another group of topics in which Apple takes significant inspiration from science relates to voice recognition, text to speech and digital assistants. We can easily connect this to Siri, the digital assistant included in Apple’s operating systems. An example paper is “PTIME: Personalized assistance for calendaring”, a publication from 2011.
A few years ago, Apple acquired PrimeSense, an Israelian company focused on 3D sensors. As can be seen in the cluster at the bottom, the patents related to this transaction also connect to various scientific sources. And according to The Verge, the tech in this cluster is part of Apple’s iPhone X.
Science-industry linkages and the way forward
Patent and scientific publication data were previously often analyzed separately because of the challenges associated with connecting the data together in a meaningful way. Combining new tools such as PatCite with the power of network visualization, it is now possible to connect these data sources and gain a strong visual view on the linkages between scientific output and related patent documents.
Network visualizations such as the ones presented above can help users to gain insights and interact with the data in new ways. From a practical perspective, businesses can now gain new insights with regard to the influence of science in their target markets, and visually assess in how far peers are taking inspiration from scientific discoveries. In the same vein, academic institutions, authors or journals can assess the impact of their output in a more data-driven way, and obtain new measures and performance indicators with regard to the influence of their scientific output in industry.
There are various other interesting questions which may be worthwhile for further analyses. For example, in how far are the citations to science in patent documents based on self-citations (for example, Google citing articles which are written by Google employees)? Also, to get an even better view on direct science inspiration by inventors, it would be valuable to distinguish between citations added by the inventors themselves versus the citations added by examiners at patent offices. Finally, a very interesting additional perspective would be to look at the citations of scientific output which are shared across organizations – for example, in which areas do Google and Apple both cite the same documents?
If you have any questions based on this analysis or if you want to share your own insights based on the interactive visualizations, please leave a comment below or drop me a line on Twitter.
[1]: Jefferson, O.A., Jaffe, A., Ashton, D., Warren, B., Koellhofer, D., Dulleck, U., Ballagh, A., Moe, J., DiCuccio, M., Ward, K., Bilder, G., Dolby, K., & Jefferon, R.A. (2018). Mapping the global influence of published research on industry and innovation. Nature Biotechnology, vol. 36, pp. 31-39.