Ever wondered how Google actually works? The search giant has recently given a rare glimpse inside the vast data centres around the globe that power its services.
The firm has revealed how the intricate maze of computers that process Internet search requests, show YouTube video clips and distribute email for millions of people works revealing it now runs over 100,000 main ‘warehouse scale’ servers worldwide, and that they communicated at a blisteringly fast 10Gb/s in the ‘Jupiter’ network.
Google also admitted that to get the network performance it needed, it has been building its own hardware.
‘Pursuing Google’s mission of organizing the world’s information to make it universally accessible and useful takes an enormous amount of computing and storage,’ said Amin Vahdat, a Google Fellow.
‘In fact, it requires coordination across a warehouse-scale computer.
‘Ten years ago, we realized that we could not purchase, at any price, a datacenter network that could meet the combination of our scale and speed requirements.
One major reason for sharing information about its network now is that Google is opening up its infrastructure and offering Google Cloud platform services to others, Google Fellow Amin Vahdat told CIO Journal.
‘Having a great distributed computing infrastructure requires a world-class network to connect servers together,’ he said. Google said it would like developers at other companies to understand they can run jobs such as Big Data analytics on its infrastructure with reliable speed and performance.
The company has not shared this level of detail previously because it considers networking a key differentiator, he said.
Google says that its data center networks are some of the largest in the world and that others are starting to encounter the networking challenges faced by large-scale Web companies. As companies begin to handle Big Data and analytics in their own data centers, they’re faced with providing a data center network that can scale to handle the traffic.
‘While Google might have faced some of these challenges earlier, everyone is faced with these kinds of issues now,’ said Dr. Vahdat. ‘The amount of bandwidth you need within the data center to process through all of your data is enormous and growing.’
The data centers represent Google’s nerve center, although none are located near the company’s headquarters in Mountain View, Calif.
As Google blossomed from its roots in a Silicon Valley garage, company co-founders Larry Page and Sergey Brin worked with other engineers to develop a system to connect low-cost computer servers in a way that would help them realize their ambition to provide a digital roadmap to all of the world’s information.
Initially, Google just wanted enough computing power to index all the websites on the Internet and deliver quick responses to search requests. As Google’s tentacles extended into other markets, the company had to keep adding more computers to store videos, photos, email and information about their users’ preferences.
The insights that Google gathers about the more than 1 billion people that use its services has made the company a frequent target of privacy complaints around the world.
Google studies Internet search requests and Web surfing habits in an effort to gain a better understanding of what people like. The company does this in an effort to show ads of products and services to the people most likely to be interested in buying them. Advertising accounts for virtually all of Google’s revenue, which totaled nearly $23 billion through the first half of this year.
Even as it allows anyone with a Web browser to peer into its data centers, Google intends to closely guard physical access to its buildings. The company also remains cagey about how many computers are in its data centers, saying only that they house hundreds of thousands of machines to run Google’s services.
Google’s need for so many computers has turned the company a major electricity user, although management says it’s constantly looking for ways to reduce power consumption to protect the environment and lower its expenses.