Miles Ward, designer and developer for Amazon Web Services. (vimeo.com)
"What I'll show you is incredibly tiny teams, little... groups of people, two people in one case, seven people in another case, building systems that serve millions of users," Ward told the Cape Town Scale Conference.
Ward used two examples of projects he worked on personally. One example is the Curiosity Mars Rover landing and the other was the "Obama for America" election process.
"First example is the NASA JPL (Jet Propulsion Laboratory) stuff... They had this... interesting challenge - they [were] going to land, basically a Mini Cooper with lasers on its head on Mars," said Ward.
However, his team faced a challenge when JPL was not allowed to use any of the tools from NASA. "NASA says, sorry, peace out JPL, you can't have our stuff. We also had six days notice."
Ward then touched on intricate details pertaining to handling a large amount of traffic to NASA and the Curiosity Rover's websites, with a large quantity of visitors accessing live high definition video streaming content without the entire system crashing.
Ward said the plan was to "design a solution that deals with the complexities of live streaming”, without the advantages of any of the pre-built systems for live streaming.
Ward said they only had "two guys" to build a system from scratch. The advantage they had was one of the two was a rocket scientist.
"Another project, also interesting... Let's imagine we're going to elect a president. What we're probably going to have to do is raise a lot of money, so much money in fact that we built the thirtieth biggest e-commerce system in the world," said Ward.
Ward said that for the Obama for America campaign, a billion dollars had to be raised.
He added the team would have to commit to processing a billion dollars worth of payments, "that in itself is a complicated, big awful thing to build".
Furthermore Ward said the team had to build 200 distinct applications, "totally different pieces of software".
"On top of that they have 50 full time query analysts, looking at hundreds of terrabytes of data trying to figure out what's going on in the election," said Ward.
Ward said the team was worried about the project and its technical challenges.
He said they often thought: "Our operations will be so scary we will never get things done and... everything will be so fragile.”
Despite all the challenges faced, the teams in question achieved the seemingly impossible. The Mars Curiosity Rover landed successfully on the red planet and Barack Obama was re-elected as president of the United States.
Furthermore Ward stressed AWS did not support any specific political candidate. AWS was approached for the services they could provide.
