This is an extended text version of a talk first presented at Voxxed Days Belgrade in September 2016 (voted best talk of the conference) and as a keynote at DevExperience in March 2017. A video recording is embedded below.

Good morning.

Today, over 3 billion people carry smartphones in their pockets. Computers are steadily getting smaller, cheaper and more capable. Countless industries have been reformed by the computer. And there are many exciting things happening in the computer world today. We’re able to see driverless cars, we have the entire world’s knowledge at our fingertips, virtual and augmented reality applications take us to other worlds or enhance our current one and artificial intelligence is already all around us.

It is quite easy to get the idea that the current state of the computer world is the climax of our great progress.

But today I want to start with a different story, a story about my father.

My father was a private pilot and when I was growing up his stories about flying were a frequent topic at our dinner table.

There is one story I remember vividly. My father grew up in a farmhouse in the countryside of Argentina. In those days ambulances were not readily available so a young group of boys and men flew around in their small airplanes, collecting casualties to bring to their nearest hospitals.

One night, they got a radio call from a small town in the jungle only accessible by air, to say there had been an accident. My father was only 18 years old at the time so his mother strictly forbade him from flying, especially at night. But on this night after receiving a call on his walkie-talkie, my father escaped through his bedroom window, cycled to the airport and flew to the small town to collect the injured.

My father’s stories affected me and my brother on a deep level. We both started drawing airplanes, building airplane models, flying them, watching movies, documentaries and enjoying nerd talks on aviation. My brother even tried to become a fighter pilot for the military.

Aviation was in my blood and it was inevitable I would one day want to be a pilot myself. After 10 years of saving and while watching friends of mine going on world trips, I stuck to my decision to learn to fly. I hope you’re able to see why I am personally so affected by airplanes.

I find the evolution of airplanes so interesting. Before airplanes, continents such as America were traversable only once or twice in a normal person’s life. Immigrants from Europe, Africa or Asia knew they would never return to the place they left behind.

But then with air travel, the world was made smaller and the nature of work changed.

Computers followed a similar evolution. The ENIAC was the first general purpose computer. It was a huge machine, costing about half a million dollars at that time, weighing almost 30 tons and needing several people attending to it at all times to function. Programmers debugged problems by crawling inside to find bad joints and bad tubes in the massive structure.

Today, our computers and devices make the world smaller and faster, changing the nature of work and allowing us to stay in contact with our loved ones all around the globe.

So before looking at the evolution of airplanes and computers in detail, it is important to make two things clear.

First, we must make a distinction between a technology and a medium. A technology is a physical apparatus, a medium is the use to which a physical apparatus is put.

A technology becomes a medium when it is given a place in our society and culture. In other words, a new technology is just a tool we can play with, but a new medium changes everything.

The airplane as a technology did not introduce transportation or travel into human society, but the airplane as a medium accelerated the rate of transportation, creating new kinds of work and leisure.

The second point I want to make is:

We shape our tools and our tools shape us.

The person who said this was Marshall McLuhan. While being a quite controversial academic figure, McLuhan’s work is viewed as one of the cornerstones of the study of media theory.

This saying comes from this book, “Understanding Media” which came out in 1964. McLuhan proclaimed that it is the medium itself, and not content that impacts our society and culture. He said each new medium whether it’s air travel, the car, radio, TV, the telephone or even computers creates a new environment for people to act in.

All media act as extensions of our human capabilities and when you introduce a medium into a society it affects our outlook and attitudes, our feelings about how we relate to the world.

I want to show you a short video with some of his words to make this point more clear:

Let’s have a look at the amazing technological progress we made both in air travel and computers.

This is one of the most amazing and successful airplanes ever built.

The Boeing 747 is nicknamed the Jumbo Jet or Queen of the Skies. Even if you’re not an aviation expert the 747 is easy to tell apart because of her size and her distinctive “hump”.

Both the 747 as well as her immediate predecessor, the 707, heralded the era of modern, intercontinental mass transportation. Their popularity led to rapid developments in airport terminals, baggage handling and air transport infrastructure. They truly paved the way for jetliners to become what they are today.

The 747 is not only an impressive result of an intensive engineering and design process, but also the spirit of its era, a time when belief in progress, including access to air travel, was a phenomenal force.

As such there are so many wonderful stories about this plane. It was two and a half times bigger than the largest passenger jet ever built at that time. They had to build a special factory to assemble it, and they were still building the factory as the first planes came off the line.

My favorite fact about the 747, however is that it was meant to be a stopgap.

Boeing expected to sell a couple of hundred units or so and only serve the airlines until planes of a supersonic era entered service in the 1970’s, at which point it would find more use as a container-carrying freighter. That famous hump on the front of the 747 was put there specifically to make it easier to load freight. This was not a plane with a glamorous future.

All the top engineers and the ambitious newcomers had ensured they were assigned to Boeing’s prestige project, the Boeing 2707 SST, an airplane designed for supersonic transport.

The 2707 was intended to fly at three times the speed of sound, about 2900 kilometers an hour. It had swing wings and it was the first-ever widebody design. Everyone believed that supersonic transport was the future of jet travel. This Pan Am advertisement from the period showcases the supersonic planes of the future.

But it wasn’t just Boeing working to build a supersonic transport. The Europeans were developing the Concorde.

And the Soviet Union was hard at work on their own version, the Tupolev 144.

Around the same time, Ivan Sutherland created Sketchpad. Sketchpad is considered to be the ancestor of modern computer-aided design programs as well as a major breakthrough in the development of computer graphics in general.

It was the first graphical, object oriented system based entirely on drawings and constraints given by the user. Here you will see a few lines being drawn and then defined to be perpendicular. Without knowing anything about the intent of the user, Sketchpad automatically translates the constraints into a reusable result. Each drawing may then be used as a master and be included in other drawings.

And while Ivan Sutherland was still tinkering with his system, the military was especially interested in supersonic planes that could either escape the enemies anti air defense or drop an atomic bomb and escape quickly, leaving only dust behind.

The supersonic transport airplanes were as it happens only a watered down version of what the military was planning and prototyping. Beginning with Chuck Yeager’s X1 and the first supersonic flight in 1947, some crazy design inhabited our skies.

This is the XB-70 Valkyrie, a strategic supersonic bomber which was able to reach speeds of over 3.200 km/h.

Or the SR-71 Blackbird, which still holds all the speed records. Being able to fly at over three times the speed of sound, its engineers had to come up with some creative solutions. It was leaking gasoline when standing on the ground, and would only stop leaking at those speeds due to the high temperature.

Just a few years later, Doug Engelbart, one of the fathers of personal computing, presented his vision of augmenting human intellect with the help of computers.

Using technology to improve human capabilities, he wanted to boost our collective intelligence. But he saw this as only a starting point of using tool-augmented behaviour and habits to further refine our tools in a continual co-evolution.

In 1968 he showcased the prototype for his vision, the NLS, also known as the Mother of all Demos. It was given this nickname as it introduced so many concepts we still use today. The computer mouse, windows, graphics, video conferencing, word processing, hypertext, revision control or a collaborative real-time editor.

And then there was space flight. A year after Engelbart’s presentation, the Americans landed on the moon while the Russians were starting to populate space. Convenient travel to the moon felt possible and imminent at that time. Pan Am had already started taking reservation requests to fly to the moon.

Just a few years later, Alan Kay, one of XEROX Parc’s prodigies, brought up a prototype for a collaborative platform for children, the Dynabook. It was a highly interactive, low-cost, battery-powered, portable computer with a touchscreen and graphics, able to access and share information over the network. He presented a detailed vision for the touchscreen tablet decades before it was practical.

Though much more than a tablet, it was envisioned as a dynamic medium you could play and interact with.

And it goes on. A decade later, designers were still going crazy inventing unconventional airplanes like the F-117 Nighthawk, the first stealth plane. It’s no wonder these times in history feature so many UFO reports. It was a glorious era of aviation.

Let’s have a look at the most advanced passenger airplane of today’s age, the Boeing 787. It features thousand-fold improvements in every technological dimension. It’s just mind boggling.

But if you aren’t an aviation nerd like me you’d be hard pressed to tell it apart from the predecessor of the 747, the 707, that was first build in 1958.

The 707 flew even faster at 960 km/h. Of course the technologies used in a 787 are much more advanced but if you take a step back the differences are quite minor if you compare them to the wild dreams we had.

You see the next generation of technology was not just a dream. It was already in the prototyping stage.

But then it all just stopped.

Few people needed to cross an ocean in three hours instead of six hours. When asked what they would do with those additional 3 hours, people said they would watch TV. So they put TV sets in the 747.

Being able to go anywhere in the world in a day is really good. A couple of thousand dollars might be expensive and those hours might feel cumbersome but not that expensive when you consider that overnight you can move yourself to virtually any point on the earth.

The people designing the supersonic planes of tomorrow became so caught up in the technology that they forgot to ask what role the new medium should play in our society. And then there were matters such as environmental damage, recessions and money cuts that caught up.

The same thing that happened to aviation is happening with computers.

Everyone takes the structure of the computer world as god-given but the computer world deals with arbitrary stuff that all was made up by somebody. Everything you see on your computer, smartphone and any other device was made up by someone.

And the computer world is not yet finished.

Yet everyone is behaving as though everything was known. At 50 years into the computer revolution, it feels like our moment of greatest progress.

This is not true.

Today’s omnipresent graphical user interface has its roots in Doug Engelbart’s ground shattering research of the mid-’60s. The concepts he invented were further developed at XEROX Parc in the ’70s, and successfully commercialized in the Apple Macintosh in the early ’80s, whereupon they essentially froze. Thirty years later, despite thousand-fold improvements along every technological dimension, the concepts behind today’s interfaces are almost identical to those in the initial Mac.

Our interface is based on metaphors with the familiar non-computer world around us. Files are represented as documents in paper folders that are placed on a desktop. They are deleted by dragging them to the trash. Nevertheless some metaphors and features work very differently in the real world. Our adaptations constrain and mislead users, and limit designers’ ability to invent more powerful interface mechanisms.

Analogies of the real world go on. Documents get written and read in sheet of paper-akin programs. Computer code is drafted line by line. Almost all of today’s representations were designed for the medium of paper. This limits us to pencil-and-paper thinking.

Sharing and collaborating is utterly hard. We deal with incompatibilities and although we have Dropbox and Google docs, collaborating on a document or design, or sharing holiday photos gives us grief.

Even before we can collaborate, we still need to manually exchange our contact information.

I was invited by Ana and Vesna to give this talk. We connected over LinkedIn and Email and had quite a conversation going on. Nevertheless my phone has no idea how I could reach them. Forget about it if I move abroad or change my phone number.

Why do we even need a phone number?

If we take a look at our phones we see a big screen made of glass that you interact with by sliding your fingers along a flat surface.

Now take a real glass — of water. You know how much water is left, first by seeing it but also by feeling the response of the weight. Almost every object in the world offers this sort of feedback.

Our hands have an incredibly rich and expressive repertoire, and we improvise constantly without the slightest thought. And yet, with an entire body at your command, we reduce our interaction with mobile devices to a single finger.

I could go on. I really could. But here is the thing:

We shape our tools and our tools shape us.

It is not as much a technological problem as a lack of perspective. We could easily, one by one, fix the problems I just described.

Or we could shape something bigger, like make use of computers to augment our most human capabilities. We have to start thinking about how we want to shape the computer in order to improve our society, our culture.

I want to show you a short video by someone you probably know.

I just love Steve Jobs’ idea of comparing computers to a bicycle for the mind. There were so many people thinking along these lines even before Steve Jobs.

I showed you three projects from our computer history earlier on. You may have wondered why.

Ivan Sutherland’s Sketchpad seems at first glance like a drawing tool but it was so much more. It clearly established the first true fundamental steps toward an interactive human computer user experience. And most importantly his focus was on creating a tool that would fundamentally amplify human capabilities.

For example, it was used to plan and build bridges, just by adding constraints. Remember, Sketchpad had no idea about gravity or how the strength of beams could be calculated. It computed its solution based on the restrictions given by the user. I was struck by how a computer was able to solve such a difficult problem without knowing which problem it’s solving actually is.

This is how collaboration between computers and humans should be in a symbiosis between man and machine.

Doug Engelbart’s vision was a collaborative system of people working together in a shared intellectual space.

He saw people collaborating and working together remotely and his vision of augmenting human intellect was much bigger than his prototype could show.

Let’s see a bit of his 1968 demo. At one point, we see the face of a remote collaborator, Bill Paxton, and Engelbart and Paxton have a conversation.

You’ve probably seen similar things if you use Skype or Hangouts but then they start working simultaneously with the document on the screen.

Look even closer and you’ll spot there are two individual mouse pointers. Engelbart and Paxton are each controlling their own pointer. And they are working together in a shared intellectual space. When I first saw this it blew my mind.

It is so far from what we have now. Our computers are designed with a single-user assumption. Mirroring a display remotely doesn’t magically transform them into collaborative environments.

Alan Kay’s Dynabook was a collaborative environment for children, a dynamic medium fostering creative thought.

His notion was of being able to play collaboratively especially games they could make themselves. He envisioned the Dynabook as a dynamic medium which could be easily learned from and extended.

Many more great people thought along the same lines. There were tangible user interfaces. The next example is a concept created by Durrell Bishop in 1992 and features small marbles that hold the information you need.

When I met him last year, he told me that this was only the first step. Marbles should hold your information and be usable by all kind of different devices — placing a marble next to an audio speaker to hear voice notes, or next to your display to see relevant information in a message, such as maps or invitations. Unfortunately it never made it out of prototyping stage.

There are exciting projects afoot. This is a recent project by a design student from Berlin, Lennart Ziburski. His project, Desktop Neo, is a conceptual desktop interface.

Federated wikis, most prominently pushed by Ward Cunningham and Mike Caulfield, are collaborative environments which allow you to jot down ideas, write reactions to readings, organize your own thoughts and connect them over time, capturing the whole life cycle of information. Actually, this is the way the web was originally thought to be.

All of these are just small glimpses into an almost unexplored area and I could show you many more. As I said before however, these are not technological problems but a lack of perspective.

We shape our tools and our tools shape us.

I actually lied to you. At least a bit. McLuhan’s full quote goes like this:

We shape our tools and thereafter our tools shape us.

You see, our technological capability changes much faster than our culture. We first create our technologies and then they change our society and culture. The lesson is that, even today, we are designing for tomorrow’s technology.

It would be a shame if going into the future we’re still using outdated metaphors, pencil-and-paper thinking, code in text files or collaborating manually. When I look at people using digital devices, I see them adjusting to the limitations of the computer. They are sharing instead of collaborating, consuming instead of producing, following instructions instead of experimenting, searching instead of thinking. Have we learned anything from this fertile recent period, and from these great ideas and visions?

If we don’t implement what we learn, it suggests we don’t care about how it will change our society for the better — or for the worse. It will show we don’t care what the medium computer will look like. That we don’t care how to make use of computers to augment our most human capabilities.

Our history shows if you free yourself from the idea that the current state of the computer world is the climax of our great progress we can almost magically give rise to new technologies, ideas and visions that do not only amplify humans, but also produce tremendous wealth for our society.

We’ve ended up focusing too much on technology, on things, on devices and not enough on ideas, on the medium computer. What if a book would not just simply give you facts, but gave you the tools to discover ideas yourself - or help you to invent new ones? What if, while reading an article, you could annotate it, challenge the assumptions and share your insights with the world? What if you could collaborate in the same intellectual space with colleagues? What if your tools would not only allow you to consume content, but be imaginative creators, to interact with it, to construct knowledge? What if you could collaborate with a computer in a symbiosis between man and machine?

The computer world is not yet finished and many ideas for the collective good are waiting to be discovered.

We shape our tools and thereafter our tools shape us.

Thank you.

acknowledgements

Thanks to Daniel Bader, Jan Peuker, Marian Edmunds and Philipp Pamer for many hours of great conversation about these issues and for feedback while these thoughts were solidifying. This talk was heavily inspired by talks from Wilson Miner, Maciej Cegłowski, Bret Victor and Alan Kay. I am very thankful for allowing me to base my work on their ideas and visions.

recommended reading

Below you can find a summary of works cited and used to build this talk based on many hours of reading and research. You can be sure that each one is fantastic and will be worth your time.