Ted Nelson – The Curse of Xanadu
It was the most radical computer dream of the hacker era. Ted Nelson’s Xanadu project was supposed to be the universal, democratic hypertext library that would help human life evolve into an entirely new form. Instead, it sucked Nelson and his intrepid band of true believers into what became the longest-running vaporware project in the history of computing.
I said a brief prayer as Ted Nelson – hypertext guru and design genius – took a scary left turn through the impolite traffic on Marin Boulevard in Sausalito. Nelson’s left hand was on the wheel, his right rested casually on the back of the front seat. He arched his neck and looked in my direction so as to be clearly heard. “I’ve been compiling a catalog of driving maneuvers,” he said. “It’s one of my unfinished projects.”
Nelson is a pale, angular, and energetic man who wears clothes with lots of pockets. In these pockets he carries an extraordinary number of items. What cannot fit in his pockets is attached to his belt. It is not unusual for him to arrive at a meeting with an audio recorder and cassettes, video camera and tapes, red pens, black pens, silver pens, a bulging wallet, a spiral notebook in a leather case, an enormous key ring on a long, retractable chain, an Olfa knife, sticky notes, assorted packages of old receipts, a set of disposable chopsticks, some soy sauce, a Pemmican Bar, and a set of white, specially cut file folders he calls “fangles” that begin their lives as 8 1/2-by-11-inch envelopes, are amputated en masse by a hired printer, and end up as integral components in Nelson’s unique filing system. This system is an amusement to his acquaintances until they lend him something, at which point it becomes an irritation. “If you ask Ted for a book you’ve given him,” says Roger Gregory, Nelson’s longtime collaborator and traditional victim, “he’ll say, ‘I filed it, so I’ll buy you a new one.’” For a while, Nelson wore a purple belt constructed out of two dog collars, which pleased him immensely, because he enjoys finding innovative uses for things.
Nelson’s life is so full of unfinished projects that it might fairly be said to be built from them, much as lace is built from holes or Philip Johnson’s glass house from windows. He has written an unfinished autobiography and produced an unfinished film. His houseboat in the San Francisco Bay is full of incomplete notes and unsigned letters. He founded a video-editing business, but has not yet seen it through to profitability. He has been at work on an overarching philosophy of everything called General Schematics, but the text remains in thousands of pieces, scattered on sheets of paper, file cards, and sticky notes.
All the children of Nelson’s imagination do not have equal stature. Each is derived from the one, great, unfinished project for which he has finally achieved the fame he has pursued since his boyhood. During one of our many conversations, Nelson explained that he never succeeded as a filmmaker or businessman because “the first step to anything I ever wanted to do was Xanadu.”
Xanadu, a global hypertext publishing system, is the longest-running vaporware story in the history of the computer industry. It has been in development for more than 30 years. This long gestation period may not put it in the same category as the Great Wall of China, which was under construction for most of the 16th century and still failed to foil invaders, but, given the relative youth of commercial computing, Xanadu has set a record of futility that will be difficult for other companies to surpass. The fact that Nelson has had only since about 1960 to build his reputation as the king of unsuccessful software development makes Xanadu interesting for another reason: the project’s failure (or, viewed more optimistically, its long-delayed success) coincides almost exactly with the birth of hacker culture. Xanadu’s manic and highly publicized swerves from triumph to bankruptcy show a side of hackerdom that is as important, perhaps, as tales of billion-dollar companies born in garages.
Among people who consider themselves insiders, Nelson’s Xanadu is sometimes treated as a joke, but this is superficial. Nelson’s writing and presentations inspired some of the most visionary computer programmers, managers, and executives – including Autodesk Inc. founder John Walker – to pour millions of dollars and years of effort into the project. Xanadu was meant to be a universal library, a worldwide hypertext publishing tool, a system to resolve copyright disputes, and a meritocratic forum for discussion and debate. By putting all information within reach of all people, Xanadu was meant to eliminate scientific ignorance and cure political misunderstandings. And, on the very hackerish assumption that global catastrophes are caused by ignorance, stupidity, and communication failures, Xanadu was supposed to save the world.
At the end of our brief but hair-raising journey toward lunch, Nelson’s battered 1970 Ford LTD came to a halt in front of The Spinnaker, a staid restaurant on the Sausalito pier. As we sat down at a table overlooking the bay, Nelson remarked that he could row his kayak from his houseboat to The Spinnaker, and the reference to water reminded him of his incomplete autobiography. “There’s a very nice opening page there,” he said, “where I talk about being in a rowboat with my grandfather rowing – and my grandmother – and running my hands, letting my hands trail in the water, when I was 4 or 5.”
As with everything else in his life, Nelson’s conversation is controlled by his aversion to finishing. There are no full stops in the flow of his speech, only commas, dashes, ellipses.
“And I remember thinking about the particles in the water, but I thought of them as places, and how they would separate around my fingers and reconnect on the other side, and how this constant separation and reconnection and perpetual change into new arrangements was – ”
Suddenly, the monologue stopped, and Nelson reached into his cache of equipment. He retrieved his own cassette recorder, tested it, and turned the microphone toward himself. “OK, I’m at The Spinnaker,” he continued, “talking about the old hand-in-the-water story and how the sense of the separation and reconnection of the places in the water made such an impression on me, and how all the relationships were constantly changing – and you could hardly hold onto it – you could, you could not, you couldn’t really visualize or express the myriad of relationships.”
The chaotic, ephemeral eddies under his grandfather’s rowboat are a perfect image of Nelson’s style of thought. I was already taping our conversation, but Nelson clearly wanted his own record. Not because he was concerned about being quoted correctly, but because his tape recorder and video camera are weapons in an unending battle against amnesia. The inventor suffers from an extreme case of Attention Deficit Disorder, a recently named psychological syndrome whose symptoms include unusual sensitivity to interruption.
If he is stopped in the middle of anything, he forgets it instantly. Only by running his own tape recorder could Nelson be confident that his words would not float off, irrecoverably, into the atmosphere.
Nelson’s anxiety about forgetting is complicated by the drugs he takes. For his ADD, Nelson takes Cylert; for his agitation, he takes Prozac; for sleeplessness, he takes Halcion. Halcion can produce aphasia: during our lunch, Nelson sometimes found himself groping for a common word in the middle of a sentence. But for the most part, he was fluent, and he took pleasure in the aptness of his own phrases. Although inconvenienced by his disorder, Nelson is nonetheless proud of it. “Attention Deficit Disorder was coined by regularity chauvinists,” he remarked. “Regularity chauvinists are people who insist that you have got to do the same thing every time, every day, which drives some of us nuts. Attention Deficit Disorder – we need a more positive term for that. Hummingbird mind, I should think.”
Xanadu, the ultimate hypertext information system, began as Ted Nelson’s quest for personal liberation. The inventor’s hummingbird mind and his inability to keep track of anything left him relatively helpless. He wanted to be a writer and a filmmaker, but he needed a way to avoid getting lost in the frantic multiplication of associations his brain produced. His great inspiration was to imagine a computer program that could keep track of all the divergent paths of his thinking and writing. To this concept of branching, nonlinear writing, Nelson gave the name hypertext.
Although the concept of hypertext made Nelson legendary in programming circles, he is not a programmer. “I have a terrific math problem,” Nelson said. “I still can’t add up a checkbook: I can add a column of figures five times, get four different answers, and none of them will be right. I’m very accident-prone and extremely impatient. I can’t work my Macintosh – I have three that are completely unfunctional and one is marginally functional.”
“I never got past the calculus,” he added, pausing to pull out a video camera, which he focused on the notebook next to his plate.
“Why are you filming your notebook,” I asked.
“I was just trying to get this thing going,” he answered. Satisfied that the camera was running, Nelson briefly panned it around the room. Then he set it down and returned to his lecture. His lunch, a large plate of pasta and seafood, had long ago been delivered, tasted, and forgotten.
Nelson has never catalogued his thousands of hours of audio- and videotape. This would be impossible, since they are coextensive with his waking life, and it would also be unnecessary, since he has no intention of viewing or studying them. He rents several storage spaces around the San Francisco Bay area, all packed with materials he is leaving for future generations to decipher, and he prays that when scholars get around to examining his vast and vastly disorganized oeuvre, they will have the necessary digital technology to analyze and follow it. This technology, he insists, is Xanadu.
Were Xanadu merely the private obsession of a talented iconoclast, the piles of papers and deteriorating magnetic reels in Nelson’s many overflowing lockers could simply be carted off to a dump. But the inventor is probably right in his prediction that Xanadu’s strange story will prove to be an important chapter in the history of technology. Out of Nelson’s discombobulation was born one of the most powerful designs of the 20th century. And Xanadu’s goals – a universal library, a global information index, and a computerized royalty system – were shared by many of the smartest programmers of the first hacker generation.
The story of Ted Nelson’s Xanadu is the story of the dawn of the information age. Like the mental patient in Thomas Pynchon’s Gravity’s Rainbow who believes he is the Second World War – feeling a great burst of rosy health when The Blitz comes and a terrible pinching headache at the Battle of the Bulge – Nelson, with his unfocused energy, his tiny attention span, his omnivorous fascination with trivia, and his commitment to recording incidents whose meaning he will never analyze, is the human embodiment of the information explosion.
Nelson records everything and remembers nothing. Xanadu was to have been his cure. To assist in the procedure, he called upon a team of professionals, some of whom also happened to be his closest friends and disciples.
In the end, the patient survived the operation. But it nearly killed the doctors.
In conversation, Nelson is by turns reproachful and gloating. A dreamy, unathletic child raised by elderly grandparents in Greenwich Village, Nelson devoted his youth to studying the art of strategy, and learned to pick up a serious weapon, such as a rock or pole, when threatened by neighborhood bullies. As a grad student at Harvard, Nelson would one day study strategy with Thomas Schelling, a renowned theorist, but as a child, his methods were instinctive. For instance, in second grade, Nelson invented a new way of crossing the street: when arriving at a busy thoroughfare, he would dramatically turn his back on traffic and step with theatrical nonchalance off the sidewalk. Drivers, frightened, would slam on their brakes.
Nelson’s heroes were famous non-conformists and businessmen, including Buckminster Fuller, Bertrand Russell, Walt Disney, H. L. Mencken, and Orson Welles. The inventor was, by his own account, a brilliant child whose speech was unusually grammatical and whose wise pronouncements would cause adults to fall silent. Nelson’s father – who kept in intermittent contact with his son – worked as a movie director (he made Requiem for a Heavyweight and Soldier Blue among other films), and inspired the younger Nelson to begin his own (unfinished) motion picture epic, The Epiphany of Slocum Furlow. Of his actress mother, Nelson says only that they do not communicate and have not spoken in a long time.
Nelson’s hatred of conventional structure made him difficult to educate. Bored and disgusted by school, he once plotted to stab his seventh-grade teacher with a sharpened screwdriver, but lost his nerve at the last minute and instead walked out of the classroom, never to return. On his long walk home, he came up with the four maxims that have guided his life: most people are fools, most authority is malignant, God does not exist, and everything is wrong. Nelson loves these maxims and repeats them often. They lead him to sympathize, in every discussion, with the rejected idea and the discounted option.
By the time Nelson reached college, his method of combating the regularity chauvinists was quite sophisticated; he put his teachers off with the theories of writer Alfred Korzybski, who denounced all categories as misleading. But this hatred of categories did not produce in Nelson a fuzzy, be-here-now mysticism. On the contrary, Nelson loved words, which were tools for memory, but he hated the way that traditional writing and editing imposed a false and limiting order. Nelson had no interest in the smooth, progressive narratives encased in books. He wanted everything to be preserved in all its chaotic flux, so that it could be reconstructed as needed.
Nelson, a lonely child raised in an unconventional family, became a rebel against forgetting, and a denier of all forms of loss and grief. (Some of Nelson’s disciples would one day take this war against loss even further, and devote themselves to the development of cryonic technology for the freezing and preservation of corpses.) Tormented by his own faulty memory, Nelson developed the habit of asserting that only a technology for the preservation of all knowledge could prevent the destruction of life on Earth. The thought that some mental connection or relationship might dissolve was unbearable. Not only was the constant churn and dispersal of his own thoughts personally devastating, but the general human failure to remember was, Nelson thought, suicidal on a global scale, for it condemned humanity to an irrational repetition of its mistakes.
Nelson earned a BA in philosophy from Swarthmore, and in 1960 became a graduate student at Harvard. Hypertext was invented during his first year at Harvard, when Nelson attempted, as a term project, to create a “writing system” that allowed users to store their work, change it, and print it out. In contrast to the first experimental word processors, Nelson’s design included features for comparing alternate versions of text side by side, backtracking through sequential versions, and revision by outline. Establishing a habit that would persist, Nelson failed to finish the coding, and had to take an incomplete for the course.
Although Nelson had originally intended to get a PhD in social relations, his attempt to finish his term project soon took precedence over other course work. Simultaneously, a cliqué of Harvard researchers was trying to create programs that would replace routine teaching chores with computerized processes.
Nelson considered this linear, mechanical approach to what was then called computer-aided instruction an insult to both students and computers, and he urged the adoption of a system that would let students explore academic material along a variety of alternate paths. He called for a system based on “nonsequential writing.”
The word hypertext was coined by Nelson and published in a paper delivered to a national conference of the Association for Computing Machinery in 1965. Adding to his design for a nonsequential writing tool, Nelson proposed a feature called “zippered lists,” in which elements in one text would be linked to related or identical elements in other texts. Nelson’s two interests, screen editing and nonsequential writing, were merging. With zippered lists, links could be made between large sections, small sections, whole pages, or single paragraphs. The writer and reader could manufacture a unique document by following a set of links between discrete documents that were “zipped” together.
Many precedents for the idea of hypertext existed in literature and science. The Talmud, for instance, is a sort of hypertext, with blocks of commentary arranged in concentric rectangles around the page. So are scholarly footnotes, with their numbered links between the main body of the text and supplementary scholarship.
In July 1945, long before Nelson turned his attention to electronic information systems, Vannevar Bush published an essay titled “As We May Think” in The Atlantic Monthly, which described a hypothetical system of information storage and retrieval called “memex.” Memex would allow readers to create personal indexes to documents, and to link passages from different documents together with special markers. While Bush’s description was purely speculative, he gave a brilliant and influential preview of some of the features Nelson would attempt to realize in Xanadu.
The inventor’s original hypertext design predicted most of the essential components of today’s hypertext systems. Nonetheless, his talk to the Association for Computing Machinery had little impact. There was a brief burst of interest in this strange researcher, but although his ideas were intriguing, Nelson lacked the technical knowledge to prove that it was possible to build the system he envisioned.
The new hypertext prophet had difficulty finding a place to preach. During the next four years, Nelson bounced around among a number of companies and research programs. Hired by the publishing giant Harcourt, Brace to advise them about computer-based business opportunities, he puzzled the executives with his radical pronouncements about the imminent overthrow of everything. Meanwhile, he put off computer scientists by taking every opportunity to inform them that they failed to understand the earth-shattering significance of their work. Despite these misfirings, the inventor’s private exploration of hypertext continued. He moved quickly into the most complex theoretical territory, asking questions that still challenge hypertext designers today. For instance, if you change a document, what happens to all the links that go in and out? Can you edit a document but preserve its links? What happens when you follow a link to a paragraph that has been erased?
Computers in the ’60s were giant machines, accessible to amateurs mainly in university computer centers, where students could divert themselves from their science homework with primitive question-and-answer games. But the trend toward smaller and faster digital tools was already apparent to insiders, some of whom wondered how computers could handle basic, personal information tasks, such as editing a term paper. In 1969, Nelson was hanging around Brown University, where an early word-processing tool was under development. The Brown project focused on a system that would output paper, but Nelson believed that paper was hopelessly retrograde and that the native territory of hypertext was on the screen, not the page. Later that year, Nelson got permission from the publishers of Vladimir Nabokov’s Pale Fire to use the elaborately annotated parody in a hypertext demonstration. The idea, like most of Nelson’s contributions, was rejected by the sponsors of the Brown experiment. Nelson was bitter over the obstruction of his work. “Thus progress must wait,” he later wrote, “for the halt and lame to catch up.”
Nelson’s characteristic anger both sustained and sabotaged him during these hard, post-college years. His most productive stretch of short-term employment may have come in 1967, while he was working for Harcourt, Brace. Although he failed to make any technical advances, he did coin a powerful trademark. Impressed by the literary employees of the publishing house, and wanting to impress them in return, he christened his hypertext system Xanadu.
It was a name of uncanny exactitude. Xanadu is the elaborate palace in Kubla Khan.”
In his famous tale of the poem’s origin, Coleridge claimed to have woken from a narcotic reverie with hundreds of lines of poetry in his head. As he was about to transcribe them, he was interrupted by a visitor, and when he returned to his writing table, the vivid oneiric composition had evaporated. In his preface to the fragment that remained, Coleridge lamented:
Then all the charm
Is broken – all that phantom-world so fair
Vanishes, and a thousand circlets spread,
And each mis-shape[s] the other….
The Coleridge fragment haunts Nelson’s grand hypertext design, just as it haunted and inspired Orson Welles. In the name Xanadu was a prescient flash of the years of remarkable heartbreak that lay ahead.
Had Nelson been able to delve into the technical reasons for which computer people found his plans for Xanadu unconvincing, he might have been too discouraged to continue. The kinds of programs he was talking about required enormous memory and processing power. Even today, the technology to implement a worldwide Xanadu network does not exist. Back in the ’70s, when Nelson was still waging the first phase of his campaign, even simple word-processing programs required users to share time on large mainframe computers. The notion of a worldwide network of billions of quickly accessible and interlinked documents was absurd, and only Nelson’s ignorance of advanced software permitted him to pursue this fantasy. The inventor was like a vaudeville performer practicing an acrobatic routine on the edge of an unseen cliff. A look into the abyss would doubtless have sent him tumbling.
Other computer people were not as boastfully buoyant. To find help, Nelson was forced to go outside official channels. The first disciples he acquired belonged to a group of hackers known as the R.E.S.I.S.T.O.R.S., which stood for Radically Emphatic Students Interested in Science, Technology, and Other Research Studies. Unlike the mainstream programmers Nelson encountered, the Resistors shared Nelson’s sense of humor, his mischief, and his lack of respect for authority. An added benefit was that they didn’t need salaries, since most of them still lived with their parents. The Resistors were members of a computer club in Princeton, New Jersey, and their average age was about 15. Nelson’s influence over some of them was lifelong. Almost 20 years later, one of the Resistors, Lauren Sarno, who was 14 when she met Nelson, would become his personal assistant. In 1987, Sarno would spend thousands of hours reconstructing Nelson’s masterpiece, Computer Lib, so it could be reprinted by Microsoft Press.
The Resistors appreciated Nelson because he took their advice seriously. “Some people are too proud to ask children for information,” Nelson preached in Computer Lib. “This is dumb. Information is where you find it.” The teenage Resistors spent quite a bit of time driving around with Nelson in his car, telling hackerish jokes and scheming to transform civilization. Their favorite activity was wordplay. One of Nelson’s Resistor anecdotes describes an afternoon when he was cruising through Princeton with his co-conspirators and growing increasingly annoyed with the loud contradictory instructions coming from the back seat. “I demand triple redundancy in the directions,” Nelson said.
“Right up ahead you turn right right away,” piped up one of the teenagers, promptly.
A picture from that era shows a grinning, boyish Nelson in a white shirt and tie, with hair down to his collar, sitting at the wheel of a car full of kids. He looks enormously pleased.
While continuing to work with the high schoolers, Nelson acquired some money from an individual investor and used it to recruit Cal Daniels, a programmer at a company called Minicomputer Systems Inc., as well as a young Swarthmore student who knew Fortran. Nelson, who regularly commuted between his apartment in Manhattan, the Swarthmore campus, and Daniels’s large house in Queens, recalls this era as one of “talking the system, hashing details.” By all accounts, it was mostly talking. But during a rare period of fierce programming, the three collaborators created an interesting data structure that governed the movement of huge sections of text in and out of the computer’s memory. They called their invention “the enfilade.”
The dictionary defines enfilade, which can be a noun or a verb, as the firing of a gun in a sweeping motion along the length of a target. The word has etymological links to threads and files, as well as to an arrangement of rooms with doorways in line with each other, and to a vista seen between columns or trees.
Unfortunately, beyond the dictionary, no further clues about the nature of the enfilade are available: the discovery is one of Xanadu’s closely guarded trade secrets, and all the people who have worked on it are prohibited from revealing its inner nature. This reticence has naturally produced doubt about the enfilade’s world-historical significance. When asked skeptically why he won’t allow anything about the invention to be published, Nelson responded with quick anger. “Because it is still hot shit,” he says.
The discovery of the enfilade and the pledge made by its discoverers to keep it secret marked a turning point in the personality of Xanadu. The first real work had been achieved, and the first concession to secrecy had been made. Xanadu was now more than a grand vision and more than a set of original ideas – it was now a proprietary software package, with its design concepts tied up in a product and its intellectual influence bound tightly to the vicissitudes of the market.
In 1972, Cal Daniels completed the first demonstration version of the software. Daniels wrote some primitive Xanadu code in a now-defunct programming language that ran on Nelson’s rented Nova computer. However, before he could show a running Xanadu system to any potential backers, Nelson unexpectedly ran out of cash and was forced to return the Nova. The programmers had working code but no machine. (Later, they would have machines but no working code.) Like Nelson’s failure to complete his college hypertext project in the mid-1960s, this bankruptcy is a Xanadu milestone, for it established the coincidence of near-success and sudden penury as one of Xanadu’s ineluctable motifs.
After this defeat, Nelson moved further and further toward the fringe of the computer industry. In 1973, he landed a job at the University of Illinois in Chicago, where he instantly and typically discovered that he couldn’t get along with his colleagues. With the prospect of winning respect from computer officialdom fading, Nelson moved in a new direction. He took some time off from attempting to build his system. He also took some time off from incessantly talking about it. He began to reach beyond the range of his voice by appealing to a more general public.
His switch was well timed. When the inventor first got a hint of the territories beyond the mainframe horizon, his plan had been to arrive first and install a decent information network. But as the promoters of American towns long ago discovered, it is not necessary to build anything to profit from the settlement of a new territory. You just have to survey the land and begin selling parcels to wishful pioneers. Nelson had failed to construct his information infrastructure, but he had designed a very nice picture of the future.
Isolated at the University of Illinois, Nelson began to write an enthusiastic tome that was part gospel, part political pamphlet, and part real-estate brochure extolling the benefits of life on the digital frontier. When he started, Nelson expected to produce about 40 pages of typewritten text, on regular, 8½, 2-by-11-inch paper. By the summer of 1974, after 18 months of manic labor, culminating in weeks of round-the-clock cutting and pasting, Nelson had a rambling, jumbled 1,200-page manuscript on his hands.
Any nitwit can understand computers, and many do,” announced Nelson in the 1974 introduction to the first edition of Computer Lib. His opus was really two books, attached to each other upside down and backward, like the old Ace Doubles or, as Nelson enjoys pointing out, like The Italian/Polish Joke Book. One cover showed a revolutionary fist inside a computer. When readers flipped the book over, they saw the cover of Dream Machines, decorated with an airborne man in a Superman cape reaching out with his finger to touch a screen. The book was large – 11 inches wide and 16 inches tall – and contained a 300,000-word manifesto of the digital revolution. The print was tiny, and the layout confusing. Nelson wrote out his rough draft, which consisted of hundreds of individual rants, on a typewriter; then cut and pasted the rants together onto sheets of cardboard; took the sheets to a printer; and returned some weeks later to pick up cartons of books. When he discovered that about a third of the books had their pages in the wrong order, he had the printer unbind the flawed copies and recollate and rebind them. Between 1974 and 1987, when Computer Lib was republished by Microsoft Press, Nelson sold at least 100 copies of his manifesto every month, sometimes many more.
An expression of the author’s encyclopedic passion, Computer Lib contains whatever enraged or inspired Nelson during the months he wrote it, including population statistics, hacker psychology, the evils of IBM, holograms, musical notation, lists of places to rent a PDP-8, Watergate, and how to program in Trac, among other topics. These remarks “didn’t fit anywhere else, so they might as well go here,” is a typical Computer Lib transition. The model for the book was Stewart Brand’s 1969 counterculture classic, The Whole Earth Catalog, but the design of Computer Lib was even more idiosyncratic. There was no index or table of contents. Specific quotes or sections were impossible to find. Although full of reference material, it could not be used as a reference without being read enough times to memorize it. Which, of course, is exactly what many young hackers did.
Dream Machines, the literal flip side of Computer Lib, was largely about the transformation of the arts through computers, but it included a relatively brief description of Xanadu. In the years since 1965, when he first attempted to make Xanadu work, the idea had grown enormously. By 1974, locally networked computers had appeared, and Nelson saw a global computer network as the natural environment for a hypertext system. Over a network, linked documents, version comparison, and non-sequential writing would create a “docuverse” capable of storing and representing the artistic and scientific legacy of humanity.
In Dream Machines, Nelson pitched the idea of Xanadu information franchises, where data shoppers could access material from the global storage system. This pitch for what he called the Xanadu Stands included a sketch of the interior, complete with snack bar, and the lyrics for a Xanadu singing commercial:
The greatest things you’ve ever seen
Dance your wishes on the screen
All the things that man has known
Comin’ on the telephone -
Poems, books and pictures too
Comin’ on the Xanadu.
The Xanadu franchises were silly, but they contained a solution to a genuinely difficult problem. If there was to be a universal library of electronic documents, who would pay for it? Nelson’s answer was to imagine a corporate information entity that resembled McDonald’s, a chain of franchises whose operating costs were paid by their individual owners out of revenue from the information-starved masses.
“The franchisee,” wrote Nelson cheerfully, “has to put up the money for the computers, the scopes, the adorable purple enamel building, the johns, and so on; as a Xanadu franchisee he gets the whole turnkey system and certain responsibilities in the overall Xanadu network – of which he is a member.” Nelson hinted that this system, the software on which the structure of the fantasy was to rest, was nearly complete.
Computer Lib was written as a popular primer, but its most profound effect was on computer programmers, who needed little persuasion about the value of computers. Its tone – energetic, optimistic, inexhaustible, confused – matched theirs exactly. Having set out to appeal to the general public, Nelson managed to publish an insider’s bible and highly intimate guide to hacker culture.
What most touched the hackers who read Computer Lib was not the instructions on how to write a program loop in APL but something more radical. Computer Lib assigned to programmers a noble role in the battle for humanity’s future, and it recruited them for the rebellion they were witnessing on their college campuses. When programmers read Computer Lib, they could discern a portrait of the book’s ideal reader – an anxious, skeptical, interested, sensible, free-thinking citizen who wanted better digital tools. At the time of Computer Lib, this popular audience for news about the digital revolution did not exist. But the people to whom Computer Lib became a bible wished that this audience existed. Computer Lib reflected back to computer programmers an idealized image of themselves. In this sense, it was a far subtler book than Nelson set out to write.
Roger Gregory, Ted Nelson’s most loyal collaborator, is a sad man. He suffers from a common, disabling ailment that Abraham Lincoln, who was similarly afflicted, called “the hypos.” His sadness grows so acute at times that he becomes incapable of working, and his fits of black sorrow go back many, many years.
When he first heard of Ted Nelson, Gregory was a science fiction fan working in a used-computer store in Ann Arbor, Michigan, called Neuman Computer Exchange. He had stringy hair and wore dirty clothes, and he tended to argue violently with people he felt were wrong. His job was annoying because he never got to play with any functioning “toys” – the refrigerator-size cabinets that contained the latest digital machinery. As soon as Gregory got some hopelessly wrecked piece of computer equipment into decent condition, Al Neuman, his boss, would sell it off. Gregory did his hacking in various computer labs associated with the University of Michigan, and he belonged to a social group – the Ann Arbor Computing Club – whose membership overlapped with the local science fiction club.
Gregory was introduced to Nelson’s work by Michael McClary, a programmer friend as well as a reader of Computer Lib. The year was 1974, and in the rest of America, the counterculture was in its final paroxysm of false triumph. The revolution had come late to computer scientists, but it did come, causing many to shed their white shirts and pocket protectors and question the servility of their discipline, which was largely devoted to making money and waging war. McClary, who like Gregory was a fan of Robert Heinlein, offered Gregory a ride to an annual science fiction convention in Washington, DC. As they puttered over the Appalachias in an ancient Ford Galaxy loaded with three other science fiction bums, McClary preached the Xanadu gospel. Nelson’s idea, McClary explained, was that the computer was the resource to get into the hands of the people. With a global publishing system, the need for printing presses could be eliminated. Censorship would be difficult, if not impossible. Plus, such a system would be extremely fun to build. McClary gave Gregory a brochure that advertised Nelson’s new book.
The inventor’s meme, delivered secondhand, couldn’t have found a more perfect host. Gregory had exactly the skills Nelson lacked: an intimate knowledge of hardware, a good amount of programming talent, and an obsessive interest in making machines work.
Gregory intended to call Nelson, but destiny moved more quickly: the repairman had hardly returned to Ann Arbor when Nelson telephoned the Neuman Computer Exchange and asked the person who answered the phone to trade a thousand copies of Computer Lib for a used PDP-11.
The PDP-11, from the Digital Equipment Corporation, was a coveted machine. It was the original computer to run a new programming language called C, which was on its way to becoming the hackers’ standard. Gregory, as it happened, didn’t have any spare PDP-11s at his disposal. But the repairman took the opportunity to question some of Nelson’s blithe predictions in Computer Lib, and Nelson, in response, unleashed his glib and bitter tirade against the conservative ignoramuses in the computer business.
Gregory, whose quick rage once led him to grab a shotgun from two would-be burglars and chase them from his door, had finally found an interlocutor who wasn’t afraid to match him outrage for outrage. Gregory’s dismissive contempt can be piercing, but Nelson’s speculative mania is indeflatable. Over the next few years, Gregory spent hundreds of dollars on long-distance telephone discussions of the hypertext project.
Did Nelson realize at the time that he had met Xanadu’s second parent? Probably not. The inventor scattered his ideas as widely as possible, with little care about where they landed. But as the decades passed, it would be Gregory who oversaw the attempt to transform Xanadu into a real product. He never received much public notice, but through all the project’s painful deaths and rebirths, Gregory’s commitment to Nelson’s dream of a universal hypertext library never waned. If Ted Nelson is Xanadu’s profligate father, Roger Gregory is Xanadu’s devoted mother, and in retrospect, his role appears to have been intertwined with a terrible element of sacrifice.
Soon after Computer Lib was released, Nelson fled the unfriendly confines of the University of Illinois and found refuge at Swarthmore, on the same quiet, undergraduate campus where, in the ’60s, he first incubated his hypertext designs. Swarthmore had offered him a nondepartmental position teaching his own work; Nelson taught classes in the social issues pertaining to technology and design.
While he was at Swarthmore, another important disciple appeared.
In 1976, Mark Miller, an insecure 19-year-old, came to address a classroom full of Ted Nelson’s students. He was nervous. Miller had read Computer Lib the year before, when he was a freshman at Yale, and the book had filled his head with reveries about the digital future. Intensely interested in computers, Miller hoped to make some small contribution to a society that would be run on rational, libertarian, and scientific principles. Nelson’s work, which described a global community united by perfect information, seemed the most important signpost yet on the road to this utopia.
As a guest lecturer in Nelson’s class, Miller ran through his ideas for a Xanadu-like software system. Afterward, he was approached by one of the students, Stuart Greene. Miller asked Greene what the reaction to his ideas had been. Not so good, Greene informed him. As always, the class had listened in dumb incomprehension. They seldom understood what Nelson was talking about, and when Miller launched into a similar enthusiastic tirade, their response, Greene laughed, was “Oh, no, we can’t believe there’s another one!”
The Yale student was not discouraged. A disheveled mathematician – part goofball, part Wunderkind – Miller loved pointing out that his name was a pun on computer programming: after all, software code consists of marks, and milling means churning or grinding. A photograph shows Miller with a smirk on his face, a shirt pocket full of pens, and a pair of Mickey Mouse ears on his head. Nelson’s rejection of human forgetfullness seemed right to Miller, who hoped to create a system that allowed his consciousness to be captured in a computer program, and thus achieve immortality. Miller’s middle name was Samuel, which he spelled $amuel, in tribute to his faith in the power of the marketplace to fill all human needs.
Nelson’s book brought him growing acclaim, and in 1979, he decided it was time to gather his disciples. He called upon Roger Gregory to lead the effort. Although Gregory was in Ann Arbor, Nelson insisted that everybody move to Swarthmore so he could exercise his influence at close range. Obediently, Gregory rented a house and invited the other programmers to join him. Mark Miller returned to Pennsylvania, where the Xanadu devotees aimed to finish the project in a single, serious summer of coding.
That summer was Xanadu’s golden age. During long afternoons and evenings, the programmers sat on the porch and scrawled on blackboards, musing over the difficulties of making hypertext code that really worked. Although they had planned to program the system during the three months Miller had off from Yale, they spent most of their time talking about data structures and reworking the design. The chief difficulty was creating a way to move data quickly in and out of the computer’s memory. Since hypertext links could connect infinitely many documents, every bit of writing in the system had to be instantly accessible. Nelson became convinced that they were making major contributions to computer science. He believed that the newest versions of the data-search algorithms, dubbed “General Enfilade Theory,” allowed the Xanadu system to grow forever without its performance degrading unacceptably.
Most computer scientists would have been suspicious of these claims, but this hardly bothered these programmers, who were working in an atmosphere of friendly competition and camaraderie. They may not have always agreed with Nelson’s aggressively optimistic predictions, but they shared a feeling that Gregory’s large, messy home in Swarthmore was nurturing a social and scientific revolution.
The question of computer performance was key. That summer, Gregory was programming on a Sol 20, borrowed from a company called Processor Technologies. Soon, he gave up trying to hammer the Sol into something more usable and decided to buy a new Onyx, with a whopping 10-Mbyte disk. The Onyx also had 128 Kbytes of RAM, which they later doubled to a screaming 256 Kbytes. Looking back at the specifics of the endeavor, the approach of the Xanadu programmers seems quixotic. Gregory and his colleagues were trying to build a universal library on machines that could barely manage to edit and search a book’s worth of text.
“The summer went a little slower than we thought,” remembers Gregory. Greene, Miller, and Gregory made some headway in the design, and in August, they wrote some code. But the real world began to press in on them, and as summer waned, the crew went off in separate directions. There were tasks to accomplish – educations to finish and careers to start – and the Xanadu reverie could not be maintained.
Except, perhaps, by Gregory. In comparison with the Technicolor geography of Xanadu, the landscape of Gregory’s regular life was as flat as Kansas farmland. While he knew how to fix and program computers pretty well, he was not a computer scientist or an élite researcher, and his persistent sadness compelled him to seek a destiny greater than tweaking corporate and commercial machines. In managing his depression, Gregory found that it helped to have something productive to do; the computer was always there, and when he felt his sorrow well up, he knew he could sit in his chair, stare at the screen, and begin to hack. By the summer of 1979, Gregory had already wound himself up too deeply in the threads of Xanadu’s alternate universe to cut himself free. Gregory knew that if he was to escape, his route would be through Xanadu, not away from it.
Come September, Gregory stayed in Pennsylvania and rented another house. As programmers came and went, the house provided a frame for Xanadu’s slow progress. Working full time on outside consulting contracts to support Xanadu and some 40 hours a week on the project itself, Gregory opened his home to anybody he felt could contribute. Mark Miller had returned to Yale to begin his senior year, but he stayed in touch and continued to offer suggestions. Eric Hill, who had been around during the Swarthmore summer, and Roland King both joined the household. Eric Drexler, a graduate student obsessed with solar sails for space travel and microscopic machines, was a frequent guest and friendly critic.
Having maxed out the Onyx, Gregory went searching for a new computer, and in 1982 became the first individual not funded by a government or educational institution to purchase a Sun. It was extremely expensive – US$26,000. The serial number was 82. With the Sun and a new, 80-Mbyte hard drive, which cost $10,000, the Xanadu code had its first decent home.
At the beginning of Xanadu’s second decade of development, Nelson was pleased with the project’s caretakers. The last time he had come this close to having a working prototype was in 1972, when time ran out on his rented Nova. Now, the inventor’s brainchild was more mature. Miller and Gregory created an addressing system that used transfinite numbers, an arcane area of calculus they had both studied in college. They called the new addresses “tumblers”; the tumbler system allowed readers to create links to any arbitrary span of bytes, whether or not the author had marked them. With tumblers, Miller and Gregory could give a similar address to every document and fragment of a document in Xanadu’s sprawling domain of words, pictures, movies, and sounds. The address would not only point the reader to the correct machine, it would also indicate the author of the document, the version of the document, the correct span of bytes, and the links associated with these bytes.
Unfortunately, though the design was innovative and the algorithms interesting, the Xanadu code was depressingly nonfunctional. As 1979 stretched into 1980, and 1980 into 1981, Nelson continued to lecture about the imminent release of the greatest information software of all time. Xanadu, he promised, would make the central concepts of computing – files, for example – obsolete. In Xanadu, there would be no immutable files, only a mass of material that could be organized according to the reader’s preference.
Miller, the prodigy, failed to return to Gregory’s house after graduating from Yale in 1980. Instead, he moved to Datapoint, a hardware company based in San Antonio, Texas, which was then the leader in networking technology. With Stuart Greene already a Datapoint employee, Miller went on to work at the company’s advanced research lab, and the two were later joined by Nelson. The move to Datapoint was a concession to the reality principle, as well as an acknowledgment that the most important aspects of the Swarthmore group’s work so far had been design rather than coding. At Datapoint, the Xanadu programmers could explore their ideas in a corporate setting that offered the latest equipment and a decent paycheck.
Gregory continued to nurse the project along. After the Xanadu programming team dissolved in the early ’80s and Nelson moved to Texas, Gregory also abandoned Pennsylvania. He returned to Michigan, where, for a while he and some Xanadu kibitzers lived at the apartment of a friend. During another period, Gregory and his fellow hackers camped out in a suburban crash pad, with a sympathetic hippie couple who fed them, lent encouragement, and helped them find jobs. Xanadu, after years of high hopes, had become a charity case, dependent for its survival upon the kindness of friends.
From its rosy expansion at the turn of the decade, the project had, by 1984, collapsed into a constricted sphere of hackers clustered around Roger Gregory. Despite its reduced scope, the hypertext dream still possessed a powerful gravitational field. Few people who came in contact with it could tear themselves completely free. Rather, programmers tended to depart on elliptical orbits that took them far away and then, eventually, brought them back.
For instance, Michael McClary, the programmer who introduced Gregory to the concept of hypertext 10 years earlier, joined the Xanadu project briefly after it returned to Michigan. McClary was quiet, hippieish and, by the time he hooked up with Xanadu, an expert in writing long, complex programs in C. His method was to take a few days to absorb the design, plot out his approach carefully, and then implement his plan in a long stretch of sustained concentration. According to his colleagues, McClary took about three times as long as most programmers to come up with a first version – but his first try usually worked.
When Gregory returned to Michigan from Pennsylvania, McClary noticed that Gregory resisted suggestions to formalize Xanadu’s business arrangements. There were no contracts, no documents, and no organization. Gregory and his irregular helpers took copious notes but never referred to them again. Gregory held a meeting once a week to try to decide what was next, but instead of addressing the programming requirements, the conversation would wander aimlessly from snide personal attacks to grand philosophical speculations. After witnessing the process for a few months, McClary got the impression that he wasn’t part of a software development team but of a sect in the process of self-destruction. McClary also noticed that there was nothing to enforce any claim that the hackers might have on the fruits of their labors. When he asked about ownership, Gregory explained casually that someday everybody would get a fair share. McClary watched as Gregory, succumbing to his innate emotional volatility and disappointed by years of unsuccessful labor, drove his associates away. McClary finally quit the project after Gregory struck him during an argument.
Yet there were rays of hope. In 1987, Nelson revised Literary Machines, a book-length description of hypertext he had first published in 1981. The style of the book was pure Nelson: it had one Chapter Zero, seven Chapter Ones, one Chapter Two, and seven Chapter Threes. In his introduction, Nelson suggested that the reader begin with one of the Chapter Ones, then read Chapter Two, then explore a Chapter Three, and then start again, passing repeatedly through Chapter Two. He also provided a diagram, with the comment: “Pretzel or infinity, it’s up to you.” The official title page reads: Literary Machines: The Report On, And Of, Project Xanadu Concerning World Processing, Electronic Publishing, Hypertext, Thinkertoys, Tomorrow’s Intellectual Revolution, and Certain Other Topics Including Knowledge, Education and Freedom.
But there was no money to turn the elaborate design presented in Nelson’s book into something concrete. Even Roger Gregory was running out of steam. With the exception of Gregory, all the major Xanadu programmers were intensely involved in other work. By the mid-1980s, the most rational hope for Xanadu was that the project would exercise its power indirectly, through the work of Miller, Greene, and others in companies scattered around the world. Miller, by now a rising professional, moved to Xerox PARC, the source of many of the personal computer industry’s most important designs.
For Xanadu’s two parents, things were more difficult. Gregory, still clinging to the incomplete collection of Xanadu code, made regular presentations to possible funders, but could not spark their interest. Nelson was living in San Antonio and, after Datapoint collapsed in a financial scandal, he began running out of money.
Enormously bitter, he was furious at the setbacks but helpless to grieve and move on. Around this time, Nelson contemplated suicide and got as far as holding the pills in his hand. He ended his revised version of Literary Machines with words of farewell: “We have held to ideals created long ago, in different times and places, the very best ideals we could find. We have carried these banners unstained to this new place, we now plant them and hope to see them floating in the wind. But it is dark and quiet and lonely here, and not yet dawn.”
The Xanadu hackers may never have produced a working implementation of their design, but they possessed a profound foreknowledge of the information crisis soon to be produced by digital technology. They were dead-accurate when they sketched a future of many-to-many communication, universal digital publishing, links between documents, and the capacity for infinite storage. When they started, they had been ahead of their time. But by the mid-1980s, they were barely ahead of it.
When Gayle Pergamit first heard about Xanadu, in about 1980, she recognized that Nelson’s concept and Gregory’s tenacity could revolutionize the software industry. She also recognized that the endeavor was foundering. Pergamit’s husband, Phil Salin, had written the studies that helped guide the breakup of the Bell system. He was well informed about computers and obsessed with the idea of electronic markets for information. During the ’80s, Salin was creating a computer network designed to allow the sale and exchange of data and expertise. He worked closely with Pergamit, a business consultant whose sympathetic nature and uncommon understanding of programmers’ wants and needs made her an ideal liaison between managers, suppliers, and hackers. Pergamit sympathized with Nelson and Gregory’s quest, but she also saw that both, particularly Gregory, were in need of help.
“At the time,” Pergamit remembers, “you could open the San Jose Mercury News and go through page after page after page of want ads for computer programmers. And out there in Ann Arbor, programmers couldn’t find work. Forget funding – they couldn’t even find work.”
Pergamit and Salin begged Gregory to move west. Like Nelson, Gregory hates to throw anything away, and the prospect of transferring his collection of old computers and his thousands of books across the country was daunting. But in 1983, he gave in, dragging with him the motley hardware on which various parts of Xanadu had been hacked. Unfortunately, Pergamit and Salin’s analysis was only partially on target; although Gregory found work in Silicon Valley, Xanadu languished.
Gregory, of course, would not admit failure. He maintained a life-support system for Xanadu, including a post office box and a few items of printed propaganda – like Xanadu’s employee roster, on which Ted Nelson was listed at the top as “Director” and Roger Gregory was in second place as “System Anarchist”; Mark Miller was described as the “Hacker,” Phil Salin was the “Accelerator,” and Gayle Pergamit was “Hidden Variable.”
One member of the Xanadu roster held the title “Speaker-to-Bankers,” but if he was speaking, the bankers weren’t listening. Over the years, Gregory had become a fixture at programmers conventions, where he displayed an unconvincing Xanadu demo. His natural habitat was at the Hackers’ conference, a yearly, invitation-only conclave that started as an informal gathering for people written about in Steven Levy’s bestselling book Hackers. Attendance at Hackers expanded over the years, and it became the chief gathering of the unofficial computer élite.
During 1987, the year of Hackers 3.0, Gregory was working, somewhat grouchily, for Cirrus Logic. He took time off to attend the conclave, located that fall at Camp Swig, a Jewish camp in Saratoga, California. The programmers stayed in stilt-raised cabins and had their general sessions in a large, rustic room with a stone fireplace. There were demonstrations of virtual reality headgear, a Coca-Cola-powered battery, and a mobile robot named Louis; there were also edifying sessions on viruses and worms, on neural networks, on fractals, and, at 2:45 on Saturday afternoon, on the question: “Can a Hacker Still Make It BIG?”
One of the attendees was a hacker who had recently made it very big. Perched stiffly by an open window during one of the sessions, wearing an open-collar, short-sleeve shirt and dark-rimmed glasses, was John Walker, the legendary founder of Autodesk. Walker was still at the helm of the company he founded, whose sales had grown from $15,000 in 1983 to more than $54 million in 1987. He knew little about Gregory’s labor of love, but he had heard of Xanadu.
Roger Gregory had had many conversations with many investors over the years. The investors would talk to Gregory at a conference, become interested, make an appointment, visit the site where Xanadu happened to be located, view the bits and pieces of the system that Gregory had managed to cobble together, go back to their companies, write memos describing what they saw, and never talk to Gregory again.
Walker was different. He once described Autodesk as an organization composed of people who would rather write a book than spend 10 minutes on the phone. Walker realized the Xanadu code was not finished, but he also noticed that Xanadu had never had the benefit of a serious, commercial development effort. He suspected that with the help of Autodesk, which was founded to give its original partners, themselves programmers, a way to produce and sell their tools, Xanadu might be transformed from a cult into a company. And when the founder of Autodesk wrote an enthusiastic note about Xanadu, his executives were inclined to pay attention.
Walker’s overture was followed by a period of intense negotiations. Phil Salin and Roger Gregory spent months working with Autodesk’s attorneys. Immediately, the Xanadu crew’s casual business arrangements came back to haunt them. Ted Nelson insisted that no sale or license to Autodesk interfere with the inventor’s grand scheme for a universal library and publishing system. Nelson wanted to ensure that if Autodesk had a working product, he would have complete freedom to use it in his Xanadu roadside information franchises.
Autodesk cared little about becoming the McDonald’s of cyberspace; its plans focused on commercial tools for sharing, distributing, and editing documents. Still, it was not easy to craft a set of contracts establishing both Nelson’s freedom to use the Xanadu technology and Autodesk’s ownership of it. In the end, the solution Salin, Gregory, and Autodesk negotiated was called The Silver Agreement, and it generously gave to Nelson the exclusive right to build a royalty-based publishing system using any Xanadu technology perfected by Gregory and Autodesk. Nelson had a right to the name Xanadu; the new company, owned largely by Autodesk, was called Xanadu Operating Company.
One benefit of The Silver Agreement was that it allowed the programmers to pursue commercial applications of Xanadu under the direction of Autodesk, without constant interference from their demanding founder. Nelson would be given a prestigious title, “Autodesk Fellow,” and an office at Autodesk headquarters, but he would have no direct role in managing the development of the software. This arrangement was important, for while Nelson’s presentations were inspiring, his high self-regard and his pronounced difficulty organizing and finishing tasks made him an ineffective manager. By granting Nelson an exclusive license to use Xanadu in any royalty-based publishing scheme, Autodesk believed it was giving the inventor what he wanted most while keeping hold of the most important commercial decisions about what Xanadu would become and how it would be sold. In time, however, the partners would discover ambiguities in The Silver Agreement. “With the benefit of hindsight,” said one former Xanadu executive, “I’d say the lawyers who crafted that agreement should be shot.”
Yet in 1988, the Autodesk deal was nothing but good news. On April 6, John Walker issued a press release announcing that Autodesk would acquire 80 percent of Xanadu. Most of the rest of the company would be shared between the programmers and various individuals who had funded Nelson and Gregory over the years. Autodesk offered Gregory steady employment as well as sufficient development support to see his long-delayed project to completion. And Autodesk promised to have Xanadu to market within 18 months.
“In 1964,” announced Walker, confidently, “Xanadu was a dream in a single mind. In 1980, it was the shared goal of a small group of brilliant technologists. By 1989, it will be a product. And by 1995, it will begin to change the world.”
Gregory had gone to Hackers 3.0 under a cloud. By the middle of the following summer, he was a lead programmer in a software company whose research budget was nearly a million dollars a year.
John Walker’s Autodesk created the dominant software in the field of computer-aided design. The Xanadu acquisition reflected Walker’s hope that Autodesk could also pioneer the fields of virtual reality, information markets, and space exploration. Along with Xanadu, Autodesk bought Phil Salin’s information-exchange company, AMIX (American Information Exchange). In a contemporary memo to his company, Walker preached to his colleagues: “Reality isn’t enough any more.”
For Xanadu’s programmers, Autodesk’s 1988 investment reversed all the vectors of the project’s history. More than a half-dozen programmers regathered. The flow of cash also turned around; suddenly Xanadu was supporting Gregory instead of vice versa. Nelson, always a gadfly, was safely tucked away inside Autodesk’s Sausalito headquarters. The programmers’ offices were more than an hour down California Highway 280, in Palo Alto.
Always dependent on the support of friends and strangers, and used to laboring on the fringes of unsympathetic institutions, the hackers of Xanadu finally had a chance to establish their own working conditions and to build an environment ideally suited to creative work. Gregory’s response to this freedom was touching. According to one of the programmers, his contract with Autodesk gave him a budget that specifically ensured the office had comfortable furniture and nutritious food.
Mark Miller gave himself up to Xanadu’s pull and rejoined the project full time. The new Xanadu site on Palo Alto’s California Avenue was remodeled to resemble the environment at Xerox PARC. The programmers’ offices opened onto a large common space, and the walls were covered with white board, which quickly became a tangle of multicolored lines, words, circles, and squiggles.
Gayle Pergamit helped Xanadu establish some basic accounting and purchasing systems, but the programmers’ attention was never focused on business details. Rather, they took the opportunity to contact everybody they felt might be able to assist them in their last, grand, 18-month push. Along with Miller, Dean Tribble also came from Xerox PARC. Other programmers who had contributed over the years, including Eric Hill and Roland King, joined the team.
Using a combination of stock and salary, the Xanadu team also lured Michael McClary to California. McClary had plenty of experience taking obscure directions from technical managers and turning them into massive, working programs in C. He abandoned his lucrative Michigan consulting practice to rejoin the project he had left unfinished nearly 10 years before.
Xanadu’s most unlikely new recruit was Marc Stiegler, who became the project’s manager. Stiegler was an even-tempered software developer whose just-published science fiction novel, David’s Sling, presented a scenario in which a hypertext system saved the world. After working for nine years in the software industry, Stiegler had earned enough money to take some time off. But Xanadu, with its daunting record of failure, enticed him nonetheless.
Before the Autodesk acquisition, Stiegler had met Nelson at a CD-ROM conference sponsored by Microsoft, where he found himself in an audience of 1,000 listening to a speaker he didn’t recognize. He was looking at a Xanadu flyer, which was absurdly amateurish, and he was listening to Ted Nelson’s presentation, which was manic. Stiegler’s first impulse was to laugh. Then, like so many earlier Xanadu recruits, he was touched by something in Nelson’s proposal that transcended plausibility. Through the primitive medium of Xanadu’s printed materials and Nelson’s barely convincing lecture, Stiegler thought he heard a call from the future. “I was looking at this, frankly, crude flyer,” he said, “and listening to this guy talk about Xanadu, and I was sitting there thinking, you know, if this guy can really pull it off, he’s going to change the world. I looked around at all the other people in business suits and I realized that I was the only person in the room who understood.”
As soon as Nelson was finished, Stiegler hurried over to the stage entrance, where he found Nelson, better known than Stiegler realized, surrounded by a dozen admirers. Stiegler waited patiently, and when everybody had had their say, he stuck out his hand. “Do you have a team?” he asked. “How are you funding it?”
“We are funding it out of our hide,” Nelson answered.
Stiegler thanked the inventor and walked away. “I knew something like this was not going to be done by three guys in their spare time,” he says.
In 1988, however, Stiegler’s desire to meet Eric Drexler brought him to the Xanadu office, and Phil Salin went to work explaining to the successful executive that Xanadu represented the chance of a lifetime.
The match between Stiegler and Xanadu was doubly unlikely; not only was Stiegler happily unemployed, but the Xanadu programmers did not seem to place high value on management personnel. As Stiegler tells it, the original plan during the first days at Autodesk was to get somebody with a good résumé and stick him in a closet until somebody from Autodesk came to visit, at which point the obedient manager could be trotted out to prove that the hackers were under control. This was hardly Stiegler’s style.
“The place was pretty chaotic,” Stiegler says, explaining how strongly he was affected by the idealistic programmers’ passionate quest and obvious need for assistance. “But Xanadu had this amazing effect – it was irresistible.”
This irresistibility derives, first, last, and always, from the grand Xanadu dream. Stiegler wasn’t sure that Xanadu would work, but if it did, the consequences would be magnificent. Gathered together in a nice, new office in Palo Alto, with fully stocked refrigerators and comfortable furniture, the Xanadu team prepared to build the ultimate hypertext system. For once, they had tools, including as much computing power as their hearts desired. Regular paychecks allowed them to be revolutionaries and pay their rent. And even their executive manager accepted that their mission was to change the world.
Of course, the new situation also had its confusing aspects. In 1988, Xanadu was forced for the first time to operate as a commercial software concern. Xanadu’s regular Tuesday meetings were messy; Nelson would arrive from his Sausalito office with his note cards and his tape recorder and his video camera and wave his hands furiously in front of the white boards. Although he did not control the development process, Nelson’s energetic lectures ensured that nothing in his grand design was forgotten. When Nelson was not presenting, Miller and Gregory argued about the value of the work completed during and since the Swarthmore summer, and the programmers played their favorite game, in which any moment of aphasia or unsuccessful mental search for the name of an author or book was followed by the traditional exclamation, “If only we had Xanadu!”
Stiegler saw that he had his work cut out for him. “It was a complicated time,” he says today. Looking around the office, he attempted to divine who could help the company move from volunteerism to profitability. Divisions were already brewing: on one hand, the Xerox PARC alumni favored the new programming language Smalltalk and found themselves often in agreement; on the other, the old-style C hackers, like Johan Strandberg, McClary’s closest friend on the project, tended to be more skeptical, traditional, and careful.
Then there was Roger Gregory. Stiegler describes his situation with a parable. “Say you’ve got a volunteer organization,” Stiegler says, “and you have to go to the North Pole. And there’s a guy who is going east, but he’s drifting north. This guy is a hero. He’s going mainly east, but he’ll eventually get to the North Pole. He’s a hero! But in a company where you are paying salaries and where you will run out of money eventually, the guy who is traveling east and drifting north is somebody you have to fire.”
Roger Gregory had promised Walker that the project would be finished in 18 months. The design was completed in the early ’80s, and the task now was to quickly embody this design in sufficiently bug-free and elegant code.
Gregory believed that he had large amounts of the Xanadu code near completion. Over the years, he and various contributors had built a prototype of a Xanadu server, the central machine that stored the information and made it available to users on remote client machines. During the first months at Autodesk, this Xanadu server was the focus of most of the programmers’ attention, and Gregory got far enough to send an alpha version out to some users to examine. They examined it. It was broken.
Perhaps the alpha version was fixable. But Miller felt the problem was deeper than the old code. Even if the server worked, it might not work well enough to rescue Xanadu’s battered reputation. Nelson had been stumping for his universal hypertext system for 25 years, and the computer industry had already grown supremely comfortable with the notion that the product was vaporware. The Autodesk acquisition inspired a new round of press coverage, which raised the level of doubt another notch. Ten years after the Swarthmore summer, Miller did not want to release a creaky and crippled version of the software he had helped design.
The basic features of the Xanadu hypertext system planned at Autodesk in 1989 were relatively unchanged from the ones discussed by the early Xanadu programmers at Swarthmore in 1979. In fact, the design was still similar in many respects to the sketches Ted Nelson made back in 1965. Xanadu was to consist of easy-to-edit documents. Links would be available both to and from any part of any document. Anybody could create a link, even in a document they did not write. And parts of documents could be quoted in other documents without copying. The idea of quoting without copying was called transclusion, and it was the heart of Xanadu’s most innovative commercial feature – a royalty and copyright scheme. Whenever an author wished to quote, he or she would use transclusion to “virtually include” the passage in his or her own document.
Nelson was frequently frustrated by his failure to convince casual questioners of the importance of his transclusion idea. Transclusion functions like the “make alias” command familiar to Macintosh users. An alias works as a fully functioning copy of a file or application, but it is really just a pointer, or virtual copy. Click on the virtual copy, and the original file or application begins to run.
The key to the Xanadu copyright and royalty scheme was that literal copying was forbidden in the Xanadu system. When a user wanted to quote a portion of document, that portion was transcluded. With fee for every reading.
Transclusion was extremely challenging to the programmers, for it meant that there could be no redundancy in the grand Xanadu library. Every text could exist only as an original. Every user in the world would have to have instant access to the same underlying collection of documents.
Miller noted that the current version of Xanadu handled transclusion in an extremely clumsy fashion. It also lacked the ability to keep track of different versions, did not scale well, had no multimedia capabilities, no security features, and performed poorly. The years of work Gregory had devoted to writing code seemed as much a burden as a resource. Miller wondered if it wasn’t time to wipe the slate clean and start again.
Soon after the Autodesk investment, the power to control Xanadu’s development began to slip from Gregory’s grasp. His erratic behavior prevented him from rallying support as Miller and Stiegler took charge. And, at least at first, Gregory trusted Miller. The two had been working on Xanadu for many years and had together invented the tumbler addressing system.
McClary, like Gregory, had less and less influence as the months passed. To the programmers from Xerox PARC, his work appeared slow. McClary’s years of consulting at large Michigan auto companies had taught him the virtue of silent forbearance, and when he became aware that his opinion hardly counted, he retreated into ill-spirited silence. Soon after his arrival in Palo Alto, McClary was shuffled into a small office he resentfully called “the phone closet,” which gave him a quiet seat from which to observe the action.
By 1991, the split in the Xanadu programming team was growing. On one side were the Xerox PARC computer scientists who were prepared to use the latest, avant-garde programming tools to completely reprogram the hypertext system. On the other were Gregory, McClary, Johan Strandberg, and a shifting group of Xanadu hangers-on who skeptically sniped at what they saw as Miller’s pursuit of an ever-receding mirage of design perfection. Eventually, Stiegler fired Strandberg. And Gregory only kept his job thanks to his history with the project, his partial ownership of Xanadu Operating Company, and his special relationship with John Walker.
Although Gregory stayed on, the Xerox PARC programmers won all the battles, beginning with the most important one. Gregory’s old Xanadu code was thrown away. The programmer’s face, seven years later, still goes slack with disappointment when he thinks about it.
For 12 years of missed deadlines, Gregory had nurtured his complex, nonworking, but possibly fixable technology. His code was the accumulation of all Xanadu’s relationships and struggles since the early ’70s. Through those years, he had been sustained by his belief that Xanadu was close, very close, to success. “Stiegler and Miller screwed up the entire thing,” Gregory says. “I had something that was within six months of shipping.”
After deciding to jettison the old code, which meant an implicit agreement to ignore John Walker’s 18-month deadline, Miller and the other lead architects made a second key decision. They chose to program in Smalltalk. To the programmers who had been at Xerox PARC, Smalltalk seemed perfectly suited to the rapid transformations of their design. Smalltalk is a programming language based on distinct modules of code that can be strung together into a working program. Program prototypes could be hacked out in Smalltalk in days.
Theoretically, a Smalltalk prototype could be handed to a programmer like McClary, who could then rewrite it in C++. This became the Xanadu strategy: prototype in Smalltalk, translate to C++. Frequently, by the time McClary started work on the translation, the design had evolved into a new shape.
Encouraged by the flexibility of Smalltalk, the Xanadu architects became obsessed with developing the widest possible applications of hypertext technology. A universal democratic library, they decided, was only the beginning. Xanadu could also provide a tool for rational discussion and decision making among very large groups. In the Xanadu docuverse, an assertion could always be followed back to its original source. An idea would never become detached from its author. Public discussion on important issues would move forward logically, rather than merely swirling ineffectively through eddies of rhetoric. In fact, any reader could, by creating and following links, freeze the chaotic flow of knowledge and grasp the lines of connection and influence. Although Nelson was up in Sausalito, Miller, Tribble, Stiegler, and another Xerox PARC alumnus named Ravi Pandya were replicating his wildest fantasies about Xanadu’s power to change the world.
In a paper titled “The Open Society and Its Media,” Miller, Tribble, Pandya, and Stiegler pointed out that with transclusion, links to critical information would remain intact no matter how many times a passage was quoted. No form of communication in history had ever offered this possibility. In books, television, and radio, the truth is a slave to a good story, and convincing lies are remembered while dry, factual refutations are forgotten. In Xanadu, this problem is solved. Transclusion and freedom to link are crucial to social progress, the programmers argued, because otherwise, the constant mutation of a discussion “would destroy selection by leaving criticisms behind.”
The echoes of evolutionary theory were intentional. During the weeks, months, and then years of sophisticated redesign of Xanadu at Autodesk, the architects began to believe they were helping human life evolve into an entirely new form.
Under Autodesk, Miller had complete freedom to pursue his mathematical solutions for data storage and retrieval, and he found enthusiastic companions in Tribble and Pandya. But the problems they were solving were general issues of hypertext design. They did not have a customer in mind, and they gave little thought to the ways their hypertext system would be used. To Gregory and McClary, the three scientists from Xerox appeared to be working purely for the sake of mental pleasure.
The split between the programmers widened when, during one Tuesday meeting, Miller held a ceremony to proclaim Tribble and Pandya “co-architects” and handed them a baton to symbolize their new authority. McClary was embarrassed and insulted. As the horizon of a release date continued to recede, the atmosphere at the company’s offices grew increasingly unpleasant. Tribble and Miller rented two-thirds of a triplex, and, according to McClary, began to hold meetings without the other programmers present.
McClary recalls numerous surreal incidents. Once, Miller called the technical staff together and lectured at length about Xanadu’s final shape. It took McClary some time to absorb everything and come up with his questions, but when he returned to Miller to explore the issues further, he discovered that every single thing he had wondered about had been completely redesigned.
“It was not rapid prototyping – it was rabid prototyping,” said one of McClary’s friends who watched the project closely. “They were just randomly hacking and coming up with these groovy algorithms.”
While Tribble, Dean, and Miller continued to hack, Jonathan Shapiro, one of the founders of HaL Computer Systems Inc., was hired to write documentation. To get his bearings, he challenged the Xanadu architects to describe a typical customer for their software. He found their answers vague. In Miller’s view, the Xanadu technology was so radical that predicting its future uses was difficult. Writers, teachers, and scientists; movie directors, commodities brokers, and sports fans – Xanadu promised to remake everything.
Shapiro also discovered that the group had been working together so long it had developed a kind of private slang. It took months to comprehend what the programmers were talking about. Most of them were book lovers and trivia mongers who enjoyed developing a metaphor based on obscure sources and extending it via even more unlikely combinations. For instance, the object in the Xanadu system that resembled a file was called a bert, after Bertrand Russell. With files called bert, there had to be something called an ernie, and so in the Xanadu publishing system, an ernie was the unit of information for which users would be billed. To understand the details of Xanadu, Shapiro had to learn not only the names for things, but also the history of how those names had come to be.
Ted Nelson also found the slow progress of Xanadu distressing, but his mind was on other subjects. He had arrived in California extremely burnt out and depressed, and on the advice of a former girlfriend, he signed up for some sex-liberation seminars. At what was then called the Stan Dale Sex Workshops (and has since been renamed The Human Awareness Institute), Nelson received what he calls a Great Healing. He soon took all the workshops.
Nelson continued to develop his general philosophy, General Schematics. One branch of General Schematics involved his Xanadu designs, but another branch was what he called “The General Theory of Status, Territory, and the Paradigm.”
A visitor to Nelson during his years at Autodesk recalls an evening when the inventor, wearing a velvet vest and a satin shirt, lectured about social status and its relationship to an internal, biological status regulator, called a biotstat. However, Nelson’s book on the topic, Biostrategy and the Polymind, which he considers the “foundation” for the next generation’s social sciences, was never published because he mislaid the computer printout with his revisions.
Nelson remained proud of his ambivalent relationship with computers. Right at the peak of the desktop-publishing frenzy, Nelson became obsessed with non-computerized xerox machines, Post-it notes, and transparencies. And yet, despite his scattered interests, Nelson continued to exercise an influence over the Xanadu programmers. For instance, Nelson’s theory of language holds that every time a concept changes, the word to describe must change as well. There ought not be any “slippage” of one term into another. New idea, new word. Applied to the development process at Xanadu, this rule meant a constant stream of fresh jargon; the system was filled not just with berts and ernies, but also with “flocks,” “shepherds,” “abrahams,” “dybbuks,” and “crums.” Because the programmers found it difficult to strictly follow Nelson’s “one-word, one-idea” rule, some slippage did occur. Working at Xanadu offered a constant flow of scholastic argument over throwing out names, switching names, and substituting names.
John Walker, Xanadu’s most powerful protector, later wrote that during the Autodesk years, the Xanadu team had “hyper-warped into the techno-hubris zone.” Walker marveled at the programmers’ apparent belief that they could create “in its entirety, a system that can store all the information in every form, present and future, for quadrillions of individuals over billions of years.” Rather than push their product into the marketplace quickly, where it could compete, adapt, or die, the Xanadu programmers intended to produce their revolution ab initio.
“When this process fails,” wrote Walker in his collection of documents from and about Autodesk, “and it always does, that doesn’t seem to weaken the belief in a design process which, in reality, is as bogus as astrology. It’s always a bad manager, problems with tools, etc. – precisely the unpredictable factors which make a priori design impossible in the first place.”
Miller, of course, did not agree. He knew that Xanadu’s delays were frustrating. But he also saw that despite the carping by critics both inside and outside the project, they were making significant progress toward a real, revolutionary hypertext system. By the end of 1991, Miller felt that the most difficult design problems had been solved. “The last year was the scariest,” Miller says, “because until you come up with those last answers, you fear that this one little thing you left until the end might blow up into a disaster. But none of them blew up! That was the most fun of all.”
Miller, having pushed through the skepticism of his colleagues, mastered the manic design process, and parsed the peculiar vocabulary, recalls that he was nearly prepared to show his work to the world when, in February 1992, Autodesk celebrated its 10th anniversary with an announcement of a bad sales quarter and a catastrophic collapse of its stock.
The stock was first pounded when the company reported an unprecedented drop in earnings for the fourth quarter of fiscal year 1991. Within a day, the stock dropped from $52 to about $40. Autodesk quickly rallied, but beneath the surface, chaos spread as managers desperately cut spending and reduced investment. When Walker returned to Autodesk from his now-permanent residence in Switzerland in January before the announcement, he found his company “heading for a train wreck of Wagnerian proportions.” Bravely, he announced the bad news personally at a major shareholders meeting, where he also outlined the company’s plans for recovery. But the stock still plummeted. From its high of more than $60 in 1991, Autodesk had dropped to 23 1/2 by mid-February 1992.
Autodesk’s pounding by Wall Street produced a number of computer-industry legends, including the story of Walker’s famous attack on his own company’s management. His managers were “genial, well-meaning, and for the most part hard-working individuals,” Walker had written, with merciless condescension, before describing their passivity and incompetence.
The poor sales figures and Walker’s undiplomatic attack, which circulated widely outside Autodesk, spelled trouble for Xanadu. To industry analysts with influence over the price of Autodesk shares, the crisis at Autodesk looked like evidence of a battle between headstrong hackers who built the company, such as Walker, and professional managers who arrived later. A reorganization along more conservative lines was inevitable, and by April, Autodesk had found a new person to take charge. Carol Bartz’s task during her first months on the job was to take a stern inventory of the company’s most promising projects. And four months after she became CEO of Autodesk, Bartz announced that the company’s investment in Xanadu was finished.
When Autodesk crashed, tales of Xanadu’s eminent demise were greeted by many in the computer industry with a smirk; after all, the whiz kids had been hacking away for four years, spending piles of corporate cash, and the system was still as vaporous as ever. But to Mark Miller, the timing of the Autodesk crisis was extraordinarily cruel. Today, Miller insists that the programmers at Xanadu had their hypertext system within a heartbeat of completion. How long a heartbeat? “Six months,” he says, sincerely.
Roger Gregory was crushed by the programmers’ failure to fulfill his promise to Walker that a working system would be available within 18 months. He had been overruled by Miller and the other more well-spoken members of the design team when they wanted to trash the first Xanadu code, he had been brushed aside by Stiegler, and he had been kept by his own bad temper and depression from influencing Xanadu’s development. Now, as the Xanaduers contemplated a cashless future, Gregory had nowhere to go. The other architects had promising research and industrial careers. Gregory was left with a small home in Palo Alto and his unrequited love for the hypertext future.
For Michael McClary, the end of the Autodesk investment was an opportunity to cut himself loose. He returned to private consulting. Stiegler also looked for an exit. He paused briefly to help AMIX, also dropped by Autodesk, make a transition to independence, and then retired to a ranch in Arizona. Xanadu had been the most frustrating professional experience he had ever had. “Xanadu,” he says now, “constantly defeated me.”
But the three lead architects – Pandya, Tribble, and Miller – were not quite ready to quit. They announced that they’d give up their larger dreams of hypertext, at least temporarily, if the right backers could be found to support a more modest product. Finding these backers became Jonathan Shapiro’s job.
Shapiro had a crucial advantage over Marc Stiegler and Xanadu’s other mentors and supervisors since 1965: he did not believe hypertext was going to save the world. The aggressive posturing, grand dreams, and boastful proclamations that typified Xanadu since Nelson first started offending his professors were part of the project’s childhood and adolescence. Now, after a few hard knocks, Xanadu seemed ready to grow up. Shapiro quickly began working with Miller and the other designers to accomplish what they had always felt free to ignore – identifying the specific, present-day commercial needs Xanadu could meet, for instance, and creating materials to show potential backers.
In a race to prevent years of effort from disappearing into the trash heap of unreleased Autodesk software, Stiegler fiercely lobbied Autodesk for some transitional funding to keep Xanadu alive. Autodesk, after much discussion, charitably gave the Xanadu team a small amount of cash. Meanwhile, Shapiro tried to find buyers.
The programmers vacated their Palo Alto offices and moved into Dean Tribble’s home. After Autodesk announced divestiture, in August 1992, ownership of Xanadu Operating Company reverted to the programmers and a few other longtime Xanadu supporters. Roger Gregory and Ted Nelson now owned about half the company.
Nelson was startled by this turn of events. Every time the inventor had asked about Xanadu’s progress at Autodesk, he had been told that the system would be ready within six months. It was not until a Xanadu meeting in the summer of 1992 that he first felt the cold shock of reality. “This feeling came over me – my God, they are not going to do it,” he says. “I had believed them all this time.”
Nelson watched the spin-out warily. When Marc Stiegler left Xanadu, Jonathan Shapiro became CEO of the newly independent company. The new executive concluded that the key to Xanadu was its potential as part of a publishing and royalty system, and he reached out to a company that was attempting to manage an enormous number of royalty and copyright contracts – Kinko’s. Xanadu’s proprietary data structure offered the possibility of a unified tracking system for all the college material Kinko’s was printing. Using the transition funding from Autodesk, along with a workable demonstration of the system, Shapiro believed he could get Xanadu into a deal with Kinko’s or another publisher within 30 days.
But the Kinko’s deal resembled the royalty-based publishing scheme to which Ted Nelson, and not Xanadu Operating Company, had exclusive rights. In the end, Jonathan Shapiro did not manage to sell Xanadu to Kinko’s. Instead, the Xanadu programmers staged one of the most bizarre shareholder battles the bewildered executive had ever seen.
Until 1987, Xanadu had been a cooperative venture, a brave band of fellow crusaders whose credo was “share and share alike.” Some, like Michael McClary, recognized the instability of such arrangements, and shied away from participating until stock could be granted and salaries paid. But The Silver Agreement in 1988 created two Xanadus. Nelson’s Xanadu was his imaginary system of information franchises. The Silver Agreement gave Nelson exclusive right to any royalty-based publishing business. Meanwhile, Xanadu Operating Company retained ownership of software being developed by Roger Gregory and others. The Silver Agreement required Xanadu Operating Company to give Nelson the Xanadu software for use in his Xanadu franchises, while allowing the company to control the development of the software and to use it in any other commercial venture.
Nelson’s success was dependent on Xanadu Operating Company’s success – there could be no franchises without the underlying technology. And Nelson remained a large shareholder in the company. But so far, Nelson’s franchises had been an illusory business based upon a nonexistent technology, a dream built on a dream.
Now that the fantasy threatened to become real, certain aspects of The Silver Agreement seemed vague. What, after all, was publishing? If Kinko’s were to track its copying business with Xanadu technology to meet agreements with the owners of the copyrights, was this not perilously close to the royalty-based publishing business exclusively reserved for Nelson? And Xanadu Operating Company had another problem. Most of the programmers owned only a negligible amount of stock. Now that Autodesk had cast them out, they were facing a period of hard work for little pay. Shapiro wanted to spread the ownership of the company a little deeper. Nelson, however, did not feel like sharing his stock.
Just as the negotiations with Kinko’s were getting under way, Nelson, whose lifelong dream was about to take its first step toward genuine, if diminished, realization, attempted to take over the company. The programmers, who had seen Nelson’s management style firsthand during the early ’80s, resisted.
“There wasn’t anything to fight over,” Shapiro says. “If we did not complete the technology and sell it, everybody would die. But Ted was determined to control it. The more determined Ted got to control it, the more determined the programmers got not to be under his thumb.”
Nelson blamed Miller, Stiegler, and Shapiro for Xanadu’s long delays. He had resigned himself to losing control of the software development process at the time of the Autodesk investment, but he had consoled himself with the thought that the professionals were qualified to complete their task. Now that these professionals had failed decisively, Nelson wanted his company back.
The programmers declined to work for Nelson. Miller and Shapiro were confident they could retain control of Xanadu, since Nelson possessed neither the skill to finish the code himself, nor the money to hire new programmers. But they were facing a master strategist who understood the power of escalation. Nelson soon found a way to provoke the desired crisis. “I nominated Roger Gregory to the board of directors,” recounts Nelson triumphantly. The two owned nearly half the company, and together they could thwart nearly any plan. “The reaction,” Nelson says, “was as if I had set fire to the curtains.”
The final battle for control over the remains of Xanadu was not nice. After Stiegler’s resignation, Shapiro had come to represent, to Nelson, the narrow-minded managers and punishing authority figures the inventor despised. To Nelson, Shapiro was “an asshole.” To Shapiro, Nelson was “an arrogant bastard.” Nelson claims not to remember the details of the conflict, but according to Shapiro, the end came at a board meeting in the end of 1992, when Nelson said frankly that he was not going to cooperate with the plans of any company that had Shapiro in control.
Shapiro countered that if Nelson would give more ownership of the company to the programmers, he would agree to resign as chief executive officer. Nelson accepted the deal, the shares were re-distributed, and Shapiro departed.
For the programmers, it was a Pyrrhic victory. By the time the battle was over, Kinko’s senior management had stopped returning phone calls, most of Autodesk’s transitional funding had been spent on lawyers fees, and the Xanadu team had managed to acquire ownership of a company that had no value.
Xanadu, so many times dead, was dead again.
While Xanadu was dying, Charlie Smith was starting a company. It was called Memex, and its first product would be a record-keeping system for insurance companies. Smith examined the remains of Xanadu, which, though it had no money, no working code, and no prospects, did possess some groovy data-storage and retrieval algorithms Smith thought he might use in his soon-to-be built software.
What Smith offered was a half-success – barely. Under Memex, the code would be torn from its integrated global information network. Xanadu, heir to so many hopes, would become an organ donor, its powerful algorithmic heart beating in the center of an insurance-industry database.
Smith had little money. But the wrangling over the dying Xanadu had sapped the participants of their possessiveness. When Memex offered to license the technology and hire some of the employees to complete it, Miller, Tribble, and Pandya, along with fellow programmers Christopher Hibbert, Eric Hill, and Rob Jellinghaus, signed on. Gregory, while remaining a large shareholder of Xanadu Operating Company, did not go to work at Memex. Jonathan Shapiro was replaced as Xanadu executive manager by a newcomer, Ann Hardy. From their headquarters in Dean Tribble’s house, the surviving Xanadu programmers moved back to California Avenue in Palo Alto, where Memex had its offices. The quarters were familiar. It was the same building that had housed Xanadu during the years it was supported by Autodesk.
The youngest member of the Xanadu programming team, Rob Jellinghaus had been with Xanadu for about two years when Autodesk cut off its support. He was a lean, inexperienced 23-year-old, yet he seemed more worldly than Xanadu programmers twice his age. In 1992, hackerdom was no longer a despised subculture. Friendly, articulate, and respectful, Jellinghaus could have been an apprentice member of any professional guild – a graphic designer, a screenwriter, a young architect. He had not yet been born when the first Xanadu designs were made, and he had received his inspiration firsthand, from primitive hypertext software, CD-ROMs and the Internet. The presence of Jellinghaus at Xanadu was a sign that Nelson’s 1965 invention was becoming common property.
Jellinghaus’s office was up in Sausalito, with the more mainstream Autodesk projects. He hadn’t been close enough to Xanadu to witness the project’s last phase of self-destruction, and he was young enough to accept the financial risk. So, after Memex licensed Xanadu, Jellinghaus and Dean Tribble tackled a long-delayed task – writing a “front end” for the system.
This need was urgent: the front end, or user interface, would show potential investors and customers what a redesigned Xanadu system could do in the service of a specific commercial task. The lack of a front end had been a perennial problem. The Xanadu philosophy had always held that if a perfect back end could be created, the front end would take care of itself. While the Xanaduers paid lip service to libertarian ideals, they imagined a more traditional revolution in which all users would be linked to a single, large, utopian system. But in their quest for a 21st-century model, they created a Byzantine maze.
“There were links, you could do versions, you could compare versions, all that was true,” Jellinghaus reports, “provided you were a rocket scientist. I mean, just the code to get a piece of text out of the Xanadu back end was something like 20 lines of very, very hairy C++, and it was not easy to use in any sense of the word. Not only was it not easy to use, it wasn’t anything even remotely resembling fast. The more I worked at it, the more pessimistic I got.”
The young programmer’s doubts were magnified by his dawning realization that a grand, centralized system was no longer the solution to anything. He had grown up with the Internet – a redundant, ever-multiplying and increasingly chaotic mass of documents. He had observed that users wanted and needed ever more clever interfaces to deal with the wealth of information, but they showed little inclination to obey the dictates of a single company. “The front end is the most important thing,” Jellinghaus slowly understood. “If you don’t have a good front end, it doesn’t matter how good the back end is. Moreover, if you do have a good front end, it doesn’t matter how bad the back end is.”
Although he sympathized with the fanaticism of his colleagues, Jellinghaus also began to question whether a hypertext revolution required the perfect preservation of all knowledge. He saw the beauty of the Xanadu dream – “How do you codify all the information in the world in a way that is infinitely scalable?” – but he suspected that human society might not benefit from a perfect technological memory. Thinking is based on selection and weeding out; remembering everything is strangely similar to forgetting everything. “Maybe most things that people do shouldn’t be remembered,” Jellinghaus says. “Maybe forgetting is good.”
Such heretical doubts did not have time to ripen, because after six months of work at Memex, the full fees owed to the Xanadu Operating Company for licensing the company’s equipment had not been paid. The programmers had expected that Smith would need some time to complete the first phase of funding for his company, but a cash crisis at Memex meant he couldn’t come through with his payments. Although Jellinghaus had the fewest financial commitments, he was also the least fanatical, and he had left a good job behind at Autodesk. After a couple of months, he began to come to his senses. “What was I doing?” he remembers saying to himself. “This is silly. This was silly all along.”
Mark Miller, who continued to invest more of his time in the project, found it harder to leave Xanadu behind. He had invested so many years in the project; even if Memex didn’t pay the bills, he could not just turn his back and walk away. One evening at the end of November 1994, a group of the programmers, with the approval of Miller and Ann Hardy, went to the Memex office and pulled the plug. They carried the machines out with them, leaving a bare space.
Roger Gregory, of course, did not participate. He was only a Xanadu shareholder, not a Memex employee, and his contact with the programmers was limited. But his response to the abduction of Xanadu machinery was wonderfully consistent with his outlook on life. “I just don’t understand,” he says. “I don’t have any sympathy for them. It is beyond my comprehension for somebody to quit just because they have not been paid for six months.”
With the computers gone, Xanadu was more than dead. It was dead and dismembered. Visitors to California Street would have seen hardly a trace. In the end, Xanadu was like a defeated rebel whose corpse is destroyed in secret so as not to become a shrine.
One rainy day last fall, I met Mark Miller in a deserted Thai restaurant near the old Xanadu offices. We sat together for several hours, listening to the faint repetitive rumbling of a bowling alley next door. Miller reiterated that when he absconded with the computers from Memex, he did not feel that he was committing a theft, since Memex was using them under an agreement with Xanadu Operating Company, whose fees for the equipment had never been fully paid.
I told Miller I understood, and we fell to talking about the World Wide Web. During the years Xanadu was at Autodesk, the graph that measures Internet growth began to go asymptotic. While Miller, Gregory, and the other coders were agonizing over their broken system, the Internet was offering a simple prototype of a universal library. Unlike the carefully guarded Xanadu code, the programming tools on which the Internet depended were open to all, and tens of thousands of users tinkered with them. In Geneva, Tim Berners-Lee, completely ignorant of the Xanadu propaganda, wrote a simple standard for hypertext publishing, which he named the World Wide Web. In Urbana-Champaign, Illinois, Marc Andreessen wrote an attractive front end for the Web, which he called Mosaic. Powered by anarchy and a passion for self-improvement, the Internet lurched toward hypertext.
I asked Miller if the Internet was accomplishing his dreams for hypertext. “What the Web is doing is easy,” Miller answered. He pointed out that the Web still lacks nearly every one of the advanced features he and his colleagues were trying to realize. There is no transclusion. There is no way to create links inside other writers’ documents. There is no way to follow all the references to a specific document. Most importantly, the World Wide Web is no friend to logic. Rather, it permits infinite redundancy and encourages maximum confusion. With Xanadu – that is, with tranclusion and freedom to link – users would have had a consistent, easily navigable forum for universal debate.
“This is really hard,” Miller said. “We made lots of progress. It was really good progress, and if the money had come through in time for the core team to have stayed with it, I have no doubts that we would have built a really amazing system that would have done all the things that had been promised in Xanadu’s name.”
Still, Miller conceded that the Web’s existence means that it must be accepted as the basis for a better form of hypertext. “We’ve got to use all of our technical insights to migrate the Web to a higher plane,” he said. Miller is pondering how to allow readers to add links to other writers’ Web pages without copying the original documents. In his view, this would be a simple step in the right direction. “It would be a little bit kludgy, but you could do it,” Miller said. “It wouldn’t take millions of dollars. I estimate something like a three- to six-man, monthlong effort.” I instinctively factored this prediction in my head, because I recognized the code in which the Xanadu programmers speak of the future: six months.
After my talk with Miller, I called Ted Nelson in Sapporo, Japan. By the time Autodesk withdrew its commitment to hypertext, Nelson had more or less run out of listeners in his homeland. The long misery of Xanadu had put him in a difficult position both financially and intellectually; among the new generation of computer hackers, the inventor was publicly tolerated and privately pitied. But in Japan, Nelson reported, he has finally found proper appreciation. “They’ve done their homework,” he said. “They understand and they listen. They care about the ideas.” Nelson added that he finds the Japanese much easier to communicate with than Americans. “It’s a matter of being totally upfront,” he theorized. “Everybody is very straightforward.”
In Japan, Nelson has been lobbying for a system of transclusion that does not depend on the Xanadu software. He has baptized this system “transcopyright.” Transcopyright is not a technology; it is Nelson’s suggestion for a contractual solution to copyright problems. Nelson argues that electronic publishers should allow anybody to republish their materials, provided that republication takes place by means of a pointer to the original document or fragment. Just as in Nelson’s imaginary Xanadu franchises, publishers of transcopyrighted documents would receive a payment every time one of their bytes was accessed.
In his description of transcopyright, the inventor admits that “certain unusual software features are required” to implement his system, including a back end that can bill users for small amounts of materials, and a front end capable of automatically editing and presenting documents that may have been purchased from several sources. In fact, transcopying is similar to Xanadu, but without the machinery. Nelson has reduced his contribution to a name and a description. But for Nelson, names and descriptions have always been the heart of the matter. On the short paper that presents his transcopyright idea, Nelson identifies himself as “Founder of Interactive Media,” and “Founder of Network Publishing.”
Nelson, who recently moved to Japan, sleeps in the afternoon and works all night, so I called him at 5 a.m. Sapporo time. Along with lobbying for transcopyright, Nelson is working with Professor Yuzuru Tanaka, a computer scientist who is developing a simple programming language. With Tanaka, Nelson is attempting to implement the earliest hypertext design, his 1965 proposal for a personal writing system that allows screen editing with transclusion and links.
Nelson’s response to the Web was “nice try.” He said it is a trivial simplification of his hypertext ideas, though cleverly implemented. And he has not entirely given up hope for the old Xanadu code. “I’d like to stress that everyone involved in Xanadu believes that the software is valid and can be finished,” he asserted.
“It will be finished,” Nelson added. “The only question is which decade.”
Roger Gregory, along with fellow hacker Keith Henson, has frozen numerous people. Both Henson and Gregory hope that each of these frozen people will be resurrected sometime in the future, when medical science has produced a cure for their ailments.
At the beginning of 1994, Henson ran into Memex founder Charlie Smith, whom he had known for some time, at a New Year’s Eve party. Henson asked his old acquaintance how things were going, and Smith answered that things were going poorly. He arrived at his office on November 30 to demonstrate Memex to a prospective funder only to find that all his machines were gone. His fledgling business, desperately in need of outside investment, seemed to have walked out in the middle of the night.
Henson, a Xanadu investor, guessed out loud that neither Gregory nor Nelson knew the machines had been taken. He was right. He contacted them both, and eventually helped move the whole project in a new direction. After Smith made clear that a lawsuit would be the likely result if they failed to return the machines, Miller and his colleagues finally abandoned their code, their computers, and their hopes for Xanadu’s eventual release. They returned the machines to Memex. And after the remnants of the Autodesk team scattered, Charlie Smith hired Roger Gregory to finish the Xanadu software, which ran to 300,000 lines of code.
Last fall, I visited Gregory in Palo Alto. His flawed directions to his home brought me to a dead end against the lawn of a Palo Alto research park. The lawn was perfectly trimmed, and it seemed to express the efficiency with which the unruly inspirations of myriad scientists are disciplined by corporate enterprise and turned into profit.
Roger Gregory, by all accounts, is incapable of such discipline. This was lucky for me, because it made his house conspicuous. At the end of the street, one front yard was decorated with at least a dozen disassembled bicycles. Peeking through a flimsy curtain of plywood were three decades of computer machinery: platters, disk drives, metal casings, fans, and layers of motherboards whose antique architectures looked as if they contained, in some quaint, unreadable form, an entire history of the digital era.
Gregory answered the door when I knocked. His dirty blue pants were unbuttoned and he was barefoot. A long-sleeve pinkish T-shirt dropped over his rounded midsection but stopped before it reached his pants.
From the outside, Gregory’s house looked like it was spilling its guts. Inside, the perspective was reversed. The back room looked out through a glass door onto a patio, where more disassembled bicycles were intertwined. Computers cluttered the entrance way, and rows of tattered books crowded the walls from floor to ceiling.
With hundreds of other pieces of stuff – children’s toys, random tools, racks of hardware, and tangles of wire – trailing across every surface, across the floor, and continuing out in the back yard, Gregory’s home seemed as if it had volunteered itself to the world as a sanctuary for discarded but possibly useful objects. A television mounted above Gregory’s computer was tuned to a shopping channel.
We spoke for some time about Xanadu. Gregory railed with unrestrained bitterness against the programmers and managers who controlled his project over the years. “Is Marc Stiegler still alive?” he marveled. “That’s too bad.” Gregory described trying to build Xanadu under the old management as being like “repairing a watch on top of a tree house in a gale. The gale was the wind of constant redesign and change.”
Quietly, while Gregory recited his complaints, I scanned the programmer’s walls. His bookshelves were overwhelming. So much anxiety was collected there: dust-covered and unread books, books piled behind other books, redundant editions and bookstore rejects; along with 20,000 other volumes, Gregory owns five complete sets of the Encyclopedia Britannica. Over his bed were the elegant codices of the Britannica’s Eleventh Edition, famous for its learned essays on every topic.
I stared hard at Gregory’s books. A hundred years ago, with only the products of the printing press to aid them, the Britannica encyclopedists built a collection of information that, while incomplete, convincingly pointed the way toward total knowledge. Today, with the advent of far more powerful memory devices, Xanadu, the grandest encyclopedic project of our era, seemed not only a failure but an actual symptom of madness. In Gregory’s house, this contradiction was evident: the arch-hacker had built himself a barricade of books, a paper dike against a flood of sorrow.
The very first story Ted Nelson told me was based on a vision of water disturbed. To Nelson, the swirling currents under his grandfather’s boat represented the chaotic transformation of all relationships and the irrecoverable decay associated with the flow of time. His Xanadu project was meant to organize this chaos, to channel this flow. Sitting with Roger Gregory, surrounded by yellowing walls of literary authority, I remembered that Xanadu’s programmers never solved the basic problem of computer performance. No matter how powerful their machines, or how elegant their code, there had always been too much data to move in and out of memory.
During the same month I interviewed Gregory at home, I also went to see him at his work. On the door of his office at Memex was a sign that said: “S.T. Coleridge and Sons. Danger.” The rooms inside were small, hardly bigger than booths at a trade fair. On a white board, in orange pen, was the admonition: “This is commerce, this is not your hobby.” A corner of the room contained a giant bag of lemons and a sleeping bag.
I had come for an interview in the late morning, but Gregory had not yet appeared. Keith Henson, wearing Birkenstocks, blue jeans, a black shirt, and a black tie, welcomed me in. At about noon, Gregory arrived. The loud hum of the machines inside his small office proved too much for my meager tape recorder, forcing us out into the parking lot. We sat an hour between rain storms in the front seat of Henson’s automobile. The low, dark roof of the car made it feel like a coffin. Gregory’s T-shirt was covered with fine, reddish dust – a trace, I guessed, of a losing battle with his electric razor. Gregory admitted being groggy – he stayed up late the night before wrestling with the remains of Xanadu.
The hacker’s voice was very quiet. He was holding his head in his hands, and for much of the hour, he spoke without shifting position. His tall, soft body was curled, his knees were bent to his chest, and his heels were planted on the edge of the driver’s seat. Gregory’s upper cheeks and forehead rested against his palms, as if in a slow-motion demo of The Scream. We scrolled back through every phase of the project, reviewing Xanadu’s cheerful beginnings in Ann Arbor, its half-dozen deaths, the euphoria of the Autodesk investment, the heartbreak of the spin-out, the bitter personal battle for control of the company, and the present-day pathos of Gregory’s single-handed attempt to salvage, somehow, the endless, incomprehensible, lines of code.
Without protest, without calculation, Gregory answered every question. “I don’t know,” he said, in answer to a query that is inaudible on my tape and that I can’t remember. “I’ve struggled.” A few times, Gregory paused and took several deep breaths, but he always resumed, determined to make his point. He talked with the air of a person who despairs of justice in this lifetime but counts on the vindication of history.
Finally, we made it to the end of the tale. There was only one remaining question, and it seemed both obvious and cruel. His project had promised an end to forgetting, but in the end, only Gregory had been unable to forget. The deluge of information had arrived. The other programmers had drifted away. Only Gregory remained with his fingers on the broken Xanadu machine.
“Why?” I asked.
“Total insanity,” Gregory answered, both hands squeezing his face.
Wired, Issue 3.06, June 1995