It looks like I will have an unexpected day free tomorrow, and perhaps part of the next day. I’m interested in suggestions from kindred spirits about things to take a look at related to planning, public transportation, sensors, education, and science. (Already in mind: going out to the University of the Philippines; I’d like to stop by the library at the School of Urban and Regional Planning and see what they have.)

IFTF hosted the second Quantified Self Show&Tell, and since then they’ve been curious about and supportive of this extended investigation into the meaning of what more academic observers call “personal informatics.” Recently, Bradley Kreit interviewed me about the implications of The Quantified Self for the IFTF Health Horizons Report. His interview and excellent editing helped me express what I think is happening in a fairly concise way. I’ve republished it with Bradley’s permission below.

—

IFTF Health Horizons

Gary Wolf is a contributing editor at Wired magazine and the co-host of The Quantified Self, a blog dedicated to self-knowledge through numbers (www.quantifiedself.org). At Wired, he has been the author of a number of the magazine’s most frequently cited articles, including “The Curse of Xanadu,” about Ted Holmes Nelson and the invention of hypertext; “The World According to Woz,” about Apple co-founder Steve Wozniak; and “The Wisdom of St. Marshall, Holy Fool,” about Marshall McLuhan. He has also written about Piotr Wozniak, creator of the memory program SuperMemo, and recently about Craigslist and its founder, Craig Newmark.

IFTF: The phenomenon of the quantified self is an early form of personal health forecasting. What is the idea behind it?

GW: Numbers play a key role in analyzing all kinds of phenomena, from the largest phenomena of the cosmos using radio telescopes to the smallest phenomena in the universe—the analysis, say, of subatomic particles. We have statistical tools of great sophistication for gathering data and finding meaning in it. It seems only natural that we would want to use some of these techniques to gain knowledge about ourselves.

This is so obvious that it might almost seem trivial, except when you realize that we usually associate self-knowledge not with numbers but with words—a kind of inner voice of consciousness and conscience. I think that supplementing that with quantitative tools is one of the most interesting trends emerging in our culture today. This interest is based on the highly practical results of experiments that people are doing in collaborative diagnosis and collaborative evaluation of treatments for chronic conditions, as well as experiments that involve the analysis and acceleration of learning.

IFTF: In some of your writing about the quantified self, you’ve talked about a concept called a macroscope. What do you mean by that, particularly as it relates to health?

GW: The word macroscope has been used quite a few times in quite a few contexts. It’s an interesting word; its meaning is trying to emerge and everyone’s taking a crack at it, but it’s finally settling down into a useful concept.

My meaning is taken from Jesse Ausubel, a climate scientist who is also a professor at The Rockefeller University. It simply refers to gathering data in nature through distributed methods, often through sensor networks, and then analyzing it on a computer. The particular pieces of technology for gathering this data are familiar; it is how they are now being combined that is interesting. We are beginning to see them being used in the context of a social process that produces data that would be inaccessible to an individual researcher trying to build this network from scratch.

The macroscope concept can be applied to the many individuals keeping track of some aspect or aspects of their lives. You have people tracking sleep, diet, exercise, productivity, symptoms, and so on. With all this tracking, a tremendous amount of health-related data is being produced. When that data is analyzed, you learn things that would be much harder to learn using the traditional methods of a clinical trial or a population study.

IFTF: Do you expect self-tracking will become widespread over the next ten years?

GW: I think it will become a mainstream, almost ubiquitous practice and at the same time will become invisible because it will be blend in with daily life. I think a good comparison is with the fate of computing. At one time, the people who used computers tended to be the kind of people who liked it. Over time, the process of computing has been incorporated into so many technologies and devices that many of the things we do that involve computing don’t seem like computing at all. Think of using a pedometer or step counter, or standing on a digital scale. The computing component is disappearing, and the self-tracking aspect will, too.

Self-tracking will disappear because it will be taken for granted. The quantitative tools in our lives will produce data that will be incorporated into some feedback mechanism; we will look at those mechanisms and they will influence us in some way. For instance, we will get biometric data in the form of feedback about how well we’re eating and sleeping, but we won’t have to peel back that information and do the analysis ourselves. Of course, the people who will be making these products and services will be highly aware of their tracking components, but if they’re successful, users won’t think about those aspects.

IFTF: Do you foresee any difficulties with privacy or concerns over control of information? Will individuals not want to share the detailed and intimate information that will be collected about them?

GW: Although gathering personal data will become mainstream, I don’t think most people will want to share their data. We can identify some people as sharer types with respect to their health and biometric data; they are closely linked to the pioneer type because they have a vision of what sharing may bring. But for the most part I think the benefits of the macroscope will be very hard to achieve under a system in which people can be punished harshly on the basis of their numbers. And we live in a world where if you have bad numbers, you will be punished.

IFTF: Isn’t one of the core challenges that the data is most useful in large-scale aggregations, but to get that you have to be able to get people to share their data?

GW: Let’s back up a bit: useful to whom? The data is very useful to you, whether or not it’s aggregated. You can see the macroscope as having multiple guises: there’s the social macroscope, which aggregates data across individuals, and that’s where the privacy issues come in, but you can also interpret the macroscope on an individual level. I can have multiple sensors at multiple times, all aggregating the data for me; I can do experiments of one, and the data never has to leave my computer.

IFTF: So how do you bridge that gap to make the social macroscope feasible?

GW: We need to articulate as clearly as possible that there must be a transformation in terms of how we look at what health and health care mean. As long as health care is considered from the perspective of the individual, there are many benefits that we’ll be missing.

Kandel remembers being dubious. He knew full well that only women could have babies. But at the same time he felt a certain anxiety. What would his mother think if he became pregnant?

That worry and Mitzi’s change of mood ended my first sexual encounter. But Mitzi continued thereafter to speak freely to me about her sexual yearnings and said that she might have realized them with me were I older.

Mitzi did not, as it turned out, remain celibate until I reached her age qualifications. Several weeks after our brief rendezvous in my bed, she took up with a gas repairman who came by to fix our stove. A month or two later, she ran off with him to Czechoslovakia. For many years thereafter, I thought that running off to Czechoslovakia was the equivalent of devoting one’s life to the happy pursuit of sensuality.

Kandel describes this lucky moment as one of his fondest early memories, and he also describes it as typically Viennese.

That erotic experience was right out of one of Arthur Schnitzler’s short stories, wherein a young, middle-class Viennese adolescent is introduced to sexuality by ein susses Madchen, a sweet young maiden, either a servant in the house or a working girl outside the house. Andrea Lee, writing in The New Yorker, has said that one of the criteria bourgeois families in Austria-Hungary used in selecting girls for housework was that they be suitable to relieve the family’s adolescent boys of their virginity, in part to entice them away from any possible attraction to homosexuality.

I just got around to reading Kandel’s memoir recently, and this anecdote caught my attention and refused to release it. Vienna is a city I know only from books. But this was a Vienna I did not know, even from books. The repression and ambivalence, the neurosis and hypocrisy detailed by Freud and his biographers (for instance, and with special relish, by Peter Gay), the self-conscious amorality of the rakes in Schnitzler’s stories, and the terrible fear of venereal disease that Stefan Zweig describes in The World of Yesterday, his own memoir of a Viennese childhood two generations earlier, is here replaced by an innocent and even rational sexuality, in which the libido – at least of a male child – is influenced by a sweet girl who, though she tells fairy tales, is paid to be gentle and accessible.

Of course Kandel was only 8 years old; he was not an adolescent. The catastrophe of the Anschluss caused his family to flee Vienna in 1938. This prevented him from growing up into that rich, contradictory Jewish Viennese culture that the Nazis erased so brutally that the preservation of its myths – even an idiosyncratic and personal one like the one Kandel gives here – is inherently interesting.

Still, one might ask: is it true? Since Kandel offers a reference to a New Yorker article by Andrea Lee, I decided to read it. The story is called La Ragazza, and it appeared in the issue for February 16, 2004. It is a beautiful story. It is not a piece of journalism or a biographical sketch, but a work of fiction, and it takes place, not in Austria-Hungary but in Turin. Narrated in the third person, it sticks closely to the point of view of an unreliable rake from Padua named Orso who likes to irritate his girlfriends with the story of the “the first cunt he ever saw.” The cunt in question belonged to a housemaid named Ida. Orso was twelve at the time:

Ida, without underpants, perched precariously, legs askew, on the edge of the kitchen table, as Orso’s brother, Remo, a year older, declaimed in a pompous, pedagogic tone: “Questa, caro mio, è la fica”—“This, Orso, my boy, is the pussy.” Remo couldn’t actually have had a schoolmaster’s pointer, yet he was indicating with enough formality to suggest one: labia majora, labia minora, mons pubis, clitoris. Was he consulting a medical dictionary at the same time? It was possible. And leaning back on her elbows, giggling shameless encouragement in her singsong Friulano accent, was beautiful, brainless Ida, tall and blond and long-necked, with a head that looked as small as a goose’s.

Describing the scene to his lovers over the years, Orso has romanticized what he saw between Ida’s legs as a rose, pink-lipped and crimson in its depths, and has added a swirling frame of old-fashioned petticoats—when in fact the girl wore a coverall of postwar cut that squashed her thighs grotesquely when pulled up. What he really thought it looked like was a sea creature—edged with pale moss or cilia and exuding a mollusk’s imperturbable smugness. An impression that was hardly dispelled a few evenings later when, much to Remo’s chagrin, Orso was the one pulled down onto Ida’s small hard bed, after she had invited him to her room to deliver some old copies of Corriere dei Piccoli, the children’s weekly she used to read with her lips moving after washing the dinner dishes.

This scene, with its mixture of sex, science, sibling rivalry, and class condescension, is presented with somewhat more irony than in Kandel’s recollection. Orso’s attempt to irritate women by recounting it is invariably successful:

“Poor thing.” This was the comment of Anne, Orso’s American second wife, the mother of his children, who was quick to take up the cause of any woman against him.

“What do you mean, ‘poor thing’? It was all Ida’s idea to start with. She was eighteen or nineteen—and no virgin. A good housekeeper and a great cook, too. Nobody could make knoederli like hers. And she ended up fine. Married a carabiniere and came to my mother’s funeral in a mink coat.”

“Poor thing.” This was Bettina, Orso’s great love, who, throughout their six-year affair, stayed married to his business partner, Grellio. “I bet she didn’t really have petticoats like a cancan dancer. Sounds too much like Belle Epoque pornography to me. And why did your mother hire a slut like that?”

“She was a brilliant laundress. Could get through my mother’s entire trousseau of linen and hemp sheets and my father’s shirts in a single day. And I think they expected her to relieve Remo and me of our virginity. That’s what bourgeois families did in those days. So we wouldn’t end up homosexual.”

“Poor thing.” This was Sveva, the twenty-two-year-old assistant accountant in Orso’s office, with whom he occasionally sneaks off for a weekend. “So typical of men of your generation. You’ve all got a proto-Fascist nineteenth-century patriarchal mind-set. You’ve made the victim into an accomplice to quiet your sense of guilt.”

Now, indeed, we are back in the world of Schnitzler. Not the charming, anecdotal Schnitzler, but the Schnitzler of self-interested rationalization, rascality, even cruelty, all wrapped in thin blanket of cultural knowingness that does not conceal its crude outline. Some of the depth of Andrea Lee’s story comes from these echoes – conscious or unconscious, though I suspect the former – of Schnitzler, even to the point of “poor thing.” This is a phrase used in the most condescending way in Schnitzler’s famous play, The Affairs of Anatol. Anatol is here talking with is friend Max about the women he has had sex with:

ANATOL: I had a fine idea of myself in those days. I used to catch myself thinking . . . Poor child, poor child!

MAX. Poor . . . ?

ANATOL. When I was very young indeed I saw myself as one of the world’s great heroes of romance. These women, I thought . . . I pluck them, crush the sweetness from them . . . it’s the law of nature . . . then I throw them aside as I pass on.

Behind Schnitzler’s Anatol, behind Andrea Lee’s Orso, there is the great icon of sexual narcissism, unnamed but unmistakable. Casanova also had his first sex as a child. It was also in Padua, and also with an older girl. She was fourteen, Casanova was eleven. She had the duty of dressing his hair.

One morning she came to me as I was in bed and brought me a pair of white stockings of her own knitting. After dressing my hair, she asked my permission to try the stockings on herself, in order to correct any deficiency in the other pairs she intended to knit for me. The doctor had gone out to say his mass. As she was putting on the stocking, she remarked that my legs were not clean, and without any more ado she immediately began to wash them. I would have been ashamed to let her see my bashfulness; I let her do as she liked, not foreseeing what would happen. Bettina, seated on my bed, carried too far her love for cleanliness, and her curiosity caused me such intense voluptuousness that the feeling did not stop until it could be carried no further. Having recovered my calm, I bethought myself that I was guilty and begged her forgiveness. She did not expect this, and, after considering for a few moments, she told me kindly that the fault was entirely her own, but that she never would again be guilty of it. And she went out of the room, leaving me to my own thoughts.

But where Orso is a creep, and Anatol is ridiculous, Casanova in his memoirs is a realist, nearly a psychologist by temperament, who respects self-interest when he finds it expressed openly and deals with hypocrites poetically by deceiving them. Casanova displays a guiltless sexual curiosity that can only viewed by a Jewish Viennese doctor – which is what Schnitzler was, before he dropped medicine for fiction – with a kind of amazement. Schnitzler wrote a long story about Casanova in 1918, a tale in which the hero commits horrible acts with a bravado and savoir-faire available only to those incapable of remorse. After compelling a man to sell his mistress and then stabbing him to death in a duel, Casanova sleeps for days, bothered by little more than a vague feeling of bitterness and pronounced physical exhaustion.

How can this be possible? Shouldn’t inner contradictions – between Casanova’s sense of personal honor, say, and the memory that he has abused a woman who did nothing to him except wound his vanity – result in an uncontrollable anxiety? But Casanova is a master of oblivion. He does not erase the content of past incidents from his consciousness – his recollections are vivid – but instead he artfully preserves elements consistent with his self-esteem while softening what would cause him too much pain.

Willful forgetting, intentional non-knowing; these remain topics of controversy in philosophy and psychology. Usually, forgetting the past is understood as weakness, especially in old Vienna:

Dormant unconscious conflicts, revived by aroused appetites, make the course of sexual conquest, like that of true love, anything but smooth. Even the middle-class adolescent’s first fumbling experiments – his initiation by a complaisant servant at home or a venal waitress in town – is, in real life, a nest of ambivalences and rationalizations, of confused arousals and bouts of panic.”

–Peter Gay, Education of the Senses: The Bourgeois Experience, Victoria to Freud

If there’s one thing Casanova is spared, it’s panic.

Erik Kandel, among his many interesting research topics, has studied panic. Panic is an easy thing to induce in a mouse: you turn on a bright light, remove all routes of escape, and give it repeated shocks. Soon the creature will be frozen in a corner. A few years ago, Kandel and his colleagues decided to try and understand the mechanism by which learned panic occurs, using the mouse as a model animal. They succeeded. Learned panic occurs through circuits that link the thalamus and the amygdala, and Kandel and his colleagues traced these circuits and described the biochemical processes at work there. They also discovered that the flip side of learned fear – learned safety – implicates a distinct region of the brain, called the striatum, that plays no role at all in learned fear. These discoveries, and parallel ones in other labs, hold out the hope for better drugs to replace the addictive benzodiazepines like Valium. And there’s another benefit. When learned fear is understood biologically, an entire mythical structure of morality becomes, if not untrue exactly, at least obsolete. The outsized neuroses of the sensitive Viennese youth, an echo chamber of past punishments and undeserved shocks, can be cured directly, without having to be brought laboriously into consciousness. We can all be Casanovas, then.

But wasn’t Casanova a monster? The snobbish rake of the Viennese tradition, the Orso or the Anatol, is at best an ass, and at worst a fiend. But the predatory side of his character is linked to the deficits of his prey; to their poverty, insecurity, naiveté, their fear. A world of Casanovas cannot be a world of victims. The morality of Casanova is utopian; in its ethic of guiltless curiosity and implicit equality it resembles science.

Do you know anything about Qoogle? I am not linking to the site directly, because I don’t know what they are up to, and though I suspect it is just a link-farming scheme, it could be something more nefarious. But here are some screenshots.

I searched this morning to check the online commentary about Ryan Sorba, who made some news with his anti-gay rant at the recent CPAC conference.(Andrew Sullivan has an account here.) Here is a screenshot of the first page of results on Google. The text on this image is hard to read, but the third link (pretty good, Mr. Black Hat!) is a web page called Ryan Sorba, and below it is a snippet that contains some “word salad”: Feb 19, 2010 … This brueghel is an ryan sorba of the gipsywort disentangled by jawless oxidization sardinian in the solon, cryogen, and dreaming of …

I’ve been getting this sort of thing quite often when searching on names. If you click on the link, you get to a fake Google page at Qooglesearch.com, showing results for Ryan Sorba that appear to be scrapped from Google, along with two “sponsored links” from “flashbuzz.net” at the top. I assumed that these sponsored links were the payload, but on subsequent clicks they have disappeared. Another link associated with Qoogle appears to be imasion-corp.com. There is an intermediate link that appears also: tdss… – but it is hard to catch and I didn’t get it.

Below is an image of the Qooglesearch page.

Sort of convincing as a Google page, if you aren’t paying too much attention.

But the best image comes from clicking on the “cached” link under the Qoogle “Ryan Sorba” entry on the original Google search page. That shows you what Qoogle is showing the Google Bot as it surfs the web. Colorful!

Well, it may be just another case of “pissing in the pool” as Craigslist CEO Jim Buckmaster calls it. But I’d like to know more about Qoogle, and if you have any tips, please send them along.

Two years ago I traveled to Costa Rica and met two amazing scientists: Daniel Janzen and Willie Hallwachs. Janzen and Hallwachs and their many colleagues – professional scientists, students, and an important group of “parataxonomist” collectors – work the Area de Conservación de Guanacaste, where they catalog and study the organisms that live there, while trying to protect and expand the park.

Automeris zugana?

A few weeks later I went to Ontario to meet Paul Hebert, a geneticist at the University of Guelph and the inventor of a novel method for identifying species using very short snippets of DNA. Hebert was using material – moths and wasps, mainly – collected in Guanacaste to prove that his idea was valid.

When I wrote about it, this method was highly controversial. Could you really take part of the job of the expert taxonomist – the identification of species – and routinize it, reduce the cost, and provide identifications of even very hard to tell apart animals as a kind of standard technical service? Or, as Dan Janzen liked to ask: could we build a species tricorder?

A couple of months ago I found out that my story, A Simple Plan to ID Every Creature on Earth, is going to be awarded the 2009 AAAS Kavli Science Journalism Award for the best science story published in a magazine during the preceding year. I’m very proud of this award, and grateful to Dan, Winnie, Paul, and their colleagues for having been willing to answer endless questions about their work. Critics of barcoding – also eminent scientists, including the Director of the Jepson Herbarium Brent Mishler - were equally open and helpful.

I took many photos on my trip, and I thought I would post a few of them along with some updated barcoding material before I head down to San Diego for the AAAS meeting.

Here is a photo of Dan at his desk. Below it is the description of him working that opened my piece for Wired.

The utopian lepidopterist holds a pin in each hand. His style is ambidextrous and probably unique. He catches two forewings of a dead moth simultaneously and pins them to his drying board, and then, in a continuous sweep, he does the same with the hind wings. He repeats these motions again and again, like a conductor with tiny batons. Outside, it is hot and bright. Inside, it is hot and dark. The lepidopterist, whose name is Dan Janzen, has been working here in this Costa Rican forest for more than 40 years. He is married to his research partner, Winnie Hallwachs, and the two of them occupy a small house with a roof of corrugated metal whose eaves cast deep shade. During the day they work under artificial light. At night bats flit through the gaps at the top of the wall, do hairpin turns in the air, and exit again without slowing. The utopian lepidopterist’s aim is to put names on all the moths and butterflies in the forest. He wants to know more than just the names, of course; he wants to know who lives where and who eats whom and to unravel the mysteries of the ecosystem. But his first question is always the most basic one. This moth, here on the drying board: What is it called?

Here is one of the rearing stations Janzen and his colleagues operate. Each bag contains a caterpillar and some of the leaves of the plant it was found on.

[The area is...] strung with ropes. Beneath the ropes hang hundreds of plastic bags full of leaves, and inside every bag there is a caterpillar, a pupa, a moth, or some flies or wasps that have managed to parasitize the caterpillar, eat the pupa, and emerge into the middle of this scientific experiment. Like the insects in the neighboring bags, the destiny of these parasites is to be frozen, dried, identified, barcoded, and shipped to a museum for reference. Here, and in 10 other caterpillar-rearing stations in the forest, Janzen, Hallwachs, and their many local collaborators have solved taxonomic mysteries that go back hundreds of years. “Some of these moths have had names forever, and their caterpillars, too, and they’ve never been recognized as the same species,” Janzen says.

Janzen is 71. I enjoyed grabbing this picture of him on the road the rearing station.

There was electricity available at the rearing station, so I got to sit outside and catch up on my notes….

What has happened after the story appeared? Was the controversy resolved? Over the last two years, the advocates of barcoding appear to have won the day. I keep up with ongoing news via Mark Stoeckle’s detailed posts on The Barcoding Life Blog. Recently, reported on a project to use barcoding to determine the species of fly larvae that are important in forensics; although the larvae which feed on corpses can be telling evidence of the time of death, figuring out what species you have can be hard when all you can see is a tiny wriggling worm, with virtually no distinguishing features.

Barcoding also made news recently in Science, with a report by Elizabeth Pennisi on the recent completion of a barcoding analysis of a collection of more than 2500 parasitoid wasps. (The image below is from Dan and Winnie’s collection of material, it shows the actual wasp larvae (Euplectrus walteri) emerging from the caterpillar – yum!)

Pennisi, in her story, The Little Wasp That Could, (full text behind a subscription wall, unfortunately) gives an excellent description of why figuring out what wasp you have on your hands is interesting, important, and – before barcoding – very hard:

The British geneticist J. B. S. Haldane once famously quipped that God seems to have had an inordinate fondness for beetles, given their numbers and diversity. If so, then he must have been besotted by parasitoid wasps. Tinkerbells of the animal kingdom, many of these insects are no bigger than fleas, yet they may well outnumber beetles.

Unlike beetles, however, parasitoid wasps aren’t exactly charismatic. “You get one in your eye and pull it out with your finger and think it’s a piece of dust,” says Daniel Janzen, an ecologist at the University of Pennsylvania. “There’s millions of individuals out there, and you don’t even know they exist.” Yet these inconspicuous insects play a crucial role in natural ecosystems and in agriculture. They destroy the eggs, larvae, or cocoons of countless species of insects and arthropods, sometimes with hugely beneficial effects: The U.S. Department of Agriculture (USDA) estimates that parasitic wasps save the United States at least $20 billion annually by controlling invasive species. “I think very few people realize what a force they are in the biology of our planet,” says Michael Strand, an entomologist at the University of Georgia, Athens.

A 2008 DNA bar-coding analysis of 2597 parasitoid wasps from his collection turned up 313 species, not the 171 researchers had previously thought. M. Alex Smith of the University of Guelph in Canada and Josephine Rodriguez of UC Santa Barbara discovered that what was believed to be a single species—a 2-millimeter-long wasp called Apanteles leucostigmus with a black body and a white rhomboid patch on its wing—proved to be 36. And there were many more examples of previously unrecognized species, Janzen, Rodriguez, Smith, and their colleagues reported in the 26 August 2008 issue of the Proceedings of the National Academy of Sciences.

Alex Smith, mentioned in the above passage, was on one of the trips I made to Costa Rica. I enjoyed his company tremendously. Here is a picture of him with a beetle on his head. He did not put it there. We were collecting moths at night, and just saw a nice spot and decided to land.

Finally, here a couple of good videos that capture some of the interesting stuff going on in this line of work. The first is a short video about parasitoid wasps; it is very well explained and there are a few great images.

But this one, a Google talk by Dan Janzen and Paul Hebert, is more informative and inspirational. Here Janzen and Hebert lay out the whole picture of what it would mean to be able to identify species on the planet, and how we could get there.

There is an excellent collection of links here, at the Ontario Genomics Institute, with more information about the relationship between DNA barcoding, species identification, and environmental conservation.

Stumbled upon the chanterelle motherload yesterday – one day late. They were waterlogged and hopeless after a solid night of rain. These are California chanterelles, which got proper acknowledgment as a distinct species only in 2008,when David Arora and Susie Dunham published an article in Economic Botany giving it a name of its own: Cantherellus californicus.

I have always wanted to write a long profile of Arora, one of the most independent thinkers I have ever met. His analysis of the relationship between ecology, conservation, and commercial use of natural resources changed my way of thinking about nature.

In a story I wrote about the commercial mushroom harvest in the Yukon territory, I had a chance to spend a week with him in the far north. I got a few paragraphs about him into the story, but most of what we talked about is still sitting in notebooks, waiting for the opportunity to become part of a story. His house in Northern California is very inspiring and strange, with a pond on the inside where he grows algae, and a forest outside that he manages to encourage the appearance of hundreds of fat Porcini when the time is right.

Arora’s giant book, Mushrooms Demystified, is used by everybody on the west coast with an interest in mushrooms. My copy is falling apart and indispensable. He has never updated it. The reasons are complicated, and related to his shifting views of the relationship between wild nature and the humans who are part of it.

But that’s a story for another day…

The story in this month’s Wired started when the magazine’s editors asked me a pointed question: how can a site that’s so good be so bad? Serving a vast community at an irresistible price (mostly free), craigslist nonetheless seemed the antithesis of what a modern web business should be. Oblivious to innovation and stuck in a 1997 mindset, craigslist was hogging the sector and holding things back. When the editors invited me in to propose my writing the story, they wanted an exposé.

Meanwhile, I had been wanting write about craigslist for years, ever since I saw founder Craig Newmark and CEO Jim Buckmaster give their now famous talk at the 2004  San Francisco Web2.0 conference.  (The full recording can be heard here.) Newmark stood on a milk crate, which put him nearly eye-to-eye with Buckmaster. They described their philosophy of branding (against it), of graphic design (against it), and, most intriguingly, of money. Money they were not against. But they were not exactly in favor of it, either. They seemed to think of money as a danger. Their extreme, almost theatrical caution in relation to cash was familiar to me. Many writers, musicians, and artists have it. But these were not writers, musicians or artists, they were two men running a classified advertising site. As craigslist grew dominant, its managers’ profession of disinterest in profit came to seem more and more anomalous. I did not assume they were hypocritical. I assumed they were interesting.

When we first discussed the story, the vehemence of the editors’ point of view caught me off guard. I use craigslist. The couch in my living room is from craigslist. I got rid of my moving boxes on craigslist. On the other hand, somebody tried to rob me once when I met them to purchase a DVD player they had listed on craigslist. I find searching the site to be absurdly laborious. I could see the points the editors were making against craigslist. Nonetheless I didn’t hate it. I didn’t know anybody who hated it. The most negative response I’d ever seen to craigslist  was a shoulder shrug and an eye roll that meant: “Hey, it’s free; what do you expect?”

But as I began to talk to people inside and outside the company, I began to understand that while the intensity of feeling against craigslist was a minority view, it was helpful to think about, because it came from a rare appreciation of the company’s power. To a user craigslist appears to be a local listings site. But behind each page of blue links is a publishing system that serves more than 45 million people each month, and produces 100 million dollars in revenue annually. More importantly, craigslist’s success has helped take down an entire sector of the publishing industry. The anger against Craigslist, an anger that I thought deserved to be expressed with some irony, was nonetheless  an honest tribute both to its significance and to the hopes and expectations it provokes.

In the end, after a couple of months of research, I told the editors I thought I was prepared to answer the question they originally asked: “Why can’t craigslist be better?” The answer exposed some normally unchallenged assumptions about what better means. In the series of posts that should go up over the next couple of weeks, I’m going to go into some of the details that didn’t make it into the magazine version of the story, details that may interest people who find themselves fascinated, as I was, with one of the most unusual businesses in the world.

[Update: I posted two follow-up pieces on Wired's Epicenter blog: Craigslist vs. eBay, and Bad Advice for Newspapers]

Watch this video published from a story in New Scientist by Lars Hall and Petter Johansson.

[I]n an early study we showed our volunteers pairs of pictures of faces and asked them to choose the most attractive. In some trials, immediately after they made their choice, we asked people to explain the reasons behind their choices.

Unknown to them, we sometimes used a double-card magic trick to covertly exchange one face for the other so they ended up with the face they did not choose. Common sense dictates that all of us would notice such a big change in the outcome of a choice. But the result showed that in 75 per cent of the trials our participants were blind to the mismatch, even offering “reasons” for their “choice”.

This is troubling enough, but there’s more. When people are fooled into thinking they made a different choice than the one they actually made, and then articulate their “reasons” for this supposed choice, they then may actually change their future preferences to conform to their confabulated preference.

Importantly, the effects of choice blindness go beyond snap judgments. Depending on what our volunteers say in response to the mismatched outcomes of choices (whether they give short or long explanations, give numerical rating or labeling, and so on) we found this interaction could change their future preferences to the extent that they come to prefer the previously rejected alternative. This gives us a rare glimpse into the complicated dynamics of self-feedback (“I chose this, I publicly said so, therefore I must like it”), which we suspect lies behind the formation of many everyday preferences.

Lars Hall and Petter Johansson lead the Choice Blindness Laboratory at Lund University, Sweden. At the end of their New Scientist piece, they suggest that learning about this experiment should make people better at understanding their own choices.

In everyday decision-making we do see ourselves as connoisseurs of our selves, but like the wine buff or art critic, we often overstate what we know. The good news is that this form of decision snobbery should not be too difficult to treat. Indeed, after reading this article you might already be cured.

Unfortunately, this is not convincing. It is common for biases persist even when we are warned about them. I suspect we are in no position to stand guard over our judgments without the help of machines to keep us steady. Assuming, that is, that deliberative consistency is a value we care to protect.

The number of Web applications to collect data and information about yourself continues to grow; if you want to track something, most likely there’s an online tool to do it. This is great – especially since a lot of the applications seem to have a lot of users, which means an interest in data… However, as users, developers, and designers, we shouldn’t be satisfied too quickly with what we have. Want more. Demand more. It’s interesting and oftentimes fun to log data about your life – whether it be when you go the bathroom, your sugar levels, or your mood. You get some nice graphs and charts, it looks cool, and maybe you learn something about yourself.

But all the self-surveillance tools so far are mostly about a single dataset or two at most. You track your weight and what you eat, but it’s more complex than that. Life is complicated and data is an abstraction of life after all. Do you eat when you’re depressed or are you depressed when you eat? Do you feel better if you exercise? What about sleep? How much sleep and exercise is best for you? What days should you exericse and how many days in a row and for how long? What truly makes you happy? I want my self-surveillance application to not only give me the ability to find these answers but to give them to me with very little effort on my part.

Nathan argues that any good solution for part of the problem ought to at least aspire to solve all of it. He wants the tools to include some data processing, and to be ubiquitous, so that you can post from anywhere.

In the end, I want all of my data in one place with some machine learning in the background and the ability to analyze and visualize easily and thoroughly. We’re not quite there yet, but I’m looking forward to when we do. Information overload? No. Better-educated decisions and a completely different view of ourselves and our surroundings? Definitely.

Nathan is building his own multi-tracker at your.flowingdata.com. Right now it is by invitation only, but you can follow yfd on Twitter to connect to the next wave of invites.

At the second QS Show&Tell, Joe Betts-Lacroix gave a short talk about his dream system: a website that could receive data and put it into a database, with data would be gathered by little devices that could beam it to the web using simple protocols. (The picture below is of Dan Brown, not Joe Betts-Lacroix; Dan happened to be in the first frame of this segment of video, which is automatically used for reference. Joe shows up a few seconds in.)

In this version of the dream the ideal Web site would have some
simple graphic tools, the ability to export data, good security, and some sharing and privacy options. Since then, there have been quite a few demonstrations of various ideas at the QS Show&Tell meetings, as well as a steady stream of products and, naturally, announcements of products (cf. Fitbit) that aim to achieve some parts of what Nathan, Joe, and other self-trackers have called for.

My own vision is slightly different. I think we are inevitably going to see a bunch of competing solutions, most of which will seem pretty good to some people and deeply flawed to others. People come at self-tracking with different goals and values. Daytum, which is mainly about self-expression, will be nifty for the person who uses data mainly as a feature of personal identity. Daytum’s origin is in the Feltron Annual Report by Nicholas Felton; an annual report serves many purposes, but data analysis is not one of them. Meanwhile, Zume Life is designed for people tracking serious health conditions, who are often trying to manage complex prescription drug regimens. Zume Life has made the data entry process almost as easy as imaginable until the day arrives when we can beam our data from monitoring devices without intervening at all. Along with an iPhone app, there is a voice transcription service. Just press a button, say your number (or food eaten, or exercise accomplished) and a person will transcribe it into a database. This is going forward by going backward, and there is a kind of genius to it. The type of person who likes Daytum is not going to bother with Zume Life, and for the user of Zume Life, Daytum is pointless.

We are headed into a messy, confusing, and interesting period in self-tracking, when lots of new solutions emerge, each claiming a piece of territory and pushing up against neighbors. There will be no ideal system, but a bunch of different sytesms, and then a bunch of different solutions for gluing different parts of different systems together. Some of the people who manage to aggregate lots of users will find excuses for holding on to them. (This may not always be a bad thing: see Dead ends and walled gardens.) But others will see that making the data collected on their system available in standard form will speed adoption.

But what is the standard? You can insert your own favorite standards horror story here. But after you’ve given yourself the shivers, you can recover with the realization that this conflict over standards occurs because everybody can finally see the prize they are wrestling for, which means that a substantial amount of agreement has been won. Once important people start making highly emotional arguments for how quotidian personal data (QPD – now it’s official) should be represented, you can start celebrating.

Note that I said “quotidian personal data” and not “healthcare information.” QPD is the type of thing you are willing transmit in a text message, and SMS is an easy bet for the ubiquitous medium for QPD. But if you call it healthcare information, I’m out. They fight dirty.