The Paradox of the False Positive

An excerpt from Cory Doctorow's latest book, Little Brother:

If you ever decide to do something as stupid as build an automatic terrorism detector, here's a math lesson you need to learn first. It's called "the paradox of the false positive," and it's a doozy.

Say you have a new disease, called Super-AIDS. Only one in a million people gets Super-AIDS. You develop a test for Super-AIDS that's 99 percent accurate. I mean, 99 percent of the time, it gives the correct result -- true if the subject is infected, and false if the subject is healthy. You give the test to a million people.

One in a million people have Super-AIDS. One in a hundred people that you test will generate a "false positive" -- the test will say he has Super-AIDS even though he doesn't. That's what "99 percent accurate" means: one percent wrong.

What's one percent of one million?

1,000,000/100 = 10,000

One in a million people has Super-AIDS. If you test a million random people, you'll probably only find one case of real Super-AIDS. But your test won't identify *one* person as having Super-AIDS. It will identify *10,000* people as having it.

Your 99 percent accurate test will perform with 99.99 percent *inaccuracy*.

That's the paradox of the false positive. When you try to find something really rare, your test's accuracy has to match the rarity of the thing you're looking for. If you're trying to point at a single pixel on your screen, a sharp pencil is a good pointer: the pencil-tip is a lot smaller (more accurate) than the pixels. But a pencil-tip is no good at pointing at a single *atom* in your screen. For that, you need a pointer -- a test -- that's one atom wide or less at the tip.

This is the paradox of the false positive, and here's how it applies to terrorism:

Terrorists are really rare. In a city of twenty million like New York, there might be one or two terrorists. Maybe ten of them at the outside. 10/20,000,000 = 0.00005 percent. One twenty-thousandth of a percent.

That's pretty rare all right. Now, say you've got some software that can sift through all the bank-records, or toll-pass records, or public transit records, or phone-call records in the city and catch terrorists 99 percent of the time.

In a pool of twenty million people, a 99 percent accurate test will identify two hundred thousand people as being terrorists. But only ten of them are terrorists. To catch ten bad guys, you have to haul in and investigate two hundred thousand innocent people.

Guess what? Terrorism tests aren't anywhere *close* to 99 percent accurate. More like 60 percent accurate. Even 40 percent accurate, sometimes.

What this all meant was that the Department of Homeland Security had set itself up to fail badly. They were trying to spot incredibly rare events -- a person is a terrorist -- with inaccurate systems

.

DHS, in Doctorow's skillfully-written book (full disclosre: I'm a long-time fan) has taken over San Francisco after a pretty horrific terrorist attack, and implemented a security net so obnoxious that everyone is a suspect, turning the "innocent until proven guilty" maxim fundamental to the US justice system on its ear in the name of "protecting" the very citizens it interrogates on every corner.

As if we needed a reminder, the recent Mumbai attacks have thrust the extraordinarily rare specter of the large-scale, well-coordinated urban terror strike back into the public spotlight. Doctorow's greatest strength has always been his ability to make the future feel incredibly near; it is not even remotely difficult to imagine a scenario like the one described in LIttle Brother playing out were another major attack to strike a US city. Almost as frightening than the idea of an actual attack (or more frightening, depending on who you talk to) is the idea of what might come after. Imagine a massive citizen-tracking web, to put London's CCTV to shame, layered on top of the already overanxious contemporary cityscape, and consider the paradox of the false positive. Not pretty.

Little Brother is available for free download under Creative Commons licence right here.

Designing SkyCar City: A Post-Studio History (Guest Post by Ella Peinovich)

Many notable architects and designers throughout history have created transportation-centric city models that represent the future or an idealized present. Historically, architectural movements have used these models to demonstrate their design fundamentals in a pure state to a world that did not, at that time, have the capabilities to achieve such idealistic forms. Each designer, in turn, walked a fine line between reality and fiction.

A few models worth mentioning would be the Futurist Antonio Sant'Elia’s Citta Nuova, and then, within the Modernist movement, Le Corbusier’s plan for Algiers and Frank Lloyd Wright’s Broadacre City. Later, the traditional utopian city changed when the line was officially crossed from being about the study of technological advances to science fiction, with Archigram’s Walking and Plug-in Cities. More recently, the 3D city model has been featured in the work of Dutch firms OMA and MVRDV through designs like the Hyperbuilding and the Five-Minute City.

Taking all of this into account, one can understand how overwhelming -- not to mention exciting -- it might be for a group of young architecture students to be presented with the open-ended challenge of creating such a complex city model by Winy Maas of MVRDV, a veteran to the topic. That was the situation I (and eleven of my classmates) found myself in during the last studio of my education in architecture.

“…create a city built for the use of a skycar, a city with 'streets' at any level, or perhaps empty of streets as we know them…”
-- Winy Maas, first day of the semester describing our assignment.

My studiomates and I were presented with an experiment: see how far transportation can shape architecture, with one constant variable -- the flying car. Winy had introduced us to a whole new set of Lego pieces; ones that were gravity-defying. We were to design a city that had no specific location, no city boundary, and no technological limits. This is a similar formula to those that have been at the heart of most city models in the past, except for the fact that we (twelve near-strangers) had no pre-established theory to drive the experiment.

We, with the guidance of our studio's co-leader, Grace La, quickly developed a pragmatic methodology of how to approach our design and then applied it to our hypothetical scene, assuming that humans would always be subordinate, or at least married, to our means of travel (this was also in line with the prescribed “sky car”). Ironically, though we were all architects-in-training, little time was spent designing architecture in the traditional sense. During our process, research, and developments we wore many hats: Urban Planner, Product Designer, Civil Engineer, Research Analyst, Historian, Editor, and then Architect. This mash-up of information with the added flavor of democratic group decisions more or less drove our re-imagining of the city form.

During this experience I learned some valuable design lessons. In no particular order of importance:

1) It is our responsibility as designers to address real issues rather than represent feel-good utopias.
A city model that is designed in reaction to current outstanding issues of common society (e.g. waste disposal, greening, traffic relief etc…) will likely get a lot of attention and praise. On the contrary, our city model chose to explore and build up a topic which currently carries a negative stigma. We suggested that, as a society, we accept that every person wants the freedom of having their own car. We chose to assume this desire of every individual and suggest that public transportation has no future. We feel our model holds its clout because it is based on realistic projections of where society IS headed, rather than where it SHOULD be.

2) Design by committee vs. the individual
When an individual is allowed to carry out an idea, they are able to create one conclusive and seamless design in a similarly seamless process. As soon as you allow everyone a voice, all you get is noise -- at least at first. It took my studiomates and I a while just to figure out how we could work together, which inevitably took away valuable design time. Admittedly I, and likely others, caved to the majority vote at different times during the design process just to move forward, and I know that many “place-holders” never got re-”placed”. However, I can confidently say that our design would not have been what it was without any one person in our studio contributing their input.

3) Build from the Bottom-up
I have to believe that the Modernists and Futurists had an idea for the final form of their models before they designed the infrastructure to support it. In my experience, a top-down method can be inefficient, continually having to cover its tracks along the way. Please do not misunderstand me; once a design has been established I think it is necessary to go back and implement the design diagram to every last detail, but I am referring the process of getting to your building diagram. On the contrary, we were very honest in our studies; we did not hide the unpolished edges that are always a possibility of functionally-driven design. Our process was prescriptive and built from the bottom-up.

4) Get Published
The most responsible thing a designer can do is contribute to the Industry by publishing and distributing their work. Being able to show something as a product of thought can create a much larger ripple than just talk, especial in a visually-oriented community. Work should be shared, flaws and all, so that others may see and critique in order to create dissemination.

There may be a conflict of interest for some in the architectural profession that view their work as an artform. But when you think of what has been entrusted to us by the public (e.g. to create efficient systems, reduce environmental impact, ect…) we owe it to everyone to find the best technologies to do the job, regardless of who authored them.

Now, well after the publication of Skycar City: A Pre-Emptive History, the record of my and my classmates' semester of city-building, it is up to the readers to make their own conclusions about the success of our research and model and decide whether it is worth mentioning alongside the great city models of the past.

------------------


Big thanks to Ella Peinovich, a design associate at Skidmore, Owings, and Merrill's San Francisco Office. Ms. Peinovich participated in a mixed studio, led by guest professor and MVRDV founding partner Winy Maas, of twelve undergraduate and graduate students at the University of Wisconsin-Milwaukee's School of Architecture and Urban Planning in the spring of 2006. The participants recorded their process and city model in the book Skycar City: A Pre-Emptive History.

Skycar City Studio co-leaders: Grace La, Winy Maas

Skycar City Studio participants: Bryan Howard, Tony Janis, Nick Moen, Ryan O’Connor, Trevor Patt, Ella Peinovich, Nick Popoutsis, Tarah Raaum, Gloribed Rivera-Torres, Scott Schultz, Tuan Tran, Andy Walsh

Edited 4/12/08 to include full list of studio participants.



Blogger's Note: The original post planned for tonight and announced on Sunday was going to focus on urbanism programs for kids in Singapore and Chicago. This post has been temporarily postponed, and will run at a later date. Sorry for any confusion!


Links:
Skycar City (Powells)

MVRDV-UWM Studio (Warning: absurdly long load time).

Urbanffffinds 016

Sustainability and the Unimaginable City

A recent post at WorldChanging offered up for discussion a fantastic Bruce Sterling quote about sustainability: "The ruins of the unsustainable are the 21st century's frontier."

That's a pretty profound statement. To think that the world around us, the things that we consider familiar and take for granted, will be to our great grandchildren what the Roman Aqueducts -- or even British colonial forts -- are to us: relics. Of course, much of our built environment (especially the unsustainable portions) are considerably less visually appealing than crumbling arches and turrets. It's fun to imagine that, somewhere outside Cincinnati in 2108, there will be a tour bus that rolls past long-abandoned McDonald's huts and IHOP psuedo-ski lodges, describing to wide-eyed tourists how these strange buildings used to be used as restaurants back when people were willing to drive twenty minutes for food that had been flown across an ocean. But the likelihood of that scenario is slim.

Most likely, metropolitan regions will adapt to changing conditions, with cities and suburbs both densifying and growing and connecting in new ways. But what will become of the buildings and structures that are left in the sustainability craze's proverbial dust? As people become more educated on and concerned with sustainable buildings and neighborhoods, it seems logical to assume that demolitions will become less frequent than they are now as architects and planners seek to preserve structures with bones worth saving, and to reclaim materials when demolition is required. Politicians will mandate it; high tear-down rates will spell doom at the voting booths.

Even moreso than buildings, large chunks of our infrastructure could very well become obsolete, and will need to be repurposed. A post today at Loud Paper described a recent live design and model-building competition with a very novel premise: the two teams of architects were asked to design futuristic outpost buildings in the Alaskan wilderness, and their primary material was repurposed oil pipeline. Take that general concept to the city; what pieces of our current urban infrastructure -- even, or especially, the invisible pieces -- might we be trying to reuse as a construction materials once the large-scale shift to renewable energy sources has taken place?

And now, another Sterling gem to throw into the mix: "To me, 'sustainability' means a situation in which your descendants are able to confront their own problems, rather than the ones you exported to them. If people a hundred years from now are soberly engaged with phenomena we have no nouns and verbs for, I think that's a victory condition."

Whoa. Kind of re-frames the whole "repurposing" thing, hey?

It is easy (and, I think, quite common) to assume the perpetuation of current conditions. Humans like to think about the future, but we are stuck living only in the present. This means that our experience of the present has a very, very heavy influence on how we envision things ten, fifty, or a thousand years down the line. We live in fairly hectic period, so it can sometimes be difficult to imagine a world where the problems are completely different from those we face today, much less that we don't even have words to describe! My question is (inevitably, if you've ever read this blog before), how will this affect cities?

It's a silly question, really. After all, the very idea being discussed here is that the problems to be faced by a post-sustainability humanity aren't even conceivable to us now, so how the heck do we go about coming up with what those problems might entail? Of course, the answer is: we don't. We can't know what these problems or their solutions will be, but we can use the scenario to appreciate the massive opportunities presented by the challenges of creating a sustainable global society. All we need is a framing device.

Enter: the Air Tree. Madrid's recent announcement that they will be building three of these miraculous little structures (which were originally the result of a design competition in 2004) marks a very important moment in architecture and urban planning that is easy to overlook: the birth of an entirely new building type. The Air Tree is a temporary structure that marks what will hopefully be a permanent change in how cities will be built in the future, since its existence signals the arrival of the building with the sole purpose of acting as a unit of production for Quality of Life. These three futuristic pavilions will create for madrileños gathering places that use natural processes to filter and cool the air. The space beneath the Air Tree is not just social; it's cleansing and renewing.

Many common features of contemporary urban areas -- not in the least the Air Tree -- would have been unimaginable to people living a century ago. Stop and think about that for a second. Now try to imagine the unimaginable city of the future. Pretty thrilling, isn't it?


Links:
The Ruins of the Unsustainable (WorldChanging)

Architecture Gone Wild (Loud Paper)

What we get if we win... (WorldChanging)

The 'Air Tree' (Architecture.MNP) (Photo credit)

WEEKEND READING: November 3-November 9, 2007 (Guest Post by Colin Kloecker)

Hi, I'm Colin and I usually write over at Blog Like You Give A Damn. But this month you'll find me here, curating Where's Weekly Reading segment. Brendan has graciously allowed me to break from his standard format for something a bit more casual. Let's call it a stream-of-thought meditation on a theme.

This week -- Near future urbanism: how an ubiquitous and multi-layered network might effect our urban environment. Photo: OiMax (via Flickr)

Moore's Law tells us that computing technology is advancing exponentially – essentially doubling every two years. With Rainbows End, mathematician, computer scientist, and science-fiction writer Vernor Vinge extrapolates on Moore's Law and envisions a near future where the physical world can be reinterpreted through "contact lenses that can overlay what the wearer sees with skins that redress the environment to your taste." [Check out this smallWORLD podcast to hear Vernor talk at length about the concepts and technologies behind Rainbows End] A similar concept is explored in William Gibson's recent novel Spook Country, in which locative artists create incredibly complex site specific geo/virtual mashups, tied to specific locations through GPS and wi-fi technologies [see gpsdrawing.com or this video for some real-world examples].

Both of these authors and their recent works are proposing a malleable virtual/ urban environment where user generated content meshes seamlessly with concrete and grass [see CSI:NY's recent foray into Second Life for a not-so-seamless example]. Essentially, if you don't like the looks of that new condo going up across the street - simply design a new facade, upload, and share with your social network of choice.

In a world where the net is everywhere and we are always online, geographic locations will be linked in as well. It won't be long before we are going about life tagging locations with keywords or personal stories like we would a page on the web (imagine geo.del.icio.us or MySpace taken literally). As technology advances and life-caching becomes more robust, you could feasibly go back to the location of your first kiss, bring up the appropriate file, and project that exact moment in real-time. Hackers and artists will upload pirate landscapes, momentarily hijacking reality for thousands of people at a time. Communities with similar belief systems might organize around a shared perception of reality or aesthetic sensibility - and then actually make that their reality.

So all of this speculation begs the question: in this media-rich user-generated landscape, what role is there for the architect and urban planner? Will cities be reduced to the skeletal equivalent of a wire frame 3D model, designed to be dressed up with an infinite and interchangeable architectural wardrobe? This is to say nothing about questions about privacy or invasive marketing strategies... phew!

As you can probably tell, I've been on something of a futurism kick. Check out futuretalks.com if you're digging it too. They recently started broadcasting an interesting series of conversations with futurists Gerd Leonhard and Glen Hiemstra. Until next week, Happy Friday!