Barely Legal Substance: More on Walker Percy, Terrorism, and English Languge Data Parsing

« "The man took his dog to the park. After playing ball and seeing a duck he went home." | Main | For Justin & My Other Ex-Czech Friends »

March 09, 2005

More on Walker Percy, Terrorism, and English Languge Data Parsing

In the 1987 novel The Thanatos Syndrome, Walker Percy sketches a couple of scenes that seemed to me quite ludicrous from a software and data angle. In response to the inevitable crisis, the love interest (apparently with l337 h4X0r 5ki11Z) runs to her home in the marsh, boots up her pc, and interfaces with four different government databases to cross-reference and visually display a map that correlates the chemical elements in state water supplies with a map of the state.

Not that the hardware is silly, nor that the data is silly. Both were quite accurately described by the incomparable Mr. Percy. What my mind balked at was to believe that four different government agencies have databases that are so easily mined, and a visualization software that can scale so well (from chemical concentrations in water to maps of the state).

Now, I think, I can finally see where we could get this. Google Maps + XML + Web Services for databases + English language data parsing.

The hard part was never getting the data (though that's interesting). Nor was it matching one thing with another (that's pretty cool too). The hard bit is, how do you come up with a thing in the first place just from raw data? There's no ontology in data, it's just data.

From the article:

"We have guys who can crack hard drives," Mr. Alexander said. "Getting the information out is easy. The hard part is sharing it, and organizing it, so that everybody in an agency, even nonexperts, can use it."

The data has always been there for the terrorists to use, or for companies to, ahem, lose track of (a la ChoicePoint). It's just that there's no easy way to pull it together into something useful.

This post is partly in response to willas comment on my last post about English language data parsing.

Post this page to your del.icio.us account
Posted by Noel at 05:44 PM tagged as: