So the conference is over. Jason, Gordon and I spent the rest of the day at the Del Coronado Hotel where we had our last dinner in San Diego before heading to New York.

There will be a hiatus until next year but the content will definitely keep relevant as most of the subjects covered are now at the edge. Who knows what will be the hot topics next year? Those who read this blog during this week and specially those who attended the conference are welcome to write your predictions for 2006. Keep coming back or just grab our RSS feed for new follow-ups.

Matt Webb who's wrapping up his presentation here at ETech has been posting his personal notes for other sessions. You can find them here.

Google is just announcing Google Code. Google's place for Open Source software. More details in a session this afternoon. http://code.google.com/

Cory Doctorow and Larry Lessig were on stage on a very interesting conversation that you can listen here.

[Via IRC back-channel, Thanks Matt Haughey]

CC launched The Code and Laws of Cyberspace wiki under the new CC-Wiki license.

Larry Lessig is on stage and unfortunately he won't be followed by Gilberto Gil this time around.

He starts telling the story of H.G. Wells' book " The Country of the Blind" passed on a village where everyone is blind and Nunez a non-blind visitor goes to a doctor that wants to remove his eyes.

Stop! Remix is nothing new. That's how cultures were created. Apple remixed, Bill Clinton remixed, we all do it. When we watch Michael Moore's film—for example—and thell our friends whether we loved it or hated it. That's remixing.

In world history remixing was never regulated. Remixing needs to be free; the ordinary ways of remixing. Ordinary ways are "word" and remixing text, writing is free.

What when the technology of remix changes? Do the freedoms change as well? He plays an audio snippet of the Grey Album. He mentions Tarnation, a film made with a budget of $218 that wowed Cannes. Follows some videos remixing Fox News and President Bush (liberal), John Kerry (conservative). The audience goes crazy with the video remix of Bush and Blair singing "My Endless Love". Always funny to watch it. "That's video creativity".

Not broadcast or NYT democracy but popular democracy. The question come back on screen: What when the technology of remix changes? Existing laws conflict with technology, laws need to be reformed (remixed?). Reform the law or reform the technology. Just like H.G. Wells story the powers to be want to reform technology, remove its  eyes.

1. Let's call "piracy" "piracy". Let's call "piracy" wrong. (Notice the quotes.)

2. Teach how powerful the technology is.

3. Demand changes in the law. Not a call for the end of intelectual property.

4. Punish. Defend and oppose the law because it will destroy our technology.

is Yahoo's resident MySQL Geek and he explains that the developer community requested Yahoo! to expose their search services. They saw it as a way to encourage 3rd party innovation. The initial goals were to keep it simple, collect feedback and create a low barrier to entry.

The Developer Community contains the SDK, a wiki, a blog, and general documentation of the API—which offers search on Web, News, Image, Video, Local plus a new feature of contextual search.

The architecture is based on backend clusters (for web, image, video, etc.) that are accessed by the application via a XML proxy developed in PHP. The proxy cleans up the XML results coming from the backend already existing internally to Yahoo! PHP is the de facto language of choice at Yahoo!

There's a restriction of 5,000 queries/day/service/IP - there's no developer token. So if you distribute your app each install wiill have its own limit.

The usual debate to decide for REST or SOAP also happened at Yahoo! "REST is 80% of the usage and SOAP is 80% of the support burden"—so REST was the choice. If there's community demand for SOAP they might consider implement it too.

Some cool apps were already created by 3rd parties and can be found here.

Clay Shirky's presentation is called Ontology is Overrated: Links, Tags, and Post-hoc Metadata.

He starts off by defining Ontology and tell us the parable of the travel agent. The periodic table of the elements is one the great examples of classification. The Library of Congress categorization contains an imbalance with very generic element representations like Asia and Africa because the criteria used was the number of books on the shelf.

Yahoo! was the first significant atempt to bring order (categorization) to the web. They hired ontologists to categorize the content. There were shortcuts to other categories if users tried to find a category in a wrong place (e.g. Books and Literature shortcut under Entertainment in case users went there to find Book and Literature). It was the change from Hierarchical categorization to Hierarchical categorization with links. The huge quantity of links made the hierarchy no longer necessary. That's when the search appeared. Even Google at some point adopted DMOZ but then discontinued as there was no one using.

So when does ontological organization work well? Only when the domain is restricted and the participants are experts. The web is not such case.

Voodoo categorization happens when one can force a categorization to users. This causes:
Signal loss. E.g. Mac, Apple and OSX; Movies, FIlm and Cinema, Queer, Gay and Homosexual.
Makes it hard to predict the future: E.g. "This book is about Dresden" vs.  "This book is about Dresden and goes in the category East Germany"

Merging ontologies is very difficult, Do we merge categories or GUIDs? In real life real minds don't think alike, that's when del.icio.us comes into scene. The distribution of tagging is a long tail—few users with lots of tag entries and lots with few. The distribution of tags for one individual user is also a long tail. Lots of tags about few subjects and lots of not so frequent tags. Modeling the distribution of how users tag one individual URL is also—you guessed right—a long tail. Lots of people tag the URL with one or two tags.

This is the called organic categorization—user and time are core attributes; one-off categories are lost in the rear end of the tail (the system is the editor); the semantics are in the users, not in the system; merges are probabilistic, not binary.

Here's the transcript of Cory Doctorow's speech. I'm sure someone will soon create a wiki and/or translate to all possible languages. Link to Cory's website.

Justin Chapweske of Onion Networks starts by talking about the HTTP protocol how it has been used since the days of Netscape 1.0  and proposes a remix.  That protocol problems today are more evident as larger data files are being transmited over the Internet (e.g contents of a DVD).  The idea is to use a swarming technique, instead of just throwing more resources (read $$$) to solve the problem. Swarmstreaming is a concept is similar to BitTorrent that breaks up a file and transmits is to various clients that transmit their pieces to each other.

Today's first presentation was from Neil Gershenfeld, from the Center for Bits and Atoms, MIT which is a group of 20 scientists incuding biologists, chemists, physicists, mathematicians, various kinds of engineers.

The state-of-the-art fabrication is not at 10 billion chip plant but at the ribosome. That's the insight. The ribosome is the living proof for the digitization of fabrication. Computers not controlling tools but computers as tools.

He shows some of the students that first applied to attend the class on the subject of personal fabrication (How to Make Almost Anything). Their projects are fun stuff stuff like the Screambody, Interpet Explorer, Defensive Dressing and an alarm clock that you have to wrestle with to prove you're awake. Some practical examples are the personal fabrication labs created in places like Ghana, India.

His new book on the subject is  FAB The Coming Revolution on Your Desktop.

Here's a recent article about Neil's work: How to make (almost) anything

Werner Vogels, CTO—who just came back from India where he was at Amazon Development Center—starts with an interplanetary vision of happy flying people that buy books.

Massive is defined as billions (world population < neurons in a brain < stars in a galaxy < galaxies in the Universe < ants in  the world—one quadrillion!) He enlists various technologies that are enablers for large scale. There are also social enablers for scale (anything that gives people instant access to sex, food and money). Talking of scale current Amazon's scale is of 32 items ordered per second. And it works now through their implementation of solid and smart systems engineering.

Biological systems could be a model to use in order to achieve scale: redundancy, feedback loops, modularity, loose coupling, purging, apoptosis, spatial compartmentalization, distributed processing, extended phenotype. (After that slide I just started to imagine a meeting between Werner and Jeff Bezos, that should be interesting.)

He suggests Epidemic Theory of Infectuous Diseases (which by the way is nowhere to be found on Amazon) as late night reading. The power of epidemics can serve as a model for robust distributed systems. Epidemics turn scale into advantage. His point is that at algorithm and protocol levels epidemics are a good approach to manage state.

Conclusion slide

• The customer is the only thing that counts

• Question your assumptions

• Learn from Chaos

• Let go of control

• Turn scale into advantage

Tim Halle, Director, The Project for Open Source Media (POSM) is presenting Taking Back Television: An Open Approach to the Development and Deployment of Next Generation Media.

The idea of 'taking back' is to open up distribution channels to any interested party. There are issues with access—that's pretty much currently locked out. There are development issues related with access to dev environments and lack of standards.

Solution?

Open up standards and reverse engineer the set top box. Ingredients? hardware components, low end PC, tuner card, NTSC output card, Linux, Mozilla, Mplayer and DCR (Digital Content Recorder).

Open up the EPG (Electronic Program Guide) with agnostic content and transport, user control and making it open to any interested party.

Extending functionality means to first of all solve the problems with ITV, as we know it: latency, slow back channel, weak browsers, "viewer experience" interruption.

Demo:
Tim and his team, then proceed to show a demo of an interactive version of the cop show Boomtown.  Next demo is of an open interactive EPG. Third demo is called Manhattan Project, a documentary that's is enriched with another layer of Flash content that shows animation of nuclear fission. The viewers can then switch back and forth.

Nelson Minar, Software Engineer at Google is talking about how his team designed and deployed the Google AdWords API. The API allows developers to integrate with the platform in order to manage bids, optmize ROI/campaigns and keywords, customize UI and integrate advertising with backend systems. Nelson was also responsible for developing the Google Search API a couple of years ago.

The technology used was SOAP, WSDL and SSL with multiple authentication mechanisms. In order to make it simple to use there's no XML or HTTP work involved, this abstraction is possible via WSDL. The idea is to make the data model as the center of the design, the API shouldn't be thought as function calls.

In reality interop is hard to achieve, WSDL and SOAP doc/lit support is not unique and varies by toolkit: .NET, Java (Axis) toolkits are good; C++ (gSoap), Perl (SOAP::Lite) are OK, Python (SOAPpy, ZSI) and PHP not as easy. He then shows 3 different ways of sending no data and how the different toolkits handle them. Still doc/lit seems to Google as the only real choice.

Things that went right, according to Nelson:

• The switch to document/literal

• Stateless design

• The developer reference guide

• Developer tokens

• Thorough interop testing

• Private beta period

• Excellent tech support group

• Bulk methods

Things that went wrong:

• The switch to document/literal

• Lack of common data model

• Dates and time zones

• No gzip encoding wroking right now

• Quota confusion and anxiety

• No developer sandbox

• SSL (hard to debug, slow)

Peter Norvig, Director of Search Quality starts off by introducing Google Labs website.  Apparently no brand-new apps. Just the following demos:

Google Suggest
Google Maps
Google Personalized
Google Suggest in Japanese
Google Sets