Last weekend we had a small barcamp in Amsterdam on the subject of Federating Social Networks, XMPP and other random musings:

I got the chance to play around with Twisted/XMPP for a bit with wokkel and it is really cool and something which could be really useful. Programming Twisted is somewhat too counterintuitive to pick up in a couple of hours.

I’m going to set aside some time to read the docs (again) and then continue on the small project of bridging an XMPP endpoint to a comet web frontend. More on XMPP later.

Yesterday a Roomware Devhouse was held at the Ex Machina office in Amsterdam. Here are some pictures:

To summarize briefly: Roomware is a piece of middleware to facilitate the running of software in a physical space. This most directly involves Bluetooth and RFID like applications which by their nature are limited to a specific area.

You set up a Roomware server and it handles the communication to and from the devices. My small project for the afternoon (we had some ideas for projects which would have been hard to finish) was making a badge clubs can put on their website to show who are in the club at a given moment. With the messaging functionality that was developed the same afternoon, it would even be possible to send your friends at a club messages.

Technically simple, the concept demonstrates how close physical computing has become. The Roomware server does a great job removes the heavy lifting in interfacing with libraries for Bluetooth and RFID.

A great next step would be to add an XMPP layer on top of the roomware server. This way you could theoretically connect your Adium to some clubs of your choice, and see people entering and leaving in your buddy list and even message them.

The only problem is what to use as a unique identifier as your Bluetooth name. I’m a proponent of using a URL but a lot of people do something like their Hyves username or their Twitter name. Fortunately this distinction is mostly blurred using Google’s new Social Graph service.

I like the vibe of devhouses and building something in a strict time limit, but working solo puts limits on your reach. Next time it would be great to work in a team and build something as a group.

Last Friday we had our very own Lunch 2.0 event at the Hyves headquarters in Amsterdam.

Some presentations and a lot of nice conversation with people from the Amsterdam startup scene. I look forward to the next lunch.

Picture by Breyten Ernsting

Picture by Katie Lips

Yesterday we had the year’s end software social. A nice evening with some drinks with startup friends in Amsterdam and talk about work and parties, Python and DiSo.

Not a lot of reminiscing but instead lots of optimism —which I share— looking forward to the coming year. Have a great New Year’s and see you in 2008.

Yesterday like I talked about, we had the Federating Social Networks meetup (Upcoming) at Mediamatic. This resulted in early mornings for those coming from outside of Amsterdam. Tijs, Mark, Pascal and myself took the early train and Blaine Cook and David Recordon had flown in from San Francisco for this meetup. Fortunately there was an espresso machine to keep us alert.

The aim of the day was to talk (see the Jaiku backchannel) about how we could use the technologies at our disposal and the data available to create better experiences for users. It looks like most of the specs are there and the conversation has been going on for most of the year. We should begin building stuff.

Mediamatic at least is dedicated to build something on top of anyMeta which needs to be ready by Q1 2008. The workshop was an effort to gather thoughts on the best way to move forward.

This week also saw the announcement of the DiSo project by Chris Messina. A way of building open social networks using Wordpress as the platform.

Problem statement

We spent quite some time on the problem statement and use cases. The discussion went all over the place both in subject matter, scope and level of detail. At this point I don’t think the philosophical considerations are very useful anymore. This is a broad subject and, yes, everybody has an opinion about it. We need to be a lot more concrete about what we want to build and steps we can take right now to get there. I thought that was clear from the event description but apparently not everybody read that.

The vision that is on the table is quite grand. By distributing and taking ownership of your own data, be it profile information, your relationships, your writing, your pictures or your videos, you get full control. Sites that want to participate in this effort will need to abide by your rules and read and write accepted standards. This means a dramatic redefinition of the way the internet works, so dramatic that it will not happen anytime soon. Also, I don’t think that we can standardize all that in an afternoon.

The problem that we need to solve and which is currently causing painful experiences is: Almost every site and application can be enhanced by adding information about the people you know. How do you do that without replicating effort both for developers and users time and time again?

Finally the list of use cases that we came up with:

• Profile Aggregator (OpenID)
• Access Privacy (profile, contacts, stuff I own, claim stuff/ publications)
• Migration of data/ ownership
• Content discovery/ finding stuff
• Set privacy (noindex, etc)
• Consolidation of data/ profiles
• Personal Messaging
• OpenId - reflection of profiles/ relations
• Referencing accross sites
• Control of representation of copy
• Pingback when your object has been used/ altered

XMPP does it all

A lot of work especially by Ralph is focused on creating a Jabber/XMPP pubsub specification which can be used to post content and updates to and making it easy for interesting parties to be notified of those publications. This is very nice and Ralph’s presentation extolls most of the virtues of XMPP.

Still it will take some time before this becomes relevant for the rest of the web. XMPP is the best thing since sliced bread and I imagine that the guys building it can make it do pretty much everything. There are two problems that hamper its adoption.
First the language and the concepts are sufficiently different that people need a lot of introducing before they are up and running with the concepts.
Secondly once you understand it, there is not much it will do for your blog running on a shared PHP host. Also if you do run XMPP on your own server, you can interface with existing services and you can do anything but there are not any well defined interactions yet.

Mediamatic is aware of this and for their own (PHP based) anyMeta sites and for the rest of the world that wants to participate they are going to provide a bridging server where websites can POST updates using HTTP and the service will publish notifications to interested parties both using XMPP and HTTP depending on the capabilities of the receivers.

HTTP may not be ideal and people fluent in XMPP describe most of the stuff it has been forced to do as hackish. Still, HTTP has a lot going for it. With Atom and REST, HTTP already drives a lot of application functionality over the internet. And with Comet style interaction starting to catch on non blocking HTTP servers will become more and more normal. This will make real time interaction and stuff that is currently not scalable easier.

Moving data

Luis Villa’s post eloquently makes the case for being able to move our data whereever we want. This is quite a big problem and not one that is going to be solved easily if at all.
Sites such as Flickr will allow you to get your data but there needs to be more incentive to open up and more standardization in container formats.

The use case that was discussed of being able to own your pictures, the permalinks pointing to them as well as the comments on those pictures and being able to move that wholesale to a different site strikes me as somewhat too utopian. A site such as Flickr offers you their hosted application and hosts your pictures for you. As it happens Flickr has an API which allows you to get your data back but you will never be able to make a 1-to-1 mapping to another service.

Owning your namespace on a server not your own is a known problem: e-mail has the same problem and it still hasn’t really been solved. A few hosts such as GMail are gracious enough to let you POP your emails off their server, but you still have a middle man that you can’t cut out. Owning your own domain and forwarding it to another service (like Google Applications) seems like the way to go.

I don’t see this issue as going to be solved any time soon. The stakes are too high, the subject is too complex and in most cases a local copy will have to suffice. I have gotten used to losing some data at every significant computer migration. You can’t have your cake and eat it. If you really want to be in control, install phpAlbum on your domain on a generic host and move that around all you want.

Concrete steps towards the future

Marc promised a documentation server with the findings and draft specifications soon. Somewhere early next year Mediamatic will publish their public HTTP to XMPP bridge. Blaine, David and Ralph were supposed to draft something of a spec, but I don’t know when it’ll be made available.

Tijs has been creating quite the list of interesting sites in this space. Like the Attribute Exchange schema supported by OpenID 2.0 which looks very interesting. And a start page for all the standards for this initiative: Data Portability.org.

Another thing would be to start implementing the wordpress plugins listed at the DiSo wiki. I have an hAvatar plugin lying around which needs some testing before release.

A Wordpress plugin that will speak to the XMPP bridge service would need to do the following:

• Add XMPP autodiscovery links to the <head> of the blog.
• Ping the bridge service using HTTP every time a post is made or updated.
• Maybe: listen to notifications from the server for stuff such as blogrolling or trackback.
• Maybe: Publish your friend list as XFN to the bridge so interested parties can subscribe to that.

This won’t be too difficult to implement but it has to wait for the pubsub bridge to become public. It’s looks like the best way to converge to each other is to create stuff.

This Saturday Mediamatic is hosting an event on Federating Social Networks (Upcoming) to further the ongoing dialogue regarding opening the social graph information.

The conversation that started earlier this year and had a big flare with the announcement of OpenSocial just before Barcamp Berlin is going on.

Ralph Meijer presented (announcement on his blog) on the subject during the Web 2.0 Expo on how to solve the problem using XMPP.

At the same time OAuth 1.0 and OpenID 2.0 are coming to fruition and Chris Messina is talking about distributing social networking applications. And Hyves —our Dutch social network— is busy opening up their API.

These are exciting times, but this technological groundwork is just the beginning. The real challenge is making understandable and usable systems using this stuff.

I was at the constitionary conference of the Dutch guild of frontend coders today. The guild is an initiative by Peter-Paul Koch to form an authoritative institution on standards based web development.

A lot of meetings and consultations with various institutions and freelancers etc. have taken place and have resulted in a mission statement, guidelines and proposed organizational structure. Information on the guild can be found on PPK’s webpage (in Dutch).

Front End Guild

The idea of the guild is to certify frontenders so clients know they are hiring somebody who knows about webstandards and is able to make standards aware choices for your web project. The guild will have a certificatory exam and candidates who succeed on this exam will receive a ‘standards aware’ certificate.

The fact that such a professional organization is being formed with standards based webdevelopment at its heart, says a lot about the standards movement and the progress that has been made. A lot of companies have been pushing standards for a long time and this is a way of formalizing that push, while at the same time serving as a wake up call for those firms who still haven’t adopted web standards.

On the other sides these kind of certification programs serve for large clients and large contracting firms to help each other out and in the process close off their markets. I have not become a member because I am naturally suspicious of certification and I don’t see any immediate benefits of membership at the moment. Furthermore I don’t think any exam taken by a commission would prove much about my knowledge or add anything of value to it.

Controversy

The guild has not been without its share of controversy. Some web developers in the Netherlands most notably from Q42 have raised questions about the objectivity and independence of the organization. These have been both questions about the place where the discussion should be hosted but also a lot of issues about strategy, mission and focus.

In the discussion on PPK’s server some comments by people were deleted without clear reason or procedure. This lack of transparency and obviously strange practice of censorship created a lot of doubt in people (including myself).
At some point Lon Boonen of Q42 posed a counter candidacy to be chair of the guild.

I visited the conference on webstandards organized prior to the constitutive meeting. Because I arrived a bit later I did not sit in at all of the panels. Here some short reports from the panels I visited and the issues raised.

Clients Panel

This panel with some clients (‘opdrachtgevers’; couldn’t think of a better translation) told about their requirements for websites and how they got aware about the necessity to code sites with standard awareness in mind.

They seemed to be a very clued in group with a lot of knowledge about standards and why to use them. One of the arguments they gave was that separation of content, style and behavior gives them more grip on a project. A lot of problems why standards could not be properly implemented was put on the CMS and editor level.

One of the issues they did raise and which directly relates to the raison d’être of the guild is that they have a hard time finding parties who can guarantee standards compliant code. They expect the guild and its certification to offer an easy way of finding standards aware front enders.

Interesting to hear the perspective of these clients, but these seem to be clued in more than your average client. It would have been interesting to hear more about ways to convert clients who do not yet see the importance of standards.

Code Review and aside on XHTML

Tom Greuter from Info.nl did a short code review on a project they did at Info.nl with a look inside of the templates. You could see that they have a lot of clue at Info.nl and that they produce some very tight templates, CSS and JavaScript. Also interesting to see that jQuery is indeed quite popular both at Info.nl and with the people at the conference.

I do keep being surprised that XTHML 1.0 Strict is heralded here and at pretty much every shop as the new gospel. What happened to using POSH and HTML 4.01 which will work everywhere. Did they simply go out of fashion because they were not new and snappy enough anymore? It’s not like there is no literature on the subject or that XML is at all alive on the web.
You can have a technical discussion on this issue, and you could side with either one considering the technical merits. The actual distribution however suggests that nobody has looked at the technical merits at all but that everybody has simply jumped onto the same bandwagon. If we want a vital and really standards aware frontend community, it would not hurt to incorporate more independent critical thought into it and teach people to make their own informed decisions.

Flash within the guild

To end the afternoon Bobby van der Sluis made the case for why Flash should be taken into the guild as a discipline in front-end coding. He took a very circumspect way of getting there and posited Flash as a plugin technology in the continuous development of the web and in its relation to the other technologies we already have to our disposal. This was a very smart way of putting things but unfortunately it did not really address any of the real issues.

Here is my take on Flash. (This could become an article in its own right.)

Whenever the issues of accessibility and search engine indexability with Flash are raised, there are always some Flash developers who say that it is possible to make accessible websites in Flash. While this may be possible in some distant galaxy, it certainly is nowhere near common practice and as long as that is the case Flash programmers need to be called on their bullshit.
It clearly is not easy enough for your common Flash developer to create an accessible Flash site. This lack of interest and capability within the Flash platform to play nice with others makes me think that there is too much work to be done to clean Flash up before it deserves an equal place in a standards based movement.

Right now Flash is only justified when used for complex visuals, animation and/or media. I have yet to see media poor Flash sites which provide the user with a decent experience. A notable exception for this argument is I’m In Like With You which had some very nice Flash integration in its previous version, and where they are pulling off some truly virtuoso work in the current.

Flash sites do not even adhere to the most basic tenets of usability. Usability dictates that people spend more time on other sites than they do on your site. Therefore your site should look and act similar to other sites.
This does not stop common Flash developers from re-inventing user interface, interaction and controls time and time again. In almost all cases they reinvent all of these paradigms very very poorly, rendering a grave disservice to their users. Every button in every Flash site is different (because it is skinned) but a button may seem like an easy control to implement, it definitely is not.
Add to this the impossibility of altering text sizes (which always seem to be too small) or to apply user style sheets for other special needs and you have a usability disaster.

Finally Flash is not a part of the Open Web. Web standards are open technology to promote an inclusive and open web on which everybody can participate. Flash is a closed proprietary technology which locks in both authors and consumers to a single vendor. For only this reason alone Flash should be avoided like the plague and handled with extreme care in the rare circumstances where the benefits outweigh the multitude of cons.

Constitutive Meeting

After the conference everybody went somewhere to get some dinner and prepare for the final constitutive meeting. I could not attend this meeting, but we should have blog posts summarizing the events of the night and the outcome of the elections some time soon.

Update: PPK himself has a post on the meeting and it seems that the differences have been solved amicably and the guild is off to a good start.

Yesterday we had the FOWA roadtrip drink with Ryan Carson in café de Jaren. It was a nice gettogether for the Amsterdam based crowd.

During the raffle Maarten Lens-FitzGerald won the free ticket to FOWA. He will undoubtedly have a great time at the conference.

It’s great to have visitors in Amsterdam and showing them the Dutch scene. Amsterdam will have a PICNIC just before FOWA which is a strange event in many ways.

This Monday night in Amsterdam there’s going to be another road trip drink for a big Web event in Europe: The Future of Webapps. Check out the road trip site (Upcoming) there are quite some people coming out to hang out, chat about webapps and say hello to Ryan Carson.

The location for this event is set in Café de Jaren just like the last one with the O’Reilly people for the Web 2.0 Expo in Berlin this October. Come say hello and join us.

Future of Webapps

The Future of Webapps looks to be an event with a great lineup of speakers scheduled this October 3-5 in London. Cristiano from Four Starters is going to attend and covert this event for us.

Monday a week ago I visited Mediamatic on invitation from Willem Velthoven to talk about how they could fit in Portable Social Networks in their anyMeta system. This meeting was inspired by our meeting in Copenhagen and the talks we had about opening up social networks.

Picture by Matt Biddulph

anyMeta and OpenCI

Mediamatic.lab implements and maintains a series of social networking sites for the creative industries (CI) in Amsterdam. These are sites built on the anyMeta system that resemble structured wikis with a strong social dimension. They are positive towards open source, but the anyMeta system is not open source for reasons of manageability of the projects.

Seeing as that these sites have a lot of overlap in both in functionality and in the people that have an account on them, they wanted to abstract and syndicate the social stuff as much as possible. Currently people can have accounts on each of the different sites, all with the same information on them.

Seeing as Mediamatic builds anyMeta themselves and they have total control, it is very feasible for them to devise and mandate the exchange of information between their own sites. To enable the exchange between their own sites, they will use their own protocol and data format to provide for a high fidelity exchange of information. Leaving implementation details for what they are, it should become possible to use one account on any of the sites in the network.

To verify your identity on the various sites of the network they are going to enable OpenID consumer and provider functionality in the next version. This way they will have a way of distributed authentication both within their network of sites and throughout the rest of the internet.

anyMeta and the rest of the web

Having solved the problem of information exchange between anyMeta sites, they would also like to play along with the rest of the internet as far as that is possible. Being able to share public information with the rest of the internet in a logical way is also on the agenda but not so straight forward.

Making public profile information available using hCard and related microformats looks easy enough. Problems arise however because the templates are made by different people and that is the location of the microformatted markup. This means the template authors have to be educated on the subject of microformats.

Whenever I advocate the use of microformats, I always have to fight against the blank looks and criticism about the aplicability of the technology. It’s a solid Catch 22 that has to be taken on with real life use cases and benefits to extoll the virtues of a dirty semantic web. For hCard there are various uses cropping up over the internet, but for the others it is a lot more limited. Having microformatted data on sites and being able to parse that using browser plugins is a first step and essential groundwork for the real use cases and richer interaction that we all want to have.

Another plan they have at Mediamatic is to first enable the sharing of information between their sites and make plugins for some of the bigger CMS’es out there (Drupal, Joomla) so they can also exchange information with those systems.

In these use cases and in the case with the internet the issue of fidelity comes up again and again. How much information can you exchange reliably and what do you do when stuff is missing? This is an important and valid question with no ready answer; though mine would be ‘get what you can, and ignore the holes where possible’.

Other stuff

I am currently not implementing anything relating to OpenID and Social Networks but I think I would like to. One idea was to make a Facebook front-end site which uses the information in Facebook to offer you a microformatted profile. There already is an hCard application but extending this with XFN, hReview and hResume would be a real winner.

Yesterday on the O’Reilly event I heard about Yme Bosma who’s job it now is to drag Hyves kicking and screaming into the world of Open Social Networks. I wish him a lot of luck as that would be a good thing to have. I have started my own work on scraping the Hyves site but that hasn’t been as simple as I would have liked.