Werd I/O

Last week at Homebrew Website Club, I was asked how Idno syndicates to third-party sites like Twitter when I post content.

Here's how it works.

First of all, Idno has a plugin system, that allows new functionality to be added system-wide. As well as new kinds of content like slide presentations, plugins are available that interact with the APIs of Twitter, Facebook, Foursquare and Flickr.

When I install any of those plugins in Idno, I'm taken through a process where I register my Idno site with the third-party API. Each of those sites has a slightly different process, but in each case it takes about 30 seconds.

Once the link has been made, the plugin shows up as an option in Idno's user settings screen. I click "settings", and then click a button to link my account to the site:

This is exactly the same procedure as logging in with any of those sites, or attaching any third-party application. It's about two clicks: in the case illustrated above, I'm taken to Twitter, which asks me to confirm that I want to give Idno permission to use my Twitter account, and then taken back to my Idno settings. Internally, my OAuth token for that site is saved to my user account.

Here's where things get interesting.

Remember I said that Idno's content types are also provided via plugins? There's a plugin for status updates, for photos, blog posts, events, etc etc. Whenever I want to add a new content type to Idno, I add a plugin. (They're really easy to write; the presentations plugin was written in about an hour, while I was recovering from a root canal operation.)

As well as descriptive content type - "status update" - each plugin announces a generic content type that maps to those used by the activity streams specification. A status update is also a "note"; a blog post is an "article". This allows plugins to extend functionality for certain kinds of content without dictating which plugin you use for that content. Someone can add extra logic for status updates, while not caring which status update plugin I actually use.

When I post new content, the system pulls up an interface supplied by that content's plugin, and also asks any syndication plugins if they're able to handle content of this type. So when I click on my "status update" button, Idno asks plugins if they're able to syndicate content of type "note".

Idno automatically renders some buttons for me based on those plugins. If I enable the "Twitter" button, my content will be syndicated to Twitter when I post it. If I enable the "Facebook" button, it'll go to Facebook, too. If I later decide to add a button for Path or LinkedIn or Friendster via a plugin, it'll show up there, and work in exactly the same way, without me having to change any of the status update plugin.

When I hit Save, the syndication plugin receives information about the content type (but not which plugin created it), as well as information about my account. It retrieves my API token from when I linked my account through my settings panel, and uses that to sign an API request posting the content to that site. It then retrieves the URL of the syndicated content and saves it to the local content in Idno, so a "syndicated to" link can be displayed underneath it. (Check at the bottom of this post's page: you'll see a link to Twitter.)

This process works throughout Idno. Photos (of generic type "image") can be syndicated to Facebook, Twitter and Flickr, and while the logic is different for each site, the user interface flow is the same for each one. This works whether I'm posting from a laptop or a phone, and whether I'm on the standard web interface, a custom interface or the API.

It's important to note that none of this takes much time, for any of the parties involved. Writing a content plugin takes about an hour; writing a syndication plugin can take much less time, if the third-party API uses OAuth. Site admins can install a plugin and set it up in a few minutes. The process for the user takes mere moments, and that's the most important thing.

#idno #posse #indieweb

When I left Elgg in 2009, I immediately started working on Outmap, a social geodata platform that you could use with any web browser. It never saw the light of day.

Outmap allowed you to collaboratively create map layers, either privately in a group, or in public. Here's two use cases that illustrate what you could do with it:

• Crowdsoucing useful free wifi hotspots Back in 2009, finding free wifi that worked well was a mission. Suddenly, you weren't alone: you could set up an Outmap space, and tell it to watch a hashtag, for example #freewifi. Users could then tweet with an address with that hashtag, or add geo-information to the tweet itself. Outmap would also watch Flickr for that tag, and check for location information either in the Flickr metadata, or in the EXIF data in the photo itself. The data then could be mapped, or simply displayed in a list based on your current location.
• Gathering (or crowdsourcing) scientific data Users could add fields with types. For example, if you were doing wildlife counts, you could take your GPS-equipped smartphone into the field, and as long as it had a web browser that supported the Javascript Geolocation API, that would be all you'd need to record a result. Your Outmap space would tie your numeric recordings of wildlife data to the location where you recorded them. And because it was all social, and tied to individual user accounts, you could examine (or even remove) recordings made by particular individuals for full accountability.

It was based on data tied to individual location points, but further developments would have allowed you to group points into areas, in order to better support some scientific applications. And again, it was all web-based used existing web standards, all social, and (like Elgg) had per-item access permissions.

Outmap couldn't be released for reasons I won't go into here, and were unrelated to the mechanisms of the platform (but were related to the fallout from my decision to leave Elgg). Let it suffice to say, it was out of my hands. I abandoned work on it in 2010, and moved on to work on latakoo.

Geolocation has evolved since 2009, and I think we all now understand that the web is something that we can access from anywhere, and that pages can know about your geographic context. Some other use cases were covered by Google Maps and (particularly) Findery, which is building up a world of memories, found objects and resources on a shared map. Geoloqi, whose founders I really like and respect, supercharges the kinds of use cases Outmap supported (as well as many others). Meanwhile, there are (and were) professional GIS platforms that are extremely powerful and used for many scientific and industrial applications.

I still think there's a place for this kind of functionality in an enterprise social platform. Luckily, I'm now in a position to work on that. Watch this space.

#outmap #idno #elggmistakes #geolocation

This piece was submitted as a position paper for the W3C's Workshop on Social Standards: The Future of Business, due to be held on August 7-8, 2013, in San Francisco.

Really interoperable interoperability

Much has been written about both the power of APIs to connect social applications in powerful ways, and vendor lock-in in the context of those APIs. Rather than usher in a new era of interoperability and open computing, APIs have allowed vendors to create new ways to lock users into their ecosystems.

In many verticals, simply gaining access to a product’s API documentation is enough to require complicated licensing arrangements, vendor evaluation of your business intent for the API, and often, an asymmetric Non-Disclosure Agreement. “Open” is the new closed: too often, the API is a proprietary product in itself. This is as true in social software as it is elsewhere.

This proprietary nature carries multiple business risks. Not only does it require that customers invest heavily in a particular vendor’s products, but should that vendor subsequently decide to discontinue those products – as happened recently in the case of Google Reader and a number of Yahoo! products – the customer must repeat that investment in another platform. Finally, recent surveillance revelations must be food for thought for any business using proprietary services to host sensitive data.

Sophisticated open API standards mitigate these risks, but developing support for these can require a significant expenditure, and the business case may not yet be clear to most vendors. There is no doubt that they occupy an important place in the emerging social landscape, but not all vendors, or their customers, can justify the level of technical expense currently required to “buy in”. Indeed, given a high enough barrier to entry, even ostensibly open APIs may inadvertently have the same ill effects as closed ones.

Proving it

Although there have been significant advances in the field over the last five years, there remains a need to prove the business value of decentralized web technologies. To many of us involved in both the industry and the movement, this seems silly: after all, the business value of other decentralized technologies, like email and the phone system, are hardly questioned. Nonetheless, in a world where centralized data siloes regularly receive multi-billion-dollar valuations, the onus is on those of us who are building more open technologies to demonstrate their worth. Note, it is not enough to argue their worth: we must build, ship, and actively demonstrate a profitable product or service with a business model where the decentralized social web is an inextricable component.

I believe that these compelling business models exist, and that they are most easily discoverable in the enterprise. However, belief is not demonstration: we must continue to test and iterate them. During this exploration phase, this means that, our software and underlying protocols must be easy to write, adapt and change. Ease of development is more important than sophistication; we must not create our own technical lock-in before we even ship.

The IndieWeb

The “IndieWeb” movement was founded by Tantek Çelik, Amber Case and Aaron Parecki, around their annual IndieWebCamp event. Although it was originally created to encourage participants to self-host their own web presences (a laudable goal in itself), over the last year it has also begun to incubate a number of simple social web protocols based around Microformats 2 and Webmention.

At its simplest level, assets on the web are marked up with appropriate Microformats 2 classes, so that any parser may obtain a consistent JSON representation of their content. Linked targets on the page are then pinged using Webmention (or pingbacks), which alerts them to the presence of that content. They may then go back and parse the source of the ping, discovering content like comments, replies, event RSVPs and favorites. Adding more content types would be trivial, and indeed, more are emerging every week. If a content type is not registered, the target page may simply register that the source “mentioned” it.

An obvious further implementation incorporates signed HTTP requests for both parsing and Webmention pings, allowing for lightweight authentication so that protected resources can be selectively revealed.

The protocols and standards under development within the IndieWeb community offer some unique advantages for testing decentralized social models:

1. They piggyback on top of an open, decentralized system that everyone has already bought into: the web itself. Indeed, on the IndieWeb, where possible, the web is the API.
2. They are extremely simple to develop for, allowing you to concentrate on building well-designed tools that meet human use cases instead of building support for social protocols.
3. Social software need not be “monolithic”. Suites can be constructed out of small, compatible pieces, loosely joined.
4. Major search engines support Microformats, so marking pages up to be IndieWeb-compatible may also yield SEO benefits.
5. The IndieWeb community actively embraces participation in existing closed networks through a process called POSSE, minimizing the potential business impact for entities transitioning to a decentralized model.

POSSE - Publish (on your) Own Site, Syndicate Everywhere, as coined by Tantek Çelik – accepts that your friends, contacts or customers are easier to reach on the social platforms they’re already using. Therefore, content on your own, independently-hosted platforms syndicate out to your audience across the networks they already use; links point back to the originals. In the short term, it becomes immediately possible to experiment with decentralized social models without losing your existing audience. Over time, it may be possible to transition those audiences to consume and interact with your web presences in a more decentralized way, ensuring that you can post on your terms, and they can consume on theirs.

While most implementations of POSSE concentrate on consumer social tools like Twitter, Facebook, Flickr and Foursquare, there is no reason why the same principle could not be applied to commercial platforms like Yammer, Avid Interplay, GitHub, Salesforce or SocialText – or any proprietary service used internally inside any enterprise, APIs permitting.

Idno as an experimental testing ground

Idno is one embodiment of an IndieWeb-compatible open source platform that can be installed across many hosting environments. It was originally designed as a replacement for older open source networking platforms, but rapidly evolved into a testbed for many of the ideas the IndieWeb community was proposing.

At the time of writing, decentralized social web activities supported by Idno include:

• POSSE posts to Twitter, Facebook, Flickr and Foursquare, and replies on Twitter
• The ability to comment on, or reply to, a post (or multiple posts) on another IndieWeb-compatible site
• The ability to “like” a post on another IndieWeb-compatible site
• The ability to RSVP to events posted on other IndieWeb-compatible sites
• The ability to post content, including status updates, blog posts, bookmarks, photos, geographic “check-ins” and events that other people with IndieWeb-compatible sites may comment on, reply to, “like” or RSVP as appropriate

Due to its framework origins, Idno allows developers to easily build new post types. Indeed, support for events and RSVPs – at the time unsupported by any other IndieWeb-compatible software – were built in a single evening, with one developer, directly after an IndieWebCamp event. Other software produced by IndieWeb developers began to support events and RSVPs the next day. By the end of the week, at least three separate software platforms supported the content type. There is no doubt that the barrier to entry is low for individuals and businesses alike.

Conclusion

The rapid development that IndieWeb standards make possible is perfect for testing business models relating to the decentralized social web. This does not undermine the technologies and successes of the wider federated social web movement, or of other open social software projects; however, it does allow models to be tested much more quickly.

The relatively low barrier to entry of the IndieWeb also may encourage more developers to take part (as has already been shown), and as such, it seems likely that the standards that community is developing may find themselves in wide use for some time to come. An obvious analogy is RSS, which is not a sophisticated syndication standard, but saw widespread use due to its ease of implementation.

Many of the prevalent models for social software are hostile to the needs of both businesses and individual users. The IndieWeb aligns software developers with their users, while providing simpler tools for development, and encouraging both wider participation and more experimentation. I believe the result will be accelerated innovation in social software, and a much faster path to validating business models for the decentralized social web.

#indieweb #w3c #socialweb #business #idno

Syndicated to IndieNews

This site runs on idno: an open source social publishing platform that I've been working on for the past few months in my own time.

You may know that I co-founded Elgg, the open source social networking engine, which is used by the likes of Oxfam, NASA, the World Bank and several national governments as a social intranet and learning platform. The original thinking around Elgg happened a decade ago. Given that, you shouldn't be surprised to learn that my original thought experiment was: What decisions would I make if I was building Elgg today, in 2013? What would I do the same way, and what would I do differently?

Some technical decisions

I knew that I could make a faster social networking platform, with a better templating engine, and a much smaller codebase - even while sticking to PHP as an underlying scripting language. Partially that's because PHP 5.3+ is a much better development platform than earlier versions. It's also because there are now some well-tested, intelligent back-end frameworks, like Symfony 2, and front-end frameworks, like Bootstrap.

One of the major decisions I made when we built Elgg 1.0 was that not only was it a hassle for plugin developers to write their own database schemas - it was undesirable to the point of being dangerous. We effectively faked a NoSQL schema in MySQL by creating a data model around entities (first-class objects like users and blog posts), metadata, annotations and relationships. People were taken aback, and it was row-intensive, but it worked, and it continues to work today.

Nonetheless, today we have NoSQL, so #idno is based around MongoDB. This means there are far fewer database transactions involved - and adding new data to an object is incredibly easy. Together with a plugin architecture based on lazy loading, and Symfony's excellent observer pattern support, as well as the framework code I've built, I'm able to write a new plugin in an hour or two. That's important for a system I'm building in my spare time!

Meanwhile, all of the things about #Elgg that were great - a plugin architecture, granular access permissions - are intact. And on top of that there's a faster framework, and a responsive front-end that works really well in a mobile browser. Great!

But that's not the end of the story.

The #indieweb community has existed for years as a force to advance the state of the independent web, and to promote ownership of our own spaces. IndieWebCamp is an annual event for creators to discuss their platforms, technologies and ideas.

One of the big concepts to come out of #indieweb has been #POSSE: Publish (on your) Own Site, Share Everywhere. The idea is that your friends or followers shouldn't have to join your site to engage with you; you should be able to post on your own site and be read on Twitter, Facebook, Foursquare, or wherever they happen to be. idno has built-in plugins for status updates, blog posts, images, checkins and events. Correspondingly, it also has plugins to #POSSE this content to Twitter, Facebook, Foursquare and Flickr - and writing more would be trivial.

That's just as well, because I've committed to only post on my own site and copy to third parties (where that's possible).

Reinventing the social web

This year, though, something else happened. Using Microformats 2 (a way to very simply embed meaningful markup into any web page) together with Webmention (a way for any web page to lightly ping the pages it references), the community participants created the first indieweb decentralized comments thread.

Using nothing more than the markup on their own web pages and a very simple protocol, the participants created the basics of a decentralized social community, where each comment is hosted on its owner's own site, but nonetheless forms a coherent, easily-readable narrative.

This is a very big deal.

It's a completely different model to traditional social networking, where content typically doesn't bleed outside the walls of a specific social site. It's also different to previous decentralized social networking efforts, which have been in many ways more sophisticated, but much harder to join in with. Because a simple IndieWeb-compatible social tool can be built in an afternoon, just as a simple RSS-compatible tool can be built in an afternoon, these concepts have a much greater chance of succeeding.

Needless to say, idno is now a first-class participant in the decentralized IndieWeb social community. I've implemented IndieWeb comments, and moved immediately to also implement decentralized events that anyone can RSVP to, as well as decentralized likes. It also integrates with Firefox's brand new Social API.

You can browse the web and reply to any page, on a site that you truly own.

As more sites and platforms implement the IndieWeb social standards, those interactions will become correspondingly more social. For now, though, you can go ahead and interact with the web already.

Beyond that, idno will continue to develop over time as a community platform in itself. I'm using it here on my own site as a single-person publishing platform, but it doesn't have to be that at all, and all those Elgg-style features will continue surface as time goes on. But there's a big, wide web out there, and it's important to embrace that as widely as possible.

idno's homepage is here. Meanwhile, I continue to do work I'm proud of in my actual job, working for latakoo to facilitate media storage and transfer for video professionals and the broadcast news industry. We're talking about using decentralized social networking there too - but more on that another time.

There's been a lot of news stories lately about how technology companies like Google, Apple, Microsoft and Yahoo! may or may not be giving your data to the #NSA as part of a project called #PRISM. They deny it, news outlets confirm it and it's hard to tell what's real and what isn't real.

Whatever your political views, or whether you think government entities should be able to snoop on your phone and electronic communications, it's hard to argue that the sheen hasn't come off the consumer Internet industry. As the economist Umair Haque said earlier today:

15. There is nothing more lethal to an economy than the idea that people have nothing more to contribute than "information".

— umair haque (@umairh) June 7, 2013

The large online services have created a world where, despite the breadth of software's possibilities, the scope of our communications are limited. On Facebook, you can post status updates, links, photos and videos; on Google+, you can post status updates, links, photos and videos. What if you want to post a game, or an interactive multimedia presentation, or a live graph connected to real-time data?

Those things are hard for centralized services, because they've got to concentrate on common denominator forms of content - like status updates and photos - but they're much easier when you control your own site. If you could install your own publishing app as easily as an app on your phone, and then add new ways of posting stuff to that site just as easily, suddenly you would be able to make your presence on the web your own. And you could let other people in - you could create online communities that fit your needs, rather than bending your communities around the limits of a Facebook group or a mailing list.

That's what inspired me to start working on idno, and these are some of the ideas that inspired other people in a community of developers called the #indieweb to build their own sites and platforms. We don't believe in treating people as data points; we believe in user-centered software for individuals. Software that you control.

#idno will be available to install onto your own site later this month; a turnkey hosted version will follow. In the meantime, if you're a developer, you can check out Idno on GitHub, or some of the other IndieWeb projects out there.

Won't you join us?

I just installed a vanilla configuration of idno onto my new website at werd.io.

idno is a personal attempt at building a publishing platform that adheres to IndieWeb principles: own your own data, publish on your own site, use existing social websites for dissemination but not as an origin. I've also tried to use microformats where possible, allowing every page to contain lightweight semantic information, as well as making it skinnable, extensible, and social.

I also wanted a chance to think about platform design again, separately from my work at latakoo (where I'm CTO) and informed by my previous work on Elgg (which I helped start nine years ago). What would I do if I was building a platform from scratch in 2013? How could I make it more useful?

I'm proud of the results so far, and the decisions I've made. Here are some highlights:

idno is based on Activity Streams, and every page is an API call

If you visit the front page of the site, or click on any user profile, the feed you see is an Activity Stream (albeit rendered to HTML). You can easily toggle a JSON version by setting application/json to be your web client's accepted type. There's automatic RSS, of course, too. And if entities have attachments - for example an uploaded file or a photo - the RSS items have automatic enclosure tags, as you'd expect. (That means idno can host podcasts and deliver other files via RSS.)

The system is designed so that all plugins automatically have an API without having to do any extra work. I'm also planning on implementing HTTP signatures, or something like them, to authenticate seamlessly in addition to standard sessions.

idno uses HTML5 (and Bootstrap, jQuery)

HTML5 needs no introduction or reasoning. Bootstrap is a very handy set of rails for responsive front-end design (with lots of pre-existing themes), and jQuery is an integral part of that. On top of that, idno bakes in some useful features behind the scenes - like autosaving works in progress using your browser's local storage, for example.

idno runs on PHP 5.4 with a MongoDB back-end.

PHP is a controversial decision, but it remains one of the most widely-deployed web scripting languages in the world. Version 5.4 is old enough to be installable on most servers, but new enough to contain some really neat features. Idno makes extensive use of the JsonSerializable interface. (It also uses a lot of 5.3 features like lazy loading and namespaces.) MongoDB, meanwhile, means that we don't have to force plugin writers to maintain schemas, nor to emulate NoSQL in MySQL, which has been a tactic I've used in the past. So far it's performing great, and it also provides GridFS, a useful file store.

All of this a work in progress, which has mostly been coded late at night, and there's still a lot to do. For example, at the time of publishing, the hashtags below aren't linked anywhere and aren't automatically marked up, but I bet if you check back in a few days, they will be. Another is that although my posts are automatically POSSEd out across the web, I can't yet posts things like replies to tweets from the idno interface. Nonetheless, I invite the brave to check out the GitHub repository (idno is released under an Apache 2.0 license), and let me know what you think.

#indieweb #indiewebcamp #idno #opensource