Archive Page 2
Tags: AS3, ASP.NET, FLEX, oAuth, openID, WRFS, XRDS, XRDS-S
An excellent in the wild post written by Josh Patterson about his floe.tv project with some feedback from me, so the credit really goes to him. For those of you unfamiliar with this series of posts, the idea is to create blog-length interviews with various in the wild apps describing their processes and the technologies that they use with regards to data portability. The goal is to profile real use cases, solutions, and lessons learned when it comes to the current state of affairs for data portability technology. Note that these posts aren’t meant to recommend or not recommend certain technology, I leave that up to the developers/architects to decide based on their needs. If you have such an app and are interested in being interviewed, please leave me a comment on one of my posts and I will get in touch with you.
Basically with floe.tv we started out just wanting to try out some video ideas we had. We started out with a simple playlist of videos on the internet linked together at a site, which evolved into a full own video editor. After a number of months of development we hit a point where the team sat down with some users and did a testing session, asking questions and gauging responses to see how well we were hitting our marks. We came to the subject of data storage, local hardrives, and getting media online, and just as a thought exercise I asked “well, what if floe.tv just knew about all your online media by your login name, and referenced it automatically in your libraries the first time you logged in — just as if it was an app installed locally on your hd?” and immediately both of them became excited and one asked “can I do that right now? when can I use that?” and I knew from experience that the market was speaking very loudly and clearly in my direction, and that I had better listen very closely.
The very next meeting I posed this question to our team:
What if our app was “inherently installed” in the internet? What if someone logged in, and the app just acted like a desktop app that “knew” about your flickr images, your youtube videos, it knew about your myspace friends, facebook friends, and automatically treated them as one logical database, one logical social graph? And someone started right into an app tutorial right off the bat with their contacts, files, and assets already referenced (but fully respecting privacy, control, etc)?
So the next question naturally becomes “that all sounds really great, but … how do we get there?”
From there we really began to push a “what if this/that” scenario, and drew up our ideas into a document entitled WRFS and from that we began to re-engineer the floe.tv app towards a truly linked data experience.
We are currently using FLEX/as3 for the editor and player with ASP.NET for the server technology. Discovery is a big deal with how I view next gen web apps — dynamically finding data at runtime without having to go through “Big Data’s” walls to get to it. How I see this happening is
- the user logs in with an identity url, be it an openID or a XRDS file location. floe.tv is an openID only application, but user’s can map multiple openIDs to one account with it.
- the app authenticates the user, and then uses multiple fallback methods to find the XRDS-S file if not specified.
- The XRDS file is parsed, and relevant data types (here image and video) location uris are pulled out
- Each uri points to a data container, which is then queried via its API for a list of resources the user has stored there.
- these results are then aggregated back together into a single “recordset” to be returned to the floe.tv application layer
The fallback methods for XRDS discovery (done in the FLEX client) are ordered as:
- First do basic yadis discovery to see if the identity delegate is the openID provider or a blog of some sort head link delegation setup. Either the openID provider or the XRDS location may here, or both. In some cases, such as the DiSo XRDS plugin, the XRDS file located in the head link tag will have the openID provider location.
- The secondary method we have kicked around is to query the openID provider for an Attribute Exchange key that points to the XRDS file. This is not well defined yet but has been discussed amongst various groups.
- Lastly we fallback to having our flex app prompt the user for a XRDS url so that we can “enhance their user experience”.
So although I think our secondary option with the AX key is a little shaky right now, overall we degrade gracefully.
For a quick demo of how some of the data query and aggregation mechanics might work, I’ve built a short demo illustrating the step by step mechanics.
We aren’t done with this application, obviously, and a lot of work remains to be done – I should note that I am currently at Step 2 of 5 as stated above. However, the application is evolving into the embryo of what I think a linked data application can and will be. What I can share are the places that we are actively looking for solutions, simple decentralized solutions, that solve these issues.
One thing that is slowly changing is the perception of cross domain scripting on the internet. As services get more complex, and require more aggregation of data from multiple sources, we are going to push more data handling duties to the client, as scalability will suffer. For flash crossdomain scripting we need the crossdomain.xml file present on the server of the api would like to call for it to work. This is a trivial thing to setup as it consists of a simple xml file located off of the subdirectory level you wish to grant access to.
Once this file is exposed, the flash runtime will then allow calls from the client to those servers.
We’ve been waiting to see how the discovery wars pan out, as it stands now XRDS-S is looking like the service index of choice amongst the big players (presently Yahoo is endorsing it, hiring Eran Hammer Lahav as their Open Standards Evangelist. How the XRDS resource is discovered automatically without a tremendous amount of user interaction is something that we are taking many approaches towards, as discussed above. For now we’re going to focus on finding XRDS files as our catalog of service endpoints for a user. The DiSo project is going to be publishing its user’s service endpoints in the XRDS format and already has a plugin for it, so I think in the short term we’ll be focusing on consuming that data in terms of an early conceptual demo in runtime linked data.
Once we find the XRDS file, we aren’t out of the woods. How do I set my XRDS file up so that I can tell the floe.tv application that I have “images in flickr”? This is a fundamental question being setup and worked on at the site http://www.xrdstype.net/ by many different groups and people, and has yet to fully be fleshed out.
There’s also the issue of once we find a data endpoint, how do we talk to it? ATOM? Some sort of standard data interop api that spits out RDF? In a perfect world I’d love to see a self organizing web, a linked data web that can find a data endpoint at runtime, find its semantic schema, and wire itself such that it can talk to that api without ever needing user intervention — it simply will understand how to talk to it.
Another key development will be the permission system, possibly using OAuth+Discovery to automate the updating of a XRDS-S file for someone when they add data to a service. I need to learn a lot more about OAuth and the direction its heading. I’d prefer a world where the user doesn’t need to manually go to a site to allow their resources to be used by a third party, but for now, this is how we have to operate.
So really, I think I have more “lessons underway” than “lessons learned”, but sharing this information is key since it sparks interest in other like minded developers who may know a lot more about some of these areas than me. There’s going to be some places that we punt and just hard code some scaffolding code in to just get going, but over time I’d like to evolve towards a linked data web that auto discovers new connections at runtime and self organizes to give a smarter and far more intuitive user experience than we’ve seen so far.
If you are interesting in linking data or trying some data interop experiments, please feel free to email me (jpatterson at floe.tv) or check out the WRFS workgroup or my blog http://jpatterson.floe.tv
Feedback and suggestions are welcomed.
Tags: FOAF, hCard, hFOAF, Microformats, XFN
No, it isn’t a proposition, so get your mind out of the gutter 🙂 FOAF stands for Friend-of-a-Friend and is a RDF taxonomy capable of describing a person and his/her friend(s) in far more glory details than XFN, in fact it is a competing standard. Reading your mind, you are probably going to ask me, what is RDF? RDF (first published specs in 1999) stands for Resource Description Framework which means absolutely nothing to most people however it is often touted as the foundation for the so-called semantic web. If you want a good introductory RDF post, check out What Is RDF. A synopsis of RDF extracted from this ReadWriteWeb post Semantic Web Patterns: A Guide to Semantic Technologies
RDF is a powerful, graph-based language for declaring things, and attributes and relationships between things. In a simplistic way, one can think of RDF as the language that allows expressing truths like: Alex IS human (type expression), Alex HAS a brain (attribute expression), and Alex IS the father of Alice, Lilly, and Sofia (relationship expression). RDF is powerful, but because it is highly recursive, precise, and mathematically sound, it is also complex.
On a high level, FOAF has the following categories
- Person’s basic information such as name, nick, title, homepage, email, surname, first name, given name, email (expressed as mbox), etc.
- Person’s extended information such as blog, interest, project, school home page, work home page, friend(s) he knows, etc.
- Person’s online accounts (includes instant messaging)
- Person’s projects and groups
- Person’s documents and images
As you can see, it allows for extensive coverage of a person’s information. If you want the full details, check out FOAF Vocabulary Specification 0.91. Assuming you have a basic knowledge of XML, let’s take a simple example to illustrate how FOAF is actually written.
The above XML snippet describes a person’s name and email, obviously John Doe is not a real name and email@example.com is not a real email but you get the idea. Technically this describes a person, not the “me” identity as defined by XFN rel-me.
Let’s take a more slightly more complex example
Basically the above FOAF says there is a person John Doe, his email is firstname.lastname@example.org, he “knows” Jane Smith, and her email is email@example.com. Note that “knows” does not necessarily equate to a friend, it simply means that you know someone, that person could be a friend, lover, co-worker, parent, sibling, or someone you met online but not in person, etc. For a more in-depth look at FOAF, check out XML Watch: Finding friends with XML and RDF.
Ok, now that you know how to create basic FOAF, what next. Unlike XFN / hCard / microformats, FOAF is delivered via a file separate from any HTML markup. Creating a FOAF file is not something that a Joe Blow user will ever do manually. However, if you are curious and want to play around with creating FOAF, you can use this handy tool FOAF-o-Matic. For most users, their service providers, the ones that support FOAF, will handle all the intricate details behind the scene. For example, if you have an account at MyBlogLog, you can check out their FOAF feature by appending “/foaf” to the end of your MyBlogLog member URL. If you aren’t a MyBlogLog member, here is Ken Brewster’s MyBlogLog FOAF file, http://www.mybloglog.com/buzz/members/kentbrew/foaf/, and check out his fun FOAF consuming app, picture below
Since FOAF is a separate file, you have to link to it from a HTML page by adding the following HTML code within your <head> tag (using Kent Brewster’s FOAF URL)
In MyBlogLog’s case, that HTML code is added automatically to a member’s profile page. I should note that in MyBlogLog’s case, the FOAF file is publicly available without any authentication allowing any search engine to crawl and index it, in fact, this information is available through Google Social Graph API but beware that there is a current limitation with Social Graph API in that Googlebot does not crawl a FOAF file linked from an HTML page, hence not making it available for the API to parse. In Brad Fitz own words,
I just verified that the problem is the crawl coverage. Historically Googlebot hasn’t cared about FOAF because it hasn’t used FOAF, so why hit your server fetching it just to throw it away? Now that the SGAPI is using it, though, I need to ask Googlebot to go get that FOAF that I know exists but is uncrawled.
A few parting thoughts…
- Chris Messina thinks that FOAF baking in relationship information is a fundamental flaw, I have to respectfully disagree because I think it is a key missing feature from XFN/hCard, at least when it comes to my XFN consuming app. Upon further reflection, I think I understand Messina’s concern about FOAF being potentially convoluted. It is due to the powerful recursive nature of RDF that can lead to circular references and if a FOAF parser doesn’t account for this, it can lead to infinite loops.
- Since FOAF only has a “knows” property, it is less descriptive, hence less useful, than XFN rel values (friend, acquaintance, co-worker, neighbor, sibling, parent, child, crush, date, etc)
- Apparently FOAF does not have properties to specify an address, e.g.
state/province, country, city, street address, quite odd indeed. However, it can be remedied by adding VCard namespace to RDF.
- The general vibe in the web development community is that FOAF is more complicated than XFN so it is more likely to see XFN in the wild than FOAF
- Firefox Operator plugin does not consume FOAF information
- FOAF information is meant for app-to-app consumption unlike XFN / hCard which are embedded in HTML. Check out this post WordPress, FOAF, OpenID – updated for a WordPress plugin retrieving FOAF profiles when users authenticate a wordpress blog with OpenID.
- There is a recent effort, hFOAF, to combine FOAF, hCard, and XFN
Quoting Burger King slogan “having it my way”, last week was a flurry of announcements, arguments, and personality clashes among tech pundits discussing, or rather arguing, the implications of user privacy following announcements from 3 big companies, MySpace, Google, Facebook, each announcing remarkably similar data portability features. I won’t belabor the details of each announcement because they have been covered in great details.
A controversial thing that came out of the announcements is Facebook blocking access to its users data from Google Friend Connect. The official reason cited by Facebook is that Google Friend Connect violated their TOS with regards to respecting their users privacy. IMO the real reason, as pointed out by several tech pundits already, is that Facebook wanted to continue to wall in their users data while conveniently citing the TOS privacy concern as the reason to block access.
What transpired afterward is even more controversial as several tech pundits, Steve Gillmor, Sam Whitmore, Marc Canter, Dana Gardner, Mike Arrington, Mike Vizard, Robert Scoble, self-invited guest Chris Saad (Gillmor’s words, not mine), weighed in about user privacy in general, and really ripped into each other at a Gillmore Gang conference call. You can read related posts on the subject at
- Battle Over Data Ownership on Gillmor Gang, Michael Arrington
- Facebook has a point where it comes to your privacy, Robert Scoble
- How SHOULD dynamic privacy work?, Marc Canter
Arrington and Scoble duked it out here Data Portability: It’s The New Walled Garden and later on in the Gillmor Gang conference call. I have also seen similar mulitiple discussions raised at DataPortability.org without any conclusion.
Details notwithstanding, I had a epiphany about user privacy while trying to sort through the numerous and different opinions about what constitutes right and wrong user privacy controls. It dawned on me that user privacy is a personal and individual thing and is also dependent on the context of usage. IMO, this means no one can or should tell me what I consider private or not. Hence it is not meaningful to define a universal bill of rights for user privacy that works for everyone.
Let’s take the hotly debated example between Scoble and Arrington. Arrington’s position is that his data is his data and he should have the ultimate control over how that data is used which includes the ability to stop someone like Scoble with whom he has shared his email with to not use it on 3rd party services like Plaxo without Arrington’s explicit permission. Scoble’s counter argument is that once Arrington shares his email that he has given implicit permission for Scoble to use it elsewhere. He cited the examples of adding Arrington’s email to his Gmail and Yahoo mail accounts so he can email Arrington from either email account and Arrington has no issue with that. However, Arrington has an issue if Scoble wanted to expose Arrington’s email to a 3rd party service like Plaxo which according to Arrington has a tendency to spam people. If Scoble doesn’t care that Arrington use his email on Plaxo, then that’s Scoble’s right to not care, but if Arrington has an issue with Scoble using his email on Plaxo, then that’s Arrington right to care. Note that Arrington’s reaction is context based because he has no problem with Scoble adding his email to Gmail and Yahoo accounts, which brings me back to my point about user privacy being a personal and individual thing and is also dependent on the context of usage.
So how does this translate to implementing the right user privacy controls. IMO, a service should implement privacy controls ranging from air tight to I honestly don’t care who sees and uses my data along with a set of sensible defaults, e.g.,
- I am a private person and I want to explicitly approve every use of my data, WARNING: this might result in excessive permission requests but it’s what you asked for, think Microsoft Vista User Account Control feature.
- I am ok with sharing some but not all of my data with my friends and the world. Here’s where the sensible defaults come in, the challenge is to define a set of sensible defaults that makes sense for a particular service but needs to be contextual aware or requires contextual approval.
- I love attention and frankly don’t care who sees and uses my data. WARNING: you might regret this later on when you run for the post of US President or mayor when there are bikini pictures of you on the web but that’s a risk you accept.
I suspect most users will pick option 2, and that’s where the real fun begins, defining sensible defaults for each service while allowing for contextual awareness or approval. It’s interesting to note that in his post on How SHOULD dynamic privacy work?, Marc Canter also mentioned the need for privacy controls to be contextual aware although his example is different than mine. I like my example better =)
The above perspective is offered purely, and selfishly, from a user’s point of view without consideration for why big companies like MySpace or Facebook would want to provide free service and making their users data fully portable, hence loosing a competitive advantage. Does a service have a right to users data for providing free service in return, I think so. However, if the industry mindshare is moving towards making data portable, big companies are forced to go along so they don’t get left behind and that is exactly what Google, MySpace, Microsoft, and Facebook did though with varying degrees of data portability but it’s a good start.
One last thought, I believe it is Gillmor that posed this argument, when you sign up for a free service and agrees to their TOS, you have agreed to the terms that they laid out regarding usage of your data, so it’s a done deal and you have to abide by the terms you agreed to. First off, to be realistic, very few people read TOS in its entirety, if you do, you are in the minority. As they say, the devil is in the details which you unwittingly agreed to without reading in full because TOS are designed to be purposely mind numbingly boring and unnecessarily long to discourage a user from reading it in full and providing CYA coverage. And even if you read it in full, it’s not easy to decipher the legal jargon thrown in for further confusion. IMO, while this is technically a correct argument, it is not an effective argument.
Tags: hCard, Microformats, XFN
Just yesterday, I was writing about rel=me, XFN, and microformats, check out the post A simple data portability project or is it if you haven’t read it already or just want some background on rel=me, XFN, and microformats.
Today I plan to attend the monthly DataPortability meetup at LinkedIn office in Mountain View. For directions, I would normally go through a series of steps to open a new tab, load google maps, find LinkedIn, etc. However, with my new found microformat knowledge, I immediately noticed that the Firefox Operator plugin lit up when the event page is loaded. As it turns out, upcoming.yahoo.com, the event service provider in this case, supports microformats.
Here’s the Operator showing the contact details for the event
To map it out, I clicked on “Find with Google Maps” and presto, it shows up on Google maps, very cool. Note that the contact information is using the hCard microformat. And hCard is actually based on another standard, vcard. hCard is just vcard expressed in HTML format. Here’s the actual HTML for that information
The hCard bits are represented in class="street-address", class="locality", class="region", etc., attributes.
Here's the Operator showing the event calendar details
If you wish to add this event to your favorite calendar like Outlook, click on "Export Event" or perhaps "Add to Yahoo calendar" or "Add to Google calendar".
Here's the Operator showing the various event tagspaces
Tagspaces is also another microformat standard but so far it hasn't been useful to me as a user. Tags are widely implemented in popular social networking sites, though in some places they are called keywords, e.g., if you add a video clip from the movie "Rush Hour 3", you can tag it with "Jackie Chan", "Martials arts", "Kungfu", "comedy", etc, you get the idea. The more tags you add, the easier it is for someone else to find it.
However, it hasn't translated well with most of the tags I have seen so far. Take a look at the tags for the event page, it lists "dataportability", "microformats", "relme", "upcomingevent472061". For "dataportability", it then list sub options for "Find products on Amazon.com", etc. Without putting my technical hat on, I would be totally puzzled by "dataportability", "microformats", "relme", "upcomingevent472061", let alone find "dataportability" product on Amazon.com. In fact, I tried to find dataportability product on Amazon.com and guess what, it came up zippo as it should because there is no such thing. However to a first time user seeing and trying it, a likely first impression is that it is useless and broken. On other microformat capable pages, I saw duplicate (sometimes 3 or 4 times) tags making it even more confusing, this probably could have been handled by Operator removing the duplicate entries though but there shouldn't be dups in the first place. Anyway, I do think this is a useful feature but it requires more usability thoughts for Joe Blow users.