Joe Lamantia.com

The good peo­ple at Chi Nether­lands just posted video of my talk “Play­ing Well With Oth­ers: Inter­ac­tion Design and Social Design for Aug­mented Real­ity” at the Web and Beyond 2010 here in Ams­ter­dam in June.  It’s couched as a col­lec­tion of design prin­ci­ples for the oncom­ing cat­e­gory of social aug­mented inter­ac­tions made pos­si­ble by the new medium of aug­mented real­ity.  But this talk is also a call to action for all mak­ers of expe­ri­ences for the emerg­ing engage­ment space of every­ware to focus on the human and the humane per­spec­tives as we explore the new inter­ac­tions made possible.

The out­line of the talk is roughly:

1. Overview of aug­mented reality
2. Social inter­ac­tion per­spec­tive on cur­rent AR experiences
3. Def­i­n­i­tion of ‘social aug­mented experiences’
4. Com­mon inter­ac­tion design pat­terns for AR
5. Social ‘anti-patterns’ lim­it­ing design of aug­mented experiences
6. Design prin­ci­ples for social aug­mented experiences

(The audio qual­ity is quite good, and the cam­era­man cap­tured most of the slides nicely — so this is a record­ing worth watching.)

This year’s TWAB fea­tured sev­eral talks on aug­mented real­ity, ubiq­ui­tous com­put­ing and related top­ics; you’ll find record­ings of these on the Chi Ned­er­land Vimeo chan­nel: http://vimeo.com/chinederland

Many thanks to the orga­niz­ers and vol­un­teers for putting on such a well-run event!

TWAB2010: Joe Laman­tia — Play­ing well with oth­ers: inter­ac­tion design and social design for aug­mented real­ity from Chi Ned­er­land on Vimeo.

Thanks to all who came to the Muziekge­bouw on a lovely early sum­mer day to talk about the emerg­ing engage­ment space of social aug­mented expe­ri­ences for the third edi­tion of The Web And Beyond con­fer­ence in Amsterdam.

For ref­er­ence, here’s the ses­sion descrip­tion from the offi­cial program:

Aug­mented real­ity blends the real world and the Inter­net in real time, mak­ing many new kinds of prox­im­ity, con­text, and loca­tion based expe­ri­ences pos­si­ble for indi­vid­u­als and groups. Despite these many pos­si­bil­i­ties, we know from his­tory that the long term value and impact of aug­mented real­ity for most peo­ple will depend on how well these expe­ri­ences inte­grate with ordi­nary social set­tings, and sup­port every­day inter­ac­tions. Yet the inter­ac­tion pat­terns and behav­ior we see in cur­rent AR expe­ri­ences seem almost ‘anti-social’ by design. This is an impor­tant gap that design must close in order to cre­ate suc­cess­ful AR offer­ings. In other words, much like chil­dren going to school for the first time, AR must to learn to ‘play well with oth­ers’ to be valu­able and suc­cess­ful. This pre­sen­ta­tion reviews the inter­ac­tion design pat­terns com­mon to aug­mented real­ity, sug­gests tools to help under­stand and improve the ’social matu­rity’ of AR prod­ucts and appli­ca­tions, and shares design prin­ci­ples for cre­at­ing gen­uinely social aug­mented expe­ri­ences that inte­grate well with human social set­tings and interactions.

I’m post­ing slides for my ‘Under­stand­ing Frame­works’ por­tion of the Beyond Find­abil­ity work­shop on strate­gic prac­tices just given at the 2010 IA Sum­mit.  This por­tion of the full-day pro­gram empha­sizes under­stand­ing and iden­ti­fy­ing the com­mon things that make up a design frame­work, con­cen­trat­ing on the sim­ple struc­ture that design­ers need to grasp in order to cre­ate their own effec­tive frame­works for solv­ing design chal­lenges. I hope you find it infor­ma­tive and useful!

I’ve posted my slides for the Where 2.0 panel “The Next Wave of AR: Social Aug­mented Expe­ri­ences” orga­nized by Tish Shute. After a review of the cur­rent state of aug­mented real­ity expe­ri­ences in terms of the social inter­ac­tions sup­ported (using the met­ric of ‘social matu­rity’), it shares 9 prin­ci­ples for cre­at­ing social AR expe­ri­ences that peo­ple will enjoy and value.

Spe­cial points to those who spot the embed­ded April Fool’s joke…

“Beyond Find­abil­ity: IA Prac­tice & Strat­egy for the New Web” the full-day work­shop that debuted at the 2009 IA Sum­mit, is back for 2010.   Fea­tur­ing an expanded lineup that includes Andrew Hin­ton, Matthew Milan, Chris­t­ian Crum­lish, Erin Mal­one, Cindy Chas­tain, and me, the work­shop explores lead­ing edge the­ory and prac­tice to equip expe­ri­ence archi­tects for the chal­lenges of design­ing social expe­ri­ences, the DIY Inter­net, engag­ing busi­ness strate­gi­cally, and more.

Read the full descrip­tion here, and then reg­is­ter here!

Bonus: All work­shop atten­dees will receive a free copy of Social Mania — the social pat­terns design card game unveiled at IDEA09.

Last year’s ren­di­tion was pos­i­tively invig­o­rat­ing, with par­tic­i­pants from experience-based busi­nesses like Zap­pos, and prac­ti­tion­ers from lead­ing firms like Adap­tive Path.  But this one goes to eleven: we hope you’ll join us!

I’ve posted the slides from my tuto­r­ial / work­shop Design For Goals at JBoye 09 on slideshare: they’re embed­ded below.

The struc­ture for this tuto­r­ial is part method review (on how to under­stand people’s goals in a struc­tured way), and part shar­ing of re-usable pat­terns found after research­ing goals.   Since the con­text of ori­gin for both the goals and pat­terns was com­plex inter­na­tional finance, some trans­la­tion of the raw mate­ri­als and exam­ples and the syn­the­sized pat­terns into a realm closer to home for ordi­nary peo­ple is likely in order.

As you’re going through the slides, I sug­gest using your own activ­i­ties that involve infor­ma­tion find­ing and mak­ing sub­stan­tial finan­cial deci­sions as a ref­er­ence.  Not all the exam­ples that I selected as the basis of exer­cises dur­ing the tuto­r­ial made across the cul­tural bar­rier between North Amer­ica and North­ern Europe: I was sur­prised at how many peo­ple (in a pro­fes­sional audi­ence) have never bought house or car…  Which proves yet again that this is one of the areas for user expe­ri­ence design to work on as a discipline.

And as we had a small, noisy, and rather warm room right after lunch, I should say big thanks to all the par­tic­i­pants and vol­un­teers — every­one — who made an effort to engage.

Even design edu­ca­tion is a work-in-progress, it seems.

The most impor­tant ques­tion fac­ing the aug­mented real­ity com­mu­nity — one whose answer will shape the future of AR — is con­tent cre­ation. Put sim­ply, it’s a ques­tion of Who can cre­ate What kind of con­tent, and How they will cre­ate it.  At the moment, a notice­able gap sep­a­rates those who can cre­ate AR expe­ri­ences from those who can­not.  High bar­ri­ers to entry in the form of skills, tech­nol­ogy, or expense like those in front of AR are accept­able for a new medium at the early stages of devel­op­ment, but in the long run, mak­ing it easy for all those peo­ple who don’t know a fidu­ciary marker from fidu­ciary trust to eas­ily cre­ate valu­able expe­ri­ences for them­selves and oth­ers is far more impor­tant to the via­bil­ity of AR than resolv­ing any of the many con­cep­tual, design, or tech­no­log­i­cal chal­lenges vis­i­ble at the moment.

In fact, unless the AR com­mu­nity makes it easy for ordi­nary peo­ple to cre­ate and share mean­ing­ful con­tent broadly, I wager aug­mented real­ity will remain a marketer’s over­worked dray horse in the near and mid­dle term future. And in the long term, aug­mented real­ity expe­ri­ences will become at best an inter­face lens [as Adam Green­field sug­gests here] sup­port­ing spe­cial­ized visu­al­iza­tion needs and a lim­ited range of inter­ac­tions (with cor­re­spond­ingly lim­ited value), all built around resources orig­i­nat­ing from else­where within the ubiq­ui­tous dig­i­tal expe­ri­ence ecosystem.

I think this is a ‘neg­a­tive out­ome’ for AR only because I see so much poten­tial. As a class of expe­ri­ences, aug­mented real­ity has the poten­tial to change our under­stand­ing of the world we are immersed in at every moment, but only rarely appre­hend in a way that makes informed inter­ac­tion with peo­ple and the envi­ron­ment pos­si­ble. As Tish Shute noted in her recent inter­view with Bruno Uzzan, I see the col­lec­tion of tools, tech­nolo­gies, and con­cepts affil­i­ated under the ban­ner of aug­mented real­ity as the lead­ing ambas­sador for ubiq­ui­tous com­put­ing and the weird world of every­ware that is ris­ing around us.

Recent devel­op­ments show progress towards bridg­ing the gap. First is Mobilizy’s pro­posal of a com­mon markup lan­guage — ARML [Aug­mented Real­ity Markup Lan­guage], based on KML — to the Aug­mented Real­ity Con­sor­tium.  Set­ting aside all other ques­tions about ARML, the pri­mary con­tent cre­ation prob­lem I see with this approach is the explic­itly geo­graphic frame of ref­er­ence in KML.  Most peo­ple sim­ply do not think in the same terms used by geoloca­tive schemes.  When I ask how far it is to the mar­ket, and some­one replies “4 min­utes north”, they’re not think­ing in min­utes of lat­i­tude.…  But rather than attempt to reori­ent the GIS / GEO loca­tion world­view to one that’s more nat­ural in human terms, I think the prag­matic solu­tion is a trans­la­tion layer in the cre­ation expe­ri­ence that avoids coor­di­nates or other non-natural lcoa­t­ive schemes, much as domain names over­lay or bro­ker IP addresses.  As an exam­ple, recall how the travel ser­vice Dopplr prompts you to enter the name of a place, sug­gests likely matches from a library of defined and man­aged place names, and only then addresses the coor­di­nates asso­ci­ated with that location.

In addi­tion, ARML will need some sort of abil­ity to cap­ture markup that is *not* depen­dent on geo­graphic ref­er­ence.  This may seem coun­ter­in­tu­itive for a medium that aims to aug­ment real­ity (which is, after all, a place), but remem­ber that peo­ple also ori­ent them­selves in terms of other peo­ple, time, activ­ity, iden­ti­fier, etc.  Hang­ing every­thing that aug­ments real­ity off of the geo­graphic skele­ton will result in instant ref­er­ence scheme hack­ery on an immense scale.  At the least, AR con­tent cre­ation expe­ri­ences based on ARML will need some means of invok­ing other ref­er­ence schemes.

The sec­ond devel­op­ment is Layar’s launch of buildAR.com, a pub­lic web-based con­tent cre­ation tool that sup­ports map based inter­ac­tion that extends the model for cre­ation expe­ri­ences beyond coör­di­nate tagged text data.  BuildAR.com is an early stage tool, but it marks a step toward the evo­lu­tion towards the goal of reflex­iv­ity; the stage of matu­rity wherein it is pos­si­ble for peo­ple who are unaware of the struc­ture and con­cepts that define the medium to eas­ily use tools pro­vided within the medium to cre­ate expe­ri­ences.  In McCLuhanesque terms, this effec­tively entails mak­ing pro­vi­sion for using the medium to extend itself.

I’m talk­ing about both direct and indi­rect cre­ation path­ways for aug­mented con­tent, though the empha­sis is on the direct end of the con­tin­uüm.  Indi­rect cre­ation could take many forms, such as trans­lat­ing exist­ing geoloca­tive tags or append­ing ARML meta­data to exist­ing dig­i­tal con­tent items; per­haps social objects like pho­tos, tweets, hotel reviews, or recipes.  Or con­tent that is cre­ated as a result of Google Wave, or the instru­men­ta­tion of urban set­tings, and our basic eco­nomic processes.  (A deep dive into the ques­tion of direct vs. indi­rect con­tent cre­ation path­ways would require map­ping out the poten­tial aug­mented con­tent ecosys­tem of linked data, and assess­ing each type of data from the cloud of apis / ser­vices / sources using tbd criteria.)

Address­ing the con­tent cre­ation gap is crit­i­cal because enabling broad-based cre­ation of aug­mented expe­ri­ences will speed up exper­i­men­ta­tion for all the sup­port­ing mod­els that need to evolve: busi­ness and rev­enue, data own­er­ship, tech­ni­cal, con­cep­tual, etc. Evo­lu­tion is needed here; the early mod­els for con­tent cre­ation include adver­tiser only (a default in the exper­i­men­tal stage for media where mar­keters and adver­tis­ers are pio­neers), sub­scrip­tion based, open source, and non­profit (aca­d­e­mic and oth­er­wise).  None of these yet offers the right com­bi­na­tion of con­ve­nience and con­text, the implaca­ble twin giants who rule the domain of value judg­ments made by dig­i­tal con­sumers and co-creaters.

Guide­lines for Con­tent Cre­ation Experiences

So what should the AR com­mu­nity offer to close the cre­ation gap?  We’ve learned a lot about what works in broad-based con­tent cre­ation from the evo­lu­tion of blog­ging and other main­stream plat­forms for social inter­ac­tion.  With­out con­sid­er­ing it exten­sively, the guide­lines for a con­tent cre­ation expe­ri­ence (mind, I’m not dis­cussing the tech­ni­cal enablers) are:

• No cost of entry: Cre­at­ing con­tent can­not require spend­ing money (at least for basic capa­bil­ity), as the effort involved is already an investment.
• No cog­ni­tive over­head: Cre­at­ing con­tent can­not require under­stand­ing new abstract con­cepts, mas­ter­ing tools with low usabil­ity, learn­ing com­plex lan­guages or ter­mi­nol­ogy, etc.
• No main­te­nance: Cre­ation tools must act like self-maintaining ser­vices, i.e. tools that do not require effort or attention
• No acces­si­bil­ity bar­ri­ers: For global adop­tion, con­tent cre­ation expe­ri­ences need to be acces­si­ble, which means low-bandwidth, multi-lingual, cross-media, and plat­form agnostic.

This is a start­ing list, but it cap­tures the essence of the offer­ings that have been suc­cess­ful in the past.

In addi­tion to the expe­ri­ence, the con­tent that peo­ple cre­ate needs to fol­low some guidelines.

• Address­able: Includ­ing find­abil­ity and search­a­bil­ity, AR con­tent must be fully address­able by a broad spec­trum of tools and pro­to­cols.  AR will fail at bridg­ing the real and dig­i­tal if the con­tent peo­ple cre­ate for aug­mented expe­ri­ences  can­not — at least par­tially — be addressed across this bound­ary, which is what makes AR an enchanted win­dow rather than a sim­ple browser / UI lens.  This seems like the sim­plest of these guide­lines (after all, what isn’t address­able in a dig­i­tal space?), but I think in the end it will be quite chal­leng­ing to realize.
• Inter­op­er­a­ble: Con­tent must work across plat­forms, for­mats, and browsers, in terms of cre­ation, shar­ing, and management.
• Portable: Con­tent must be mov­able or portable for peo­ple to make the effort of cre­ation; it can­not be con­fined to a sin­gle stor­age loca­tion, ser­vice, tool, owner, etc.  This touches on the famil­iar ques­tions of data own­er­ship and the commons.

The goal of these sug­ges­tions is to push AR toward matu­rity and broader adop­tion as quickly as pos­si­ble, using lessons from the evo­lu­tion of the Web.  What sug­ges­tions for guide­lines for con­tent cre­ation expe­ri­ences and the nature of AR con­tent do you have?

If I am off base in think­ing the cre­ation bar­rier crit­i­cal at this early stage of aug­mented reality’s rise above the exper­i­men­tal water­line, then what is more important?

Since I wrote about the user expe­ri­ence of aug­mented real­ity less than two weeks ago, the most impor­tant devel­op­ment is the arrival of aug­mented iPhone apps (unof­fi­cially for the moment, offi­cially in September).

Why is this so impor­tant, when Wik­i­tude and other AR Android apps have been avail­able for almost a year?  Bring­ing aug­mented real­ity to the iPhone changes the cul­tural assump­tions made about AR expe­ri­ences as a class of offer­ing. Endors­ing AR expe­ri­ences for iPhone users moves aug­mented real­ity from the geek realm of Android and Google, to the chic world of Apple.  Cul­tur­ally, the assump­tions we make about the new prod­ucts and ser­vices from Apple and Google are dri­ven largely by the dif­fer­ences in way we per­ceive the two brands.  Apple is chic, while Google is geek.

Look­ing Ahead

Con­nect­ing the Apple brand to aug­mented expe­ri­ences will per­suade many peo­ple to try out AR.  Yet as I’ve said, and many oth­ers as well, get­ting the user expe­ri­ence of aug­mented real­ity ‘right’ is absolutely the crit­i­cal ele­ment to the long term via­bil­ity of this new class of expe­ri­ences.  This entails two efforts.

First, design­ers must refine the expe­ri­ences offered by all those AR appli­ca­tions based on the four clas­sic inter­ac­tion pat­terns known so far — Head-Up Dis­play, Tri­corder, Holochess, and X-ray Vision.  Two fac­tors make refine­ment essen­tial: com­pe­ti­tion from other AR offer­ings that reduces the nov­elty value of your expe­ri­ence, and increased ‘load’ on the UX in the form of actual use for every­day pur­poses in the com­plex set­ting of real life.  Think about try­ing to choose where to get lunch for the after­noon by sort­ing through 1500 list­ings for cof­fee shops and restau­rants while stand­ing on a street cor­ner in the rain in Lon­don hold­ing your phone aloft.  The func­tional aspects of AR expe­ri­ences just aren’t refined enough to han­dle the inter­ac­tion design, visu­al­iza­tion, and con­tex­tual sen­si­tiv­ity chal­lenges implied. [Pre­dic­tion: AR usage cases will nat­u­rally set­tle on a set of com­mon sce­nar­ios that bal­ance the strengths and weak­nesses of each of the four clas­sic pat­terns.  More spec­u­la­tion on that in a later post.]

Sec­ond, design­ers must address the gaps in the set of con­cepts now used as the basis for imag­in­ing new aug­mented expe­ri­ences.  I flagged six ‘miss­ing’ pat­terns in the range of expe­ri­ences offered so far; Loner, Sec­ond Hand Smoke, Pay No Atten­tion To the Man Behind the Cur­tain, The Invis­i­ble Man!, Tun­nel Vision, and AR for AR’s Sake (see the arti­cle for details).  I’m sure the very savvy read­ers of this blog can iden­tify even more.

I hope all the AR inno­va­tors, design­ers, and entre­pre­neurs work­ing hard on the crest of this break­ing wave of tech­nol­ogy find ways to take on both of these tasks.  If they can’t refine the exist­ing mod­els and fill in those expe­ri­ence gaps, then nei­ther Apple chic nor Google geek cred will suf­fice to make aug­mented real­ity viable in the long term.  And what could lit­er­ally be a new way of see­ing the world — one with legit­i­mate poten­tial for chang­ing our behav­ior with regard to urban spaces, the envi­ron­ment, social struc­tures, play, and eco­nom­ics, among just a few spheres of human activ­ity — will remain lit­tle more than a cam­era obscura style curiosity.

Mar­shall Kirk­patrick of Read­WriteWeb links to Inside Out: Inter­ac­tion Design for Aug­mented Real­ity (the lat­est Every­ware col­umn) in two recent sto­ries track­ing the fast-moving aug­mented real­ity space; Aug­mented Real­ity: Five Bari­ers to a Web That’s Every­where and, and Robot­Vi­sion: A Bing-powered iPhone Aug­mented Real­ity Browser. 

Thanks, Mar­shall!

And as a bonus, Tim O’Reilly tweeted about Marshall’s arti­cle.  I doubt that Tim reads this feed, but it’s always nice to be rec­og­nized, even indirectly.

I’m very happy to announce that Inside Out: Inter­ac­tion Design for Aug­mented Real­ity — the lat­est install­ment of my col­umn Every­ware @UX Mat­ters -  is live now.  (Tim­ing is some­times the writer’s friend, as I was at the Layar event Mon­day night here in Ams­ter­dam just the day before, and had the chance to talk with some of their team.)

AR is more of a per­spec­tive and class of expe­ri­ences than an instance of new tech­nol­ogy, so I wanted to approach the sub­ject from the spe­cific per­spec­tive of user expe­ri­ence and inter­ac­tion design.  Reac­tions from the aug­mented real­ity com­mu­nity are pos­i­tive so far; Claire Boon­stra of Layar, and no less than the inim­itable Tish Shute of Ugo­Trade, have all been kind enough to rec­om­mend it.  Thanks to them and to every­one who’s tweeted and posted this one.

As we explore the role aug­mented real­ity will play in our gigan­tic exper­i­ment with every­ware, we should keep in mind that the map is not the ter­ri­tory.  But there is no deny­ing an effec­tive map will surely help point the way as you try to find your way around a strange new coun­try.