Augmented Reality for the Masses
Speaker: Siamak Ashrafi
@biocodes

For more blog posts, see The Oracle Code One table of contents

Imagination

• Imagination is the beginning of science
• Imagination will spark AR (augmented reality) on mobile
• Imagination is not limited. Can grow it.
• Necessity is not the mother of invention. It is plausible intellectual curiosity
• Science fiction has to be rooted in plausible science. Many famous sci fi authors have science educational background
• Arthur C Clarke’s rules:
  • When a distinguished elderly scientist states something possible, usually right. When says impossible, probably wrong
  • Way of discovering limits of possible is to venture a little past them into impossible
  • “Any sufficiently advanced technology is indistinguishable from magic”
• Magic/imagination/design/science have a positive feedback loop
• “Truth is stranger than fiction, but it is because Fiction is obliged to stick to possibilities. Truth isn’t” – Mark Twain.

Initial AR to present

• 1901 – First mention of concept in Wizard of Oz. Marker on forehead about moral character. Read using electronic spectacles.
• 1957 – First patent in space of AR
• 1992 – Term coined by Tom Caudell at Boeing
• 2007 – Smartphones exploded giving sensors and power needed to bring to the masses
• Humans think linearly. So we overhype at beginning and then it exponentially grows and disruption happens. Currently in development phase.
• Last year, Lowe’s started where can view your kitchen with the new products in AR.
• This year presented at medical conference.
• Apple and Google will bring to masses
• Other early examples – virtually try on clothes, tell you about interest facts nearby, lanes least crowded in front of you (but caused traffic jam from everyone moving there)
• Arise game – can only cross bridge when it is solid. Which depends on perspective. [I don’t get how this is AR. It seems like a game independent of the real world]

AR vs VR

• AR – still living in real world. Digital information. Mixed mode
• VR – for living in a new world – isolates from everything around. Covers eyes/ears.
• AR on phone isn’t user friendly because awkward to hold phone in front of us.

Referencing locations

• Fiducial markers – object uses for reference. Mattel used to play virtual Rock Em Sock Em robots. Just printed out a “map” and two people could play on phones
• GPS markers – actual location
• Legacy mixare.org – used camera/compass/etc to do markers around you

Technologies

• Tango 3D Mapping – 7 degrees of freedom.
• Google killed Tango phone in less than a month.
• Superseeded by AR Core
• IMU – Inertial Measurement Unit – reports force/rate and sometimes magnetic field. Humans do this automatically. Uses dead reckoning.
• With one camera can capture two (or more) frames. Build point cloud to see what pixels same and where moved. Doesn’t work with low light/shadows/on tripod.

AR Core

• Motion tracking – pose (position and orientation of phone). Also COM (concurrent odometer and mapping) to keep virtual objects in right position. Hard math
• Environment understanding – detect horizontal services so can put objects on them
• Light estimation – make virtual objects right brightness for ambient light

AR KIt

• iOS library. Supports iPhone 6s+ and old iPads (and current ones)
• Same terminologies and principles as AR Core.
• Library existed before AR Core.
• AR Kit 2 came out recently – adds sharing experience. Can save/reuse environment.

Code/Getting started

• Repo: https://github.com/google-ar/arcore-android-sdk
• coursera.com/learnar
• 90 line program
• Have XML AR fragment
• Then build model
• Many models available online

Demo

• He said the demo always crashes his machine. Running on beta version of OS/emulator/plugin [good to do with 4 minutes left then!]
• Three calls in Android app – load fragment, create model and add listener. It’s about a page of code as two of the calls have nested lines of code.
• Solar system with multiple models what build on each other.
• Built point cloud. Use virtual sensors using IDE controls.

My take: He did a plug for voting thumbs up before actually starting. I don’t think I’ve seen that approach before. I was more interested in the higher level, but it was cool to see the details of the Android library as well. The speaker said he was surprised to have an Android talk accepted. I’m not. It’s in the ecosystem. Overall, it was cool seeing what’s out there.