Category: Uncategorized

For the last 3 days I have been at San Jose at the third official Oculus Connect conference (OC3) and its been amazing to see some of the prototypes, talk to a few developers and just learn from other folks who are charting the VR space. Its been an amazing mix of education and fun and given me a lot of mull over on the flight back tomorrow. Here are some highlights though:

The Social Focus: Putting people at the center

Social was definitely the big message at OC3, and the Facebook Social concept that Mark Zuckerberg demoed was really well done.

I really liked the avatars and the way they worked here. What was a little weird was it was at the same stage where a few minutes later they talked about the new Oculus Avatars system which as developers we are encouraged to use. These 2 projects are totally independent of each other and a later talk by Facebook’s Mike Booth talked about a lot of learnings that they developed while building Facebook’s avatar system that flies in the face of the look of the official Avatars app/sdk. Hopefully these two will me merged at some time in the future but there is enough of a chance that they may not.

 

Oh and I hope you didn’t liked the boxy avatars from Oculus Social app, cause that effort seems to be dead.

Oculus Touch

Oculus finally revealed the pricing and availability of the Touch controllers. At $200 they are a little pricey and make a full Rift setup be almost the exact same price as the Vive. That aside the controllers are really nice and bringing your hands into VR does up the level of immersion in VR tremendously (at work we had used the Leap Motion sensor on top of the DK2 to get some hand tracking in a demo but thankfully we can leave those kludges behind). The only unfortunate thing with the Touch controllers being optional purchase is that developers can’t really rely on them being available which might prevent them from leveraging them to avoid splitting the market. Hopefully most Rift owners choose to get it cause I will say they work really well.

Room Scale

Oculus can now do Room-scale VR but requires a third sensor that you can now buy for $79. I can’t imagine a lot of people going for this at least immediately and Room scale might remain the domain of the Vive for now.

Video

One of the messages Oculus apparently wanted to send was that passive experiences shouldn’t be dismissed as apparently usage of Oculus is equally divided between games and video apps. At the keynote, Oculus also announced a Video SDK that will let video publishers create content but let Facebook host and distribute that content efficiently based on their research around optimized 360 video streaming (Foveated rendering etc). I need to dig more into this.

Misc items:

Other things also interesting included:

• Facebook continuing to fund more VR development with another 250M fund for VR apps and games
• Oculus is adding an Education category to their stores so expect more apps and games for that
• Cheaper Oculus Ready certified PCs including a $500 one. Oh and Oculus Ready certified laptops for you developers on the go.
• Oculus will cover Unreal Engine license fees for apps sold through the Oculus Store for up to the first $5 million in gross revenue.
• Lots of effort being put into audio and ambisonic rendering. A new higher audio quality headphone for the Rift was also introduced and given away to the attendees.
• Untethered stand-alone VR headset in the works in the lab.
• Oculus will now support WebVR and introduced a VR flavor of their popular React JavaScript framework

Favorite Game:

Most of the OC3 event was about trying out the demos for different games coming soon. Lots of good ones to choose from but Eagle Flight was totally awesome.

Favorite moment: Talking to John Carmack

Okay so this was a totally nerd moment, but I have been a big fan of John Carmack for a long time and he was kinda like a geek hero of mine. Being able to talk to him for a bit was really amazing. I even captured a part of it on video (vertical cause I saved it from Periscope 🙂 )

For the last few weeks I have been involved with Penn’s Research Experience for Teachers program (RET). The 6 week NSF funded program paired 10 teachers from local high schools in Philadelphia with a research engineer at Penn’s General Robotics, Automation, Sensing & Perception (GRASP) laboratory at Penn and an industry mentor in an effort to expose these teachers to advanced engineering concepts that they could take back to their students.

I had the pleasure to help with Matthieu Lecce and Danelle Ross on an effort to use computer vision to detect the volume of transparent liquid in a glass container. The project built on previous effort by Matthieu and the research team on seeing glassware (more information on that research can be found here). Technically Philly did a great writeup on their site as well on the final day of the project.

The program was really intense for the teachers and major props to Danelle for completing it and going through this intense learning experience. My own contribution was fairly limited as I spent only a couple of hours a week with the team answering how groups like Comcast Innovation Labs (where I work) investigate such technology and how machine vision is being used in Virtual Reality and Augmented Reality, a domain I am currently very involved in at the lab.

More than anything though, it was a great learning experience for me. I learnt a lot of new concepts on how machine vision is approached (my only real experience with raw machine vision previously had been some OpenCV experiments in Processing and some half complete Android projects with face detection). Learning the core concepts that go into machine vision and the current state of the art in that field was a great experience.

Sometimes I really miss grad school.

For the last 8 months, I have been working on VR prototypes for Comcast, one of which we showed off recently at the Cable Show and have been talking about at different conferences (you can see the deck here).

We recently started adding some new faces to my team and so I figured it might be a good idea to put down a quick how-to on GearVR apps. A lot of it can be found at different links on the internet, but this might be useful to go from zero to a quick “Hello World” app using Unity.

1. Setting Up GearVR for Android build:

Since  VR apps for the GearVR are Android apps, the standard Android setup for development is required:

1. Go to the Settings app on your Samsung phone and go to About Phone settings.
2. Tap the “Build Number” item on the list repeatedly till you see a “You are now a developer” message
3. Now in the main Settings list, you’ll see a Developer options menu item
4. Tap on it and in the list that comes up, turn on `USB debugging`  toggle button.
5. Connect your phone to your laptop and an alert will appear asking you if you’d like to allow the PC to debug your app. Tap yes.

2. Create a Unity app

1. Install the latest version of the Unity IDE including the Android plugin.
2. Create a quick new project (just put a cube in front of the camera so that you have something to see when the app launches)
3. From the Unity IDE toolbar, click on File -> Build Settings
4. Unity probably defaulted you to a Mac/Windows app export. From the platform list, click on Android and then click on Switch Platform. This might take a couple of minutes as Unity converts the project.
5. When done, tap on the Player Settings. In the settings that appear, tap on the Android tab, and in the Other Settings, select the Virtual Reality Supported option and enable the Oculus option.

3. Enable your Android device to run Oculus apps on device.

1. Create a profile at developer.oculus.com 
2. Connect your phone to your computer and use the adb devices command on the command line to detect the connected device. Copy the device id.
3. Use the device id at the Oculus osig file generator. Paste the id in the form field and you’ll get an osig file.
4. Save the osig file in your project in the Project > Plugins > Assets folder. This will allow Unity to package the file into the generated apk

4. Build and Run 

1. Click the Build and Run item in the Unity toolbar. Unity will pack and deploy the app to your phone.
2. You’ll probably get an alert saying “The app cannot be launched because there is no Launcher Activity in the app. But the app will be deployed to the device.
3. Back on the phone, go to Settings > Application Manager and find an app called “Gear VR Service”. Select it.
4. Tap Storage and then  Manage Storage
5. Tap the VR Service Version label multiple times till you get a notification saying “You are now a developer”. Two other options will appear below: Developer Mode and Add Icon to app list 
6. Tap on the Add Icon to app list. This will add an icon at the phone’s app list (where you find all your other apps)
7. Tap the icon from the apps list. The launching activity will list all your available VR apps. Find your app and tap on it. You’ll get a screen instructing you to insert the phone into the GearVR. Inserting it will launch your app 🙌

 

 

Yesterday Jack Zankowski (who leads the next gen UX at Comcast) and I gave a talk on the design and engineering challenges in building VR experiences for TV content at the WICT Tech It Out event at Villanova University. While there, we were also able to check out their pretty interesting VR cave as well.

The talk is based on a VR prototype we demoed recently at the Cable Show and the Code Conference. Personally its been a very educational experience. In a way, working with Unity is like working with Flash all over again, with similar challenges ( managing visual assets, code architecture, working in a team of varying skillsets from design to development). Hopefully I’ll do some more write-ups here on those challenges. But for now, the deck from the event is embedded below.

Thanks to the folks at WICT for having us.

 

 

Quite a few years back, I got really interested in Treemaps. The whole project had started as an academic discussion between a friend and I on how hard a treemap would be to build (they seemed to be a pretty popular data visualization method back then, though I don’t see them around much these days). Anyway, what I thought would be trivial weekend hack turned out to be a lot more involved and I ended up reading and implementing Mark Bruls, Kees Huizing, and Jarke J. van Wijk’s algorithm for an optimum Squarified Treemap implementation (in ActionScript 3, you can find all the code which I open sourced here).

To demo the algorithm, I took the visual aesthetics of Marumushi’s NewsMap but instead used it to show trending topics on Digg.com. The project, not so creatively named DiggGraphr, got fairly popular and was mentioned on a few data visualization blogs and even won an award for a Digg.com API contest.

DiggGraphr has been dead for a while now, but I was pleasantly surprised to have it be included in a research paper titled ‘A case study on news services using big data analytics (뉴스빅데이터 서비스 사례 및 모델 개발 연구)’  conducted by professor Kim of the Korea Aerospace University and three researchers from a non-profit research organization (Media and Future Institute).

If anyone of you can read Korean, feel free to read the paper here

 

At last week’s AndroidPhilly event, I was surprised to find a lock screen notification for a “Physical Web Page” on my phone

Tapping on that notification linked to an explanation page of the Physical Web pages and then a link to Nick Dipatri’s BLE geo-fencing app.

This was the first time I have seen the Physical Web pages in action, though Google has talked about them for a while. While not a lot of people talk about iOS’s iBeacons much anymore (compared to the rage they were when they were announced), the Physical Web pages approach is different with Google Chrome being the receiver app that detects the beacons and notifies the user. This is great for developers who don’t have to worry about having their app installed but also means users wont be able to skip notifications from services they don’t care for.  Bundling the beacon technology within Chrome also means that the Google approach is more cross platform and will work on multiple devices.

It’ll be interesting to see how this evolves in the next few years.

 

 

I have been reading a lot of interesting things on robotics and AI lately.

• Boston Dynamics’ new Atlas 2 Robots video is fascinating to watch. While the capabilities of the robot to balance itself and track items is great, almost every person who I showed the video felt sorry for the robot who keeps getting shoved, kicked and its box pushed away. The empathy towards a machine is really a fascinating thing to observe.
• If you are really interested in Atlas, give DARPA’s YouTube channel a look, specially their playlist on the Robotics challenge thats pushing Robotics just as their autonomous cars challenge did to cars in 2012.
• I just finished the book “Machines of Loving Grace“. While the book doesn’t have any unique answers to the question of human-machine balance in the coming years, the book is full of fascinating information on the history of AI and robotics. Definitely worth a read.
• On the other hand, I couldn’t get too far into “Our Final Invention: Artificial Intelligence and the End of the Human Era“. The few chapters that I did read were so speculative, lacking real science that I just couldn’t finish it.
• While Google has been in the news quite a bit on its Robotics initiatives, Wired ran a really interesting article on Andy Ruben’s new AI and Robotics incubator: Playground.global, that recently closed a $300M funding round
• With the dramatic increase in interest in Robotics and AI, IBM wants to reassure everyone with a series of TV ads that Watson will not become the next Skynet and take over the world. Here’s hoping…

I was in New Delhi, India for the last couple of weeks visiting family. The trip has also been an interesting experience looking at how people use mobile phones here. Here are some observations.

Devices

Most folks in India are on Android, though I did see more iPhones than I thought I would. The Android devices are those I have never seen before. Pictured below are my parents’ and brother’s phone. The one on the left is from a company called CoolPad, the middle is a Xiaomi Mi 4i and the one on the right is an Asus. All of these are ₹10,000 to ₹15,000, which translates to around $150 to $225. And they work really well. I am particularly impressed by the Xiaomi which has a very nice UI language and generally performs pretty well (why can Xiaomi do such a good job at UI and the much bigger Samsung cannot is a mystery to me)

Also, I didn’t see many phablets, though I did see a bunch of cheap and really small Android phones.

 

Apps

Almost everyone in India seems to be on WhatsApp (unsurprisingly the #1 free app in India), and are on numerous WhatsApp groups that range from family groups to local neighborhood communities. WhatsApp has surprisingly few features compared to other messaging apps and yet, even though there may be other apps with features (specific to commerce for example), WhatsApp is where all the commerce happens.

 

ShareIt, a file transfer tool by Lenovo, is the #2 free app on the store beating even Facebook which is at #3. Guess file transfer between phones is a big thing here.

Another thing I was surprised to notice was that people were pretty open to accepting calls from unknown numbers (I mostly let those go to voicemail and I know most of my American friends do as well). This also makes Truecaller a pretty essential app here.

I actually like the Truecaller experience, which opens a small overlay every time a call comes in with information on the caller. As of this writing, Truecaller is #7 free app on the Google Play Store in India.

Pre-Paid plans and Missed Calls

Unlike the US where almost everyone is on unlimited calls, people here still have plans with limited number of minutes. A lot of communication happens by just missed calls, whether its just signaling (“Send me a missed call when you have reached so-and-so place”) or with a second call from a land-line by the receiver of the missed call (incoming calls are free). Even ads on TV and radio mention phone numbers that you give a missed call to and they’ll call back with more information.

(While the idea of the missed call seems weird, it reminds me of the Yo app. Both allow a very simple message to be interpreted in a variety of ways)

People also add minutes to their plans using scratch cards, which also apparently make great give aways at events.

Phone: The original Conversational Commerce app

The term conversational commerce has been getting pretty popular lately as companies build AI and Natural Language based systems where you would be able to talk to different services (like Uber) as opposed to using an app. The original version of that is how services are still used in India. Almost every service like groceries, food delivery etc is a phone call away and most people I met prefer to just call rather than deal with an app or a website. Apps like Whatsapp will also soon allow you to call a merchant from right inside the app which seems to be how folks prefer to do business in India.

Challenges

I met a few entrepreneurs at a local event in Delhi and it was interesting to see their challenges. Some are familiar, like general lack of engagement with apps in a very crowded app market. Others seem more unique to India, like a general unwillingness to pay for digital services for lack of trust or low perceived value for the services. Poorer connectivity is also an issue and quite a few apps (like Google Transit for example) give you options to download the data for offline usability.

An interesting point that came up in the conversation is the growing need for localization in apps. While a fair bit of the country speaks English, as mobile devices become more accessible to even the remotest part of India, support for regional languages is becoming more and more important.

WordPress.com prepared a 2015 annual report for this blog.

Here’s an excerpt:

The concert hall at the Sydney Opera House holds 2,700 people. This blog was viewed about 35,000 times in 2015. If it were a concert at Sydney Opera House, it would take about 13 sold-out performances for that many people to see it.

Click here to see the complete report.

I gave a quick 10 minute talk recently about some lessons I have learned after working in a “labs” group for close to 5 years. The deck is embedded below: