Airglyph: from a web page to a native app

Airglyph started as a web page called Baguette AR. In Chrome, on a phone that supports augmented reality, you slid your finger across the screen and the stroke appeared in 3D, fixed in the room. No app to install: WebXR and Three.js did the work.

The native app changes the gesture. You no longer touch the screen: you raise your index finger in front of the camera and draw in mid-air, then open your hand to stop.

That meant rewriting everything as a native Android app, in Kotlin: ARCore to anchor the strokes in the room, a rendering engine of its own built on OpenGL ES 3, an interface in Jetpack Compose, and MediaPipe's hand tracking running on the phone.

Hand tracking runs on its own thread, and the rendering never waits for it, so the image stays smooth while the app follows your finger.

The brushes came along with it: neon, real flames, lightning, portals that open onto another world, and 3D objects to place in the room, from planets to scanned sculptures. The film above was recorded on a phone, with every effect rendered live by the app.

Airglyph is coming soon to Google Play. Leave your email below and we'll write once, the day it is available.