I have an Oculus Quest 2. It’s really fun. Experiencing things in VR is a completely different experience to standard 2D screens. So how do I start building fun things to experience in VR?
Most of my backend programming is performed in Python. This is the home of TensorFlow, Keras, PyTorch, SciKit Learn, OpenCV etc. If I want to build AI-enabled toys, I need Python on the backend. Plus I don’t want to program complex mathematical functions in JavaScript (ergh).
However, Python is rubbish on the front end. I don’t think there are any out-of-the-box 3D rendering solutions for Python.
I’ve played around a lot with Oculus Link and Virtual Desktop. Annoyingly I have a Nvidia GTX 1060 3GB, which is just out of support. My small experiments conclude that doing anything in VR via the desktop is a bit of a mess at the moment, and requires a desktop gaming machine that costs upwards of £2k. VR is still in the garage phase.
three.js
Most promising for quick toy mock-ups seems to be WebXR with three.js running as the renderer. Three is a widely supported JavaScript library for 3D rendering within the browser. WebXR is a relatively new API that allows easy access and control of headset functions. The two together look to be the quickest way to play with 3D content within VR worlds. The Oculus Quest 2 supports WebXR content. I’ve played around with some of the online examples from within the headset and it works really well. Hence, the easiest way to get 3D content into my headset seems to be to host it on a local website as WebXR content.
To start off I had a play around with the tutorial set out here: https://halfbaked.city/tutorials/getting-started-with-webxr-part-1.
This is a good way to get started. To serve the HTML file I opened a terminal in the project folder and ran a simple python HTTP server (“python -m http.server“). To find out your local IP address run ifconfig on the command line (e.g. it’s likely something similar to 192.168.1.111). To view the VR content, pop on the headset and fire up the headset browser. On the Quest 2 this is actually Chromium under the hood. This is good because WebXR is disabled on non-secured (i.e., non-HTTPS) websites by default. Before viewing your simple HTML page, you thus need to set an experimental flag to avoid setting up an HTTPS server (which is a pain if you have local functions and you are behind a firewall anyway). I followed the instructions at the end of this issue here: https://github.com/immersive-web/webxr/issues/60#issuecomment-864471668 – you enable the “Insecure origins treated as secure” flag and add your IP address and port to the text box (e.g., “http://192.168.1.111:8000/”). The last thing to do is to then navigate to the IP address in the browser (e.g., type “http://192.168.1.111:8000/”) and you should be able to view the simple rotating cube in full VR.
How are VR Games Developed?
Most VR games are developed using a games engine. The two main engines are:
- the Unreal Engine; or
- Unity.
These can both export content in a form that enables access via WebXR. However, having a look through tutorials on both, the learning overhead is high for little return; these appear best left to the professionals.
A-Frame
I came across A-Frame when searching for ways to develop for WebXR. This post sets out the various options – https://constructarcade.com/learn/ – and A-Frame appears to be the easiest. It appears to be a user-friendly HTML wrapper for the three.js functionality we played with above.
Checking this out I can confirm this is the case. The simple HTML page example allows you to very quickly create content (I quickly placed an example HTML file in the same directory as the previous index.html file and was able to access the VR content). This also appears a great option for an interface with backend Python – you can use the power of Jinja templates and on-demand HTML service to change the HTML on the fly.
Robotic Ape 