XR Widget

Augmented Reality (AR) and Virtual Reality (VR) play roles in CS, transforming the way we interact with and perceive digital information. These technologies open up new possibilities for exploration, innovation, and advancement. You can use the 'XR widget' to add AR/VR-related interactivity.

Getting Started: Overview of ADB and WebADB

There are instances where you need your PC to interact with an external Android device. This can include installing an application directly from the local machine to your Android devices. ADB is a command line tool that facilitates communication between Android phones and computers. This communication can involve file transfers.

In the context of general Android development, ADB enables communication between a PC and Android phones. However, in the case of WebADB, communication occurs between the PC's browser and Android devices.

The XR widget is designed to sideloadInstalling an app on a device without using an official application store. APKs and execute commands via the adb shell from the Educative platform into Android devices via USB cable. From a user perspective, the widget provides us with two communication channels, i.e.:

  • ADB from your local machine to Android devices

  • WebADB from Educative platform (browser) to Android devices.

In the case of WebADB, you don't need local ADB in your PC but in the other case, it is mandatory.

Note: Don't forget to install ADB locally and add path environment to absolute location of ADB.

This is what it looks like:

Edit mode
Edit mode

What do I need to know?

There are currently two methods available:

  1. Upload APK File: To sideload an APK from the Educative platform to the connected Android devices.

  2. Add Command: To execute the commands on users' Android devices via the adb shell.

Method 1: How to upload an APK file

To begin uploading an APK file, click the "Upload APK File" button and follow the steps below:

  • Step I: You can select the APK file from your PC. The maximum supported size is 100 MB. Once selected, the widget will start uploading the APK.

  • Step II: Once the APK is uploaded, you should enter the appropriate package and activity name in the “Package Name” and “Activity Name” fields, respectively. You can use this website (as an example) to retrieve the names.

For example for this sample APK, the following are the required package and activity names:

  • Package name: com.DefaultCompany.ARFoundationTut

  • Activity name: com.unity3d.player.UnityPlayerActivity

Please note that the failure to set appropriate package or activity names will lead to non-optimal results, such as the application not launching for the learners automatically.

Method 2: How to add command

This method enables you to execute the adb shell commands on learners' Android devices. To begin, click the "Add Command" button and enter the command such as the following in the "VR Command" field:

am start -a android.intent.action.VIEW -d https://immersive-web.github.io/webxr-samples/immersive-ar-session.html)

You don't need to prepend the adb shell before the command; the widget will do it for you.

How learners interact

Learners can access the APKs or commands in their Android devices connected via a USB cable. When running the application, they'll see two modes: browser and local.

Learners run the application
Learners run the application
1 of 2

Do interact in preview/published mode to see if it's working.

The "Browser" mode

This mode is enabled by default and it allows the Educative platform to directly interact with the Android device. There are a few configurations that need to be done to experience the APK or command method.

Close Google ADB (by running the adb kill-server command in the terminal or closing the adb.exe from Task Manager) and all programs that may use ADB (e.g., Android Studio, Visual Studio, Godot Editor, etc.).

To begin, enable the developer mode on your Android device by following the steps below.

  1. Go to your Android device settings and tap on "About phone".

  2. Locate the "Build number" option. Tap on the "Build number" repeatedly until you see the message "You are now a developer!".

  3. Go back to the main settings menu.

  4. You should now see a new option called "Developer options". Tap on "Developer options".

  5. Toggle the switch to turn on developer mode.

The process to enable developer mode may vary slightly depending on the Android device.

Once you have enabled developer mode, connect your Android device to your PC using a USB cable. When prompted on your Android device, select the "Allow" option for USB debugging.

Next, connect the XR widget by following the steps below.

  • Step I: Click the "Run Application" button and then the "Connect" button.

  • Step II: When prompted to allow USB Debugging on your Android device, select the "Allow" option.

  • Step III: A message box will appear with the message "www.educative.io wants to connect". Your device should be listed in the message box. Select your device and press the "Connect" button.

  • Step IV: A message will then appear on your Android device asking for permission to allow USB debugging. Press the "Allow" button.

  • Step V: Now that your device is connected, you can press the "Start Activity" button.

  • Step VI: The APK will be downloaded and installed or a command will be executed, depending on the author's widget configuration of the widget. After a few moments, the application shall launch automatically.

  • Step VII: Grant the application permission to access the camera.

You can now use the application successfully. If you encounter any issues, you can press the "Restart Application" button or press the "Disconnect" button.

The "Local" mode

This mode doesn't require killing the ADB processes. To begin, run the application, switch to the local mode, and make sure to abide by the configuration instructions. Next, copy the provided command and paste it into your local terminal.

Examples

Following are some use cases of the XR widget:

  1. Augmented reality

  2. Virtual reality

  3. WebXR development

Let's see an example for each below.

Limitations

  • Currently, no Apple device is supported.

  • Only one XR widget can run at a time.

  • Browsers like Safari and Firefox aren't supported.