Building ARCore apps using Sceneform — Part 3

Hello AR using Sceneform

Image credits — to me :-P

In this following series, we’ll learn about the fundamentals of Augmented Reality, ARCore SDK, Sceneform Framework — using simple examples. You will be walking away with an AR app at the end of this series.

Series Pit Stops

• Part 1 — ARCore Fundamentals
• Part 2 — Sceneform vs OpenGL
• Part 3 — Hello AR using SceneForm I (you’re here)
• Part 4 — Hello AR using SceneForm II

Prerequisites

• Part 1 — ARCore fundamentals
• Part 2— ARCore vs OpenGL
• Android development experience (SDK, library usage, Fragment etc.)
• Kotlin experience (codebase will be in Kotlin)
• Augmented reality support device or the emulator supporting them

I know what’s on your mind..

Yes. I know. You want me to stop talking and show some code and progress. Yes! that’s what we’re gonna do today. So, get your neurons sparked up and ready!

What’s our Agenda?

We’re going to build a simple AR app using Sceneform, an app I call as SpreadLove, where you can place 3D model hearts (look at the screenshot below). We may also make it look lively by rotating it.

our end result

Note: If you need to skip the explanation and checkout the code, Please feel free. But I recommend going step by step

Hariofspades/SpreadLove

Step 1: Keeping the assets ready

In order to show the assets on the screen, we need to have the heart asset. Let’s find one and download.

Every 3D models are usually called Assets and the texture or the skin that it bears are commonly known as Material

There are many free 3D model sites are available, but the detailing and rendering are too much for our app. So, Google has a nice place called Poly, where we can get nice 3D models.

Heart

Just temporarily teleport your self to the above link and download the model as OBJ file.

Just to remind you, that Sceneform will render models of the format OBJ, FBX, glTF

Step 2: Creating a new Android Project

I assume most of the readers here know how to create an Android project, but just to stay on the same page, I’m sharing my steps on the configuration of the project that I created.

• New Android Project with Kotlin Support
• Minimum SDK as Android-P (I’m using Pixel device, so created just like that, use your own min SDK)
• MainActivity as the BasicActivity — with a FAB at the bottom

Step3: Configuring Sceneform

Next we need to add Sceneform to our project. Again, I assume that the readers are aware of working with libraries. Anyway, for others add the following to your app’s build.gradle file

implementation "com.google.ar.sceneform.ux:sceneform-ux:1.0.0"

Look out for the latest version for the above. By the time you’re reading, Google would have released a lot of versions :-P

Build it! and we’re good to go!

Step 4: Installing Sceneform tools plugin

As said earlier, Android studio provides us with the plugin Google Sceneform Tools, which will help us in exporting assets to our project. To install this, Go to..

• Android Studio -> Preferences -> Plugins
• Select Browse repositories… -> Search Google Sceneform Tools
• Click on Install

This tool provides us with the following

• Easy asset exporting — we can export all supported 3D models to our project
• Automatic gradle configuration for our models — Whenever we export the models, few configurations in the gradle needs to be injected — like the one below. But not to worry, this tool does it for us.

sceneform.asset('sampledata/Heart.obj',
        'default',
        'sampledata/Heart.sfa',
        'src/main/assets/Heart')

• Finally, 3D viewer! yes you heard right! This tool will give us a 3D viewer, where after exporting we’ll be able to see how our models will look like in our app.
  Also, this viewer is WYSIWYG (what you see is what you get) so if it looks like something, that’s who it will look in our app too.

Step 5: Exporting the asset

Now that we have the heart, sceneform tool plugin, it’s time to export it to our project.

• To hold all the sample data, Android studio now has a new folder called sampledata. It will hold all data that will not go within our APK. To add the sample data folder, right click app -> new -> sample data
• Copy all the contents (Heart.mtl and Heart.obj) to the sample data folder
• Now we have the models in our project, we now need to export them to Sceneform asset and binary files. That’s where our plugin comes to the rescue. Right-click Heart.obj -> Import Sceneform Asset
• By exporting, the tool will create 2 files Heart.sfa (SceneForm Asset description) and Heart.sfb (SceneForm Binary). The binary file is the one that will be shipped along with the APK and it stays at the assets file. The converted description file — sfa will remain in the assets. This is where we’ll change the object properties and it will be converted to the binary file.

Importing heart asset

• If you’re seeing the above screen, then click on Finish
• After this process, the gradle will build and it will create couple of files — as said before : Heart.sfa in sampledata folder and Heart.sfb in assets folder
• And also, the studio opens up our 3D viewer, where we can explore our 3D model. Once again, its WYSIWYG!

Our assets are in place and we’ve added sceneform SDK as well. It’s time to put them together in the scene.

• Also, open Heart.sfa file and change the scale value to 0.0010. I’ll explain more about modifying sfa files in another post. The reason why we changed the scale value is that, the model will be quite big by default, so we have just reduced it.

If you feel this post is bit lengthy, yes it is.. Now it’s the right time to take a break and appreciate yourself.

Step 6: Creating ARFragment

Due to the lengthy steps, we’ll be seeing more of the code explanations in the next pit stop. Don’t worry if you did not get what you’re copy-pasting or doing, you will understand it in the next post. Now let’s quickly build stuff! I don’t want to overload information to you!

There are several aspects of making a great AR experience that involves interacting with multiple views. Things like displaying a non-text indicator that the user should move the phone in order for ARCore to detect planes and handling gestures for moving and scaling objects. To achieve this, let’s add ArFragment in our content_main.xml (if you have created a BasicActivity)

Now, let’s configure it in our Activity file, to enable our ARView. Add the following lines in your MainActivity.kt.

Step 7: Configuring Manifest file

Inorder to use camera, we need add few permissions. To be exact, we need permission for camera and we need a request to use ARFeature. Let’s go ahead and add them in our AndroidManifest.xml file

Open app/manifests/AndroidManifest.xml and in the <manifest> section add these elements

<uses-permission android:name="android.permission.CAMERA" />
<uses-feature android:name="android.hardware.camera.ar" android:required="true" />

Then add the metadata in the <application> section

<meta-data android:name="com.google.ar.core" android:value="required" />

Complete Manifest file will look like below

Step 8: Run and check!

If you have done the above steps correctly, you should be seeing the following action in your ARCore supported phone

The ArFragment we just added handles the permissions, ARCore session creation, and the plane discovery UI. Then you should be seeing the request for permission.

permission request

Then you should see the indicator that you need to move the phone in order to detect a plane. Planes are detected on flat surfaces that have some “texture” or pattern and it usually won’t work effectively with white planes and backgrounds.

Moving phone to detect planes

Once if the planes are detected, you can see white dots spread around it, telling us that it has detected the following plane.

Showing detected plane

My house is all white, so I just used the doormat here :-P

To be continued in the next pit stop..

The post has become quite lengthy. So I’m gonna stop here and we’ll be continuing in the next pitstop. Don’t worry I’ll make sure that I release both the pit stops at the same time.

Summary

• We fixed our goal to create a AR app using Sceneform
• We downloaded the asset
• Created new Android project
• Configured Sceneform in our project
• We installed the Sceneform tool plugin
• We exported the asset
• We added Sceneform fragment
• We configured Manifest file
• We also invoked Sceneform fragment from our Activity

Head to the next pit stop

Building ARCore apps using Sceneform — Part 4

Follow me on Twitter, Facebook or LinkedIn, will post an update when I write the next pit stop.

Thank you for using your precious time to reading this article. Liked it? Clap your 👏 to say “thanks!” and help others find this article.