How to code Augmented Reality Mobile Application with Android ARCore SDK - Part 2

21 Jan 2021
How to code Augmented Reality Mobile Application with Android ARCore SDK - Part 2

In the previous blog, I described the basics of AR and how to set up Android Studio to start with AR Core SDK to develop AR App. In this blog, we will develop an AR app for Home Decor step by step.

  1. Add Sceneform plugin in Android Studio.
  2. Select minimum SDK for AR projects to API 24 that is Android 7.0 (Nougat)
  3. Add Gradle dependencies for ARCore SDK and Sceneform
  4. Configure Android Manifest for Camera permission, OpenGL, and AR requirement
  5. Create a layout including ARFragment.


All the above requirements are explained in detail in the previous blog [How To Code Augmented Reality Mobile Application With Android ARCore SDK - Part 1].


Import 3D model


Now after basic setup, we need to import 3D models. You can create your 3D model or directly import 3D models from poly.google.com


Select a 3D model and download the original OBJ file.


We will not copy these models to our project directly in resources. Instead, we will create a sample data directory and put them there. Sceneform will convert them to the format that our app will understand.


In the app folder, create a directory named sampledata/models.


Copy images of the models in a drawable folder under res.


We require sceneform plugin in android studio to import 3d models in our project.


Now import the models using a command in build.Gradle

sceneform.asset (
'sampledata/models/Chair2.obj',
'default',
'sampledata/models/Chair2.sfa',
'src/main/res/raw/chair'
)

sceneform.asset (
'sampledata/models/Lamp.obj',
'default',
'sampledata/models/Lamp.sfa',
'src/main/res/raw/lamp'
)

sceneform.asset (
'sampledata/models/Houseplant.obj',
'default',
'sampledata/models/Houseplant.sfa',
'src/main/res/raw/houseplant'
)


Sync the Gradle file and rebuild the project. A raw folder is created and the 3D model is imported into the raw folder.


Loading models from resources


As 3D models are imported to the raw folder we have to load the selected model into the ArFragment.

fun loadModels(callback : (ModelRenderable) -> Unit) {
val modelRenderable = ModelRenderable.builder()
.setSource(this, selectedId)
.build()
    CompletableFuture.allOf(modelRenderable)
.thenAccept() {
callback(modelRenderable.get())
}
.exceptionally {
Toast.makeText(this,"Unable to load Models", Toast.LENGTH_LONG).show()
null
}
}


Here selectedId is the id of the 3D model selected from UI layout by the user. It will be like R.raw.piano if the piano is selected by the user.


We then add the 3D model to ArFragment.

fun addNodeToScene(anchor: Anchor, modelRenderable: ModelRenderable) {
var anchorNode = AnchorNode(anchor)
modelNode = TransformableNode(arFragment.transformationSystem)
.apply {
renderable = modelRenderable
localScale = Vector3(0.1f, 0.1f, 0.1f)
localPosition = Vector3(0.0f, 0.0f, 0.0f)
setParent(anchorNode)
getCurrentScene().addChild(anchorNode)
select()
}
    modelNode.setOnTapListener{_,_ ->
if
(modelNode.isTransforming) {
if(buttonLayout.isVisible) {
buttonLayout.visibility = View.GONE
} else {
buttonLayout.visibility = View.VISIBLE
}
}
}
}
fun getCurrentScene () : Scene = arFragment.arSceneView.scene
private fun setupDoubleTapArPlaneListener() {
arFragment.setOnTapArPlaneListener { hitResult,_,_ ->
loadModels { modelRenderable ->
addNodeToScene(hitResult.createAnchor(),modelRenderable);
}
}
}


In the Double-tap listener, we register in onCreate of Activity.

override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState)
setContentView(R.layout.activity_main)
arFragment = fragment as ArFragment
buttonLayout = layoutInflater.inflate(
R.layout.button_layout, null) as LinearLayout
setupDoubleTapArPlaneListener()
}


We have to add buttons for the proper positioning of our 3D object. 

Here is the complete code:

button_layout.xml

<?xml version="1.0" encoding="utf-8"?>
<LinearLayout
xmlns:android="http://schemas.android.com/apk/res/android"
android:layout_width="match_parent"
android:layout_height="wrap_content"
android:gravity="center"
android:background="#fff"
android:orientation="horizontal"
>
<ImageView
android:layout_width="wrap_content"
android:layout_height="wrap_content"
android:src="@drawable/leftarrow"
android:layout_marginRight="10dp"
android:layout_marginLeft="10dp"
android:onClick="leftClicked"
android:id="@+id/button1"
/>
<ImageView
android:layout_width="wrap_content"
android:layout_height="wrap_content"
android:layout_marginRight="10dp"
android:layout_marginLeft="10dp"
android:src="@drawable/uparrow"
android:onClick="upClicked"
android:id="@+id/button2"
/>
<ImageView
android:layout_width="wrap_content"
android:layout_height="wrap_content"
android:layout_marginRight="10dp"
android:layout_marginLeft="10dp"
android:src="@drawable/downarrow"
android:onClick="downClicked"
android:id="@+id/button3"
/>
<ImageView
android:layout_width="wrap_content"
android:layout_height="wrap_content"
android:layout_marginRight="10dp"
android:layout_marginLeft="10dp"
android:src="@drawable/rightarrow"
android:onClick="rightClicked"
android:id="@+id/button4"
/>
</LinearLayout>


activity_main.xml

<FrameLayout xmlns:android="http://schemas.android.com/apk/res/android"
xmlns:app="http://schemas.android.com/apk/res-auto"
xmlns:tools="http://schemas.android.com/tools"
android:layout_width="match_parent"
android:layout_height="match_parent"
tools:context=".MainActivity"
>
<fragment
android:name="com.google.ar.sceneform.ux.ArFragment"
android:id="@+id/fragment"
android:layout_width="match_parent"
android:layout_height="match_parent"
/>
<include layout="@layout/button_layout" />
<LinearLayout
android:layout_width="match_parent"
android:layout_height="wrap_content"
android:layout_gravity="bottom"
android:layout_marginTop="10dp"
>
<HorizontalScrollView
android:layout_width="match_parent"
android:layout_height="100dp"
android:id="@+id/horizontalScrollView"
>
<LinearLayout
android:layout_width="wrap_content"
android:layout_height="wrap_content"
android:orientation="horizontal"
>
<ImageView
android:layout_width="wrap_content"
android:layout_height="100dp"
android:src="@drawable/piano"
android:onClick="pianoSelected"
android:id="@+id/button1"
/>
<ImageView
android:layout_width="wrap_content"
android:layout_height="100dp"
android:src="@drawable/lamp"
android:onClick="lampSelected"
android:id="@+id/button2"
/>
<ImageView
android:layout_width="wrap_content"
android:layout_height="100dp"
android:src="@drawable/chair"
android:onClick="chairSelected"
android:id="@+id/button3"
/>
<ImageView
android:layout_width="wrap_content"
android:layout_height="100dp"
android:src="@drawable/plant"
android:onClick="plantSelected"
android:id="@+id/button4"
/>
</LinearLayout>
</HorizontalScrollView>
</LinearLayout>
</FrameLayout>


MainActivity.kt

class MainActivity : AppCompatActivity() {
    lateinit var buttonLayout: LinearLayout
lateinit var arFragment : ArFragment
var selectedId = R.raw.piano
var x : Float = 0.0f
var y : Float = 0.0f
lateinit var modelNode : TransformableNode
    override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState)
setContentView(R.layout.activity_main)
arFragment = fragment as ArFragment
buttonLayout = layoutInflater.inflate(R.layout.button_layout, null) as LinearLayout
setupDoubleTapArPlaneListener()
}
    fun leftClicked(v : View){
x = modelNode?.localPosition.x - 0.1f;
modelNode?.apply {
localPosition = Vector3(x, y , 0.0f)
}
}
    fun rightClicked(v : View){
x = 0.1f + modelNode?.localPosition.x;
modelNode?.apply {
localPosition = Vector3(x, y , 0.0f)
}
}
    fun upClicked(v : View){
y = modelNode?.localPosition.y + 0.1f;
modelNode?.apply {
localPosition = Vector3(x, y , 0.0f)
}
}
    fun downClicked(v : View){
y = modelNode?.localPosition.y - 0.1f;
modelNode?.apply {
localPosition = Vector3(x, y , 0.0f)
}
}
    fun pianoSelected(v : View){
selectedId = R.raw.piano
}
    fun lampSelected(v : View){
selectedId = R.raw.lamp
}
    fun chairSelected(v : View){
selectedId = R.raw.chair
}
    fun plantSelected(v : View){
selectedId = R.raw.houseplant
}
    fun loadModels(callback : (ModelRenderable) -> Unit) {
val modelRenderable = ModelRenderable.builder()
.setSource(this, selectedId)
.build()
        CompletableFuture.allOf(modelRenderable)
.thenAccept() {
callback(modelRenderable.get())
}
.exceptionally {
Toast.makeText(this,"Unable to load Models", Toast.LENGTH_LONG).show()
null
}
}

    fun addNodeToScene(anchor: Anchor, modelRenderable: ModelRenderable) {
var anchorNode = AnchorNode(anchor)
modelNode = TransformableNode(arFragment.transformationSystem)
.apply {
renderable = modelRenderable
localScale = Vector3(0.1f, 0.1f, 0.1f)
localPosition = Vector3(0.0f, 0.0f, 0.0f)
setParent(anchorNode)
getCurrentScene().addChild(anchorNode)
select()
}
}
    fun getCurrentScene () : Scene = arFragment.arSceneView.scene

private fun setupDoubleTapArPlaneListener() {
arFragment.setOnTapArPlaneListener { hitResult,_,_ ->
loadModels { modelRenderable ->
addNodeToScene(hitResult.createAnchor(),modelRenderable);
}
}
}
}


Output

Share
You can reach us at hello@47billion.com