Camera2 - Everything You Wanted To Know
We all use the camera on our phones and we use it, a l-o-t. There are even some applications that have integrated a camera as a feature. On one end, there is a standard way of interacting with the camera. On the other, there is a way to customize the interaction with the camera. This distinction is an important one to make. That’s where Camera2 comes in.
While it has been available since API level 21, the Camera2 API has got to be one of the more complex pieces of architecture developers have to deal with. Even if you dealt with the camera previously, it is such a drastic change from the former camera API, that you might as well forget all that you know. There are a ton of resources out there that try to showcase how to use this API directly, but some of them may be outdated and some don’t present the whole picture. So, instead of trying to fill in the missing pieces by yourself, this article will (hopefully) be your one stop shop to interacting with the Camera2 API.
Use Cases
Before we dive into anything, it is important to understand that if you only want to use the camera to take a picture or to record a video, you do not need to hassle yourself with the Camera2 API. The primary reason to use the Camera2 API is if your application requires some custom interaction with the camera or functionality. If you are interested in doing the former than the latter, I can suggest you visit the following documentation from Google:
There you will find all the necessary steps you need to take. In this article, the main focus will be on Camera2.
In either case, there are some things we need to add in our manifest file,
Camera Permission
Camera Feature
You will obviously have to deal with checking if the camera permission has been granted or not, but since this topic has been covered widely, we won’t be dealing with that in this article.
Components - Setup
The Camera2 API introduces several new interfaces and classes. Let’s break down each of them so we can better understand how to use them.
First of, we’ll start with the TextureView.
A TextureView is a UI component that is used to display a content stream (think video). We need to use a TextureView to display the feed from the camera, whether it be by preview or before taking the picture/video. Two properties that are important to use regarding the TextureView are:
- SurfaceTexture field
- SurfaceTextureListener interface
The first is where the content will get displayed at and the second has four callbacks:
- onSurfaceTextureAvailable
- onSurfaceTextureSizeChanged
- onSurfaceTextureUpdated
- onSurfaceTextureDestroyed
The first callback is crucial when using the camera since we want to be notified when the SurfaceTexture is available so we can start displaying the feed on it. Be aware that only once the TextureView is attached to a window then it becomes available.
Interacting with the camera has changed since the previous API. Now, we have the CameraManager. This is a system service that allows us to interact with CameraDevice objects. The methods you want to pay close attention to are:
After we know that the TextureView is available and ready, we need to call openCamera to open a connection to the camera. This method takes in three arguments:
- CameraId - String
- CameraDevice.StateCallback
- A Handler
The CameraId argument signifies which camera we want to connect to. On your phone, there are mainly two cameras, the front and the back. Each has it’s own unique id. Usually, it is either a zero or a one. How do we get the camera id? We use the CameraManager’s getCamerasIdList method. It will return an array of String type of all the camera ids identified from the device.
The next argument are callbacks to the camera state after we try to open it. If you think about it, there can only be several outcomes for this action:
- Either the camera manages to open successfully
- Either the camera disconnects
- Some error occurs
And that’s what you will find inside the CameraDevice.StateCallback
The third argument deals with where this work will happen. Since we don’t want to occupy the main thread, it is better to do this work in the background. That’s why we need to pass a Handler to it. It would be wise to have this handler instance instantiated with a thread of our choosing so we can delegate work to it.
With everything that we have done, we can now call openCamera:
Then in the onOpened callback, we can start to deal with the logic on how to present the camera feed to the user via the TextureView.
Components - Presenting The Feed
We got our camera (cameraDevice) and we got the TextureView to show the feed. But we need to connect them to each other so we can show a preview of the feed. To do that, we will be using the SurfaceTexture property of TextureView and we will be building a CaptureRequest.
- We get the surfaceTexture from our TextureView
- We use the cameraCharacteristics object to get the list of all output sizes
- Getting the desired size, we set it for the surfaceTexture
- We create a captureRequest where we pass in TEMPLATE_PREVIEW
- We add our input surface to the captureRequest
- Starting a captureSession with our input and output surfaces, captureStateCallback and passing in null for the handler
So what is this captureStateCallback? If you remember the diagram from the beginning of this article, it is part of the CameraCaptureSession which we are starting. This object tracks the progress of the captureRequest with the following callbacks:
- onConfigured
- onConfigureFailed
When the cameraCaptureSession is configured successfully, that’s when we set a repeating request for the session to allow us to show the preview continuously. To do that, we use the session object we get in the callback:
You will recognize our captureRequestBuilder object that we created earlier as the first argument for this method. We enact the build method so the final parameter passed in is a CaptureRequest. The second argument is a CameraCaptureSession.captureCallback listener, but since we don’t want to do anything with the captured images (since this is a preview), we pass in null. The third argument is a handler and here we use our own backgroundHandler. This is also why we passed in null in the previous section, since the repeating request will run on the background thread.
Taking a Picture
Having a live preview of the camera is awesome, but most users will probably want to do something with it. Some of the logic that we will write to take a picture will be similar to what we did in the previous section.
- We will create a captureRequest
- We will use an ImageReader and it’s listener to gather the photo taken
- Using our cameraCaptureSession, we will invoke the capture method
But what is this ImageReader? Well, an ImageReader provides access to image data that is rendered onto a surface. In our case, it is the surface of the TextureView. If you look at the code snippet from the previous section, you will notice we have already defined an ImageReader there.
As you can see above, we instantiate an ImageReader by passing in a width and height, the image format we would like our images to be in and the amount of images that it can capture.
A property the ImageReader class has is a listener called onImageAvailableListener. This listener will get triggered once a photo is taken (since we passed in it’s surface as the output source for our capture request).
⚠️Make sure to close the image after processing it or else you will not be able to take another photo.
Recording a Video
To record a video, we need to interact with a new object called MediaRecorder. The media recorder object is in charge of recording audio and video and we will be using it do just that.
Before we do anything, we need to setup the media recorder. There are various configurations to deal with and they must be in the correct order or else exceptions will be thrown. Below is an example of a selection of configurations that will allow us to capture video (without audio).
Pay attention to the setOutputFile method as it expects a path to the file which will store our video. At the end of setting all these configurations we need to call prepare.
The mediaRecorder also has a start method and we must call prepare before calling it
After setting up our mediaRecoder, we need to create a capture request and a capture session.
Similar to setting up the preview or taking a photograph, we have to define our input and output surfaces. Here we are creating a Surface object from the surfaceTexture of the TextureView and also taking the surface from the media recorder. We are passing in the TEMPLATE_RECORD value when creating a capture request. Our captureStateVideoCallback is of the same type we used for the still photo, but inside the onConfigured callback we call media recorder’s start method.
Now we are recording a video, but how do we stop recording? For that, we will be using the stop and reset methods on the mediaRecorder object:
It’s a lot to process and if you made it here, congratulations. There is no way around it - only by getting your hands dirty with the code, you will start to understand how everything connects together. You are more than encouraged to look at all the code featured in this article here:
Bear in mind that this is just the tip of the iceberg when it comes to the Camera2 API. There are a lot of other things you can do, like capturing a slow motion video, switching between the front and back cameras, controlling the focus and much more.
