Cloud Face Recognition for Mobile Applications: Overview
Best Face Recognition APIs in Comparison
Face Recognition has become very trendy: it is used in distributed surveillance, service robots or domestic appliances such as TV-sets. However, mobile devices need a faster and a more independent way to recognize faces. One possible solution is to use a cloud-based server. Today there is a number of cloud services that offer this functionality.
This series is about how we can use face recognition in mobile applications and it consists of three parts:
Part 1: Overview of some popular face recognition services
Part 2: Implementation example in Android
Part 3: Results and performance of chosen face recognition providers
After looking for suitable face recognition providers for a long time, I came to the conclusion, that there are only five face recognition services that are worth it: Amazon Rekognition, Microsoft Azure, Face++, Kairos, Luxand. There were other smaller competitors in question, such as Animetrics, Macgyver, Betaface and Lambda labs, but in the end it was either impossible to find pricing information on these providers or their demo versions were broken. I am also not considering such well known tech companies as Google and IBM, because they only provide face detection services and no face recognition. Open source face recognition projects, such as OpenFace or Face Recognition are also excluded, because they run in Python environment and implementation within an Android device would be too complicated. Certainly, there are tools that enable running Python code in Android environment, however, this remains a topic for future discussions.
Overview
First of all I wanted to know what to expect from these providers and decided to compare them in terms of the following criteria:
- Form of access: do they provide an Android SDK or do I have to implement an API on my own?
- Documentation: is there documentation, code samples and tutorials available?
- Data hosting: where is the data stored?
- Community: how many related requests are there on StackOverflow?
- Security: how is data secured?
- Trial: is there a trial period and how long is it?
- Costs: how much does the service cost?
- Extra features: are there any extra features besides basic face identification?
Developer experience
After comparing several face recognition services in terms of preliminary criteria, I couldn’t wait to try them out as a developer.
Microsoft Azure
Microsoft Azure offers an Android SDK along with a sample app, which makes it really easy to explore and implement the API. The documentation is very well written and has enough code samples.
What’s more, Azure has a person entity feature. It means that Azure enrols persons and each of the persons has a list of face. Having a group of persons in the cloud is a great feature, because otherwise the identification function returns faces instead of persons. For example, if someone has visited the building five times, there must be five similar photos of him in the cloud database. As a result, on his sixth visit, the identification service will return an array with those five faces. If you are only interested in the person’s identity, you’ll need to implement a client-side logic of filter through the returned faces. Currently, my app looks for a visitor in the local database for every returned result from the cloud services. In case one of the services returns the same person several times, only the result with the highest confidence is chosen for the list of candidates.
Being the most detailed face recognition service, Microsoft Azure requires more time to get the hold of all the available methods and parameters. However, after getting to know all the features and implementing them correctly, it works seamlessly. The photos can be uploaded to the cloud either as URL or as an InputStream. The confidence is measured by Azure in decimal figures.
Amazon Rekognition
Just like Azure, Amazon also provides a convenient way of implementing the API via Android SDK. It also has detailed documentation with clear code samples.
However, unlike Microsoft Azure it only offers a Free Tier which allows to analyse 5000 images and store 1,000 images per month. What’s more, the registration process required entering credit card details and on the whole was rather cumbersome. The setting up of the AWS console takes a lot of time, and the console itself is not helpful in terms of monitoring the Free Tier usage.
Amazon Rekognition follows the standard face recognition interface flow by first enrolling a face to the database and then identifying the visitor using the enrolled faces. On the good side, it allows to set an externalImageId:String for the image. As my system only enrols photos with one face, I used this option to relate the local database to the cloud.
The photo itself can be uploaded either as base64-encoded image bytes or as a reference to an image in an Amazon S3 bucket. The confidence level returned by Amazon Recognition is measured in percentage, so to be consistent with other services, it has to be converted to decimal.
To sum up, Amazon Rekognition seems a very controversial service in terms of developer experience. On the one hand, it provides all the necessary tools for easy integration, but on the other hand, it is very complicated to monitor your budget. Furthermore, it is difficult to get in contact with the support, which is not available for free tier plans, and the questions posted on the official forum do not get attention for weeks. Amazon Rekognition is a cloud service that will make you paranoid.
Face++
The service provided by the Chinese company Face++ is another completely free option. It does not require any credit card details during the registration, only the email address and the phone number. Moreover, there are almost no restrictions on the usage of the API, apart from the 1000 image limitation on a FaceSet. The documentation is full and concise, with relevant response samples.
Although they also provide a full Android SDK for limited use, in case of a small application where only some API features are used, it is easier to implement the API using Retrofit. Face++ face recognition API follows the face recognition interface strategy, the only pitfall was having to remove all face tokens from a particular FaceSet before deleting this FaceSet. Moreover, API key and API secret have to be sent as headers with every call.
In terms of relation to the local database Face++ recognition service behaves like Amazon Rekognition and allows to set an external id to every enrolled face. The search in local database is analogous to the one for Amazon Rekognition.
The images can be sent to the cloud either as a URL, a File or a base64 encoded String. The confidence is measured in percent and has to be recalculated to decimal.
On the whole, the experience implementing face recognition service by Face++ proved to be rather pleasant. The documentation is helpful, there are no hidden costs and it has full functionality even in its trial version.
Kairos
Kairos is an independent company specializing on Face Recognition. They do not offer an Android SDK, but the API itself is easy to implement with Retrofit. Documentation has sufficient information on request and response models. The trial version is only available for 14 days and registration requires entering credit card details. Face recognition flow is very simple: only three API methods had to be used to implement the logics.
Therefore, this service seems to be appropriate for small and unsophisticated applications. What’s more, it was rather easy to get in touch with the support and they responded quickly.
The relation to the local database is similar to Amazon Rekognition and Face++ scheme. Photos can be submitted either as a publicly accessible URL, Base64 encoded photo or as a file upload. The confidence is measured in decimal and does not require conversion.
To sum up, Kairos is similar in functionality to other face recognition services and is very easy to use. However, it has basically no trial version, so the performance has to be exceptional if the service provider wants the software developers to use it.
Luxand
Luxand is another small independent company providing face recognition. Like Kairos, it has 14 days trial period and requires credit card details. But unlike other services, the documentation is rather poor. Even though it has various code samples in different programming languages, sometimes it was difficult to find general information on the endpoints. For example, the documentation does not mention the threshold parameter, which is set to default = 0.78 and can be changed if sent with the request body.
One interesting moment about this service is that just like Microsoft Azure it has a person entity. So, it is necessary to first enrol the person and then add a face to this person. As a result, the search request returns a person instead of a face. The images can be uploaded to the service as a File and the confidence is measured in decimal figures.
However, there was difficulty performing search requests, because even after setting the confidence threshold to 0.0, the service returned only one person in the resulting array. It was easy to get in touch with the support and developers and I found out, that the service always returns only one person, which it sees as the best match for the requested face. After a feature request, Luxand developers quickly added a parameter that could be sent to obtain not only the highest likely face, but an array of all the candidates with corresponding confidence levels. In this regards I found Luxand developers to very effective and approachable, unlike the support of big corporations such as Microsoft and Amazon. Despite this fact, the face recognition API still lacks some important features.
Generally, Luxand Face Recognition seems to be a rather young service with good perspectives. On the one hand, the documentation lacks clarity and the trial period is only 14 days, but on the other hand, it provides good technical support and flexibility, so it has all chances to compete with other service providers.
To sum up, all of the five services seem to be promising options for building a face recognition application. All of them have good and rather clear documentation. Some services like Azure and Rekognition seem to be more evolved coming from major IT giants with its vast functionality and SDKs available. The others, like Face++, Kairos or Luxand may be the dark horses in face recognition.
In the next part of this cloud face recognition series you will learn more about what requirements have to be considered before building a prototype and how I designed it to compare the performance of all five providers.
