Contactless Retail and Digital Communication: Biometric Authentication, Live Video Streaming, AR, VR, and AI

User Authentication: Biometrics

With the influx of online customers during Covid 19 outbreak, biometrics became one of the most reliable inputs for client authentication, and is applied either for user identification or verification. 

Identification consists in comparing a particular type of biometric data received in real time to the whole database with corresponding data that refers to multiple people, and seeing if there are any matchings. 

User verification is a simpler process, since we compare the just received data only to earlier obtained input about this user (e.g. an ID photo), and estimate the degree to which they coincide.

Let's take a closer look at peculiarities of each biometrics type.

Face recognition

Face recognition is based on a set of defined parameters (e.g. the International Civil Aviation Organization (ICAO) standards) that may include positions of whole face, eyebrows, eyes, mouth, distance between the eyes, proportions of distances between facial bones, etc. that, basically, do not change much over the lifetime.

During passive recognition, the user is directed to take a selfie which will be compared to their ID photo (and possibly to their other photos that already are in the app's database.) 

However, when a high level of security is required, just passive face recognition may be not enough, since nowadays there are plenty of photographs on social networks that can be used for fraud.

Active recognition is meant to assess liveness of a user to make sure that the interaction is with an alive person, not a bot. One of the ways to do it is by displaying the red dot on the device screen in a live video mode that the user has to follow with their eyes. 

A 2-second record is enough to determine whether that is a spoof or a real person. During this process, particular facial parameters are being measured multiple times, and their correlation with the dot location proves that that, actually, is a live user.

Some applications are capable of building a 3D face model with the received data. It allows omitting impediments that are significant when processing 2D images, e.g. the ones caused by the light angle, head position, beard, glasses, etc. 

Security precautions: it makes sense to erase liveness data after each login, so there is no risk of them being accessed by hackers. 

Voice recognition

Voice recognition is a type of biometrics that analyzes the frequency patterns of human speech, their duration, pitch, intensity, etc. Although it is considered to be of behavioral origin, it also has a solid physical background, since the anatomy details and the way a sound is formed are unique for each individual.

Voice recognition is among the most spoof-vulnerable biometric methods: 

• its accuracy may be impacted by background noise
• there may be attempts to use a high-quality speech record instead of an actual person speaking. 

This can be improved by increasing the complexity of an artificial neural network (ANN) responsible for voice recognition, and the amount of its training and testing data (voice records), which results in the app's larger size and slower response. 

Nevertheless, voice recognition has laid the fundamentals of voice commerce. Parallelled with other authentication methods (multi-factor authentication), it's an absolutely brilliant solution when it comes to voice navigation through the website, product search, contactless ordering, pick-up, and delivery! 

With a priority in real-time interaction, digital communication implements voice recognition techniques for contactless purchasing and banking-related operations, e.g. checking the credit card balance. 

Besides interpreting and following a voice command, applications with such functionality carry out the identity and liveness check in the background, providing secure and effortless user authentication. Same feature is available for AI-based chatbots for answering FAQs. That's also what stands behind voice assistants like Alexa, Google Assistant, and Siri.

Note:

• If you do prefer voice recognition authentication for your app, make sure you also implement liveness check. 
• To ensure higher accuracy and security, it's better to use voice recognition combined with other biometric methods, e.g. face recognition.

Fingerprint recognition

Fingerprint recognition is one of the most common types of biometrics that has been around for centuries: starting with fingerprint signatures in Babylonia in 2000 BC, and coming to current applications for physical access granting (international borders policy, door locks, laptop and smartphone user authentication, etc.), banking solutions, and others.

Fingerprint recognition is highly reliable from the physiological point of view, providing >99.5% accuracy. However, when we deal with large databases and user identification, it's necessary to take profound security measures to prevent data leakage.

Although biometrics provides excellent solutions for user authentication, for business owners it's crucial to foresee a backup plan of renewing access to their services, e.g. in case when the user undergoes an injury and their physical or behavioral features change.

Signature recognition

Signature recognition besides the image shape similarity considers pressure distribution on the surface, writing dynamics, some other parameters and their mutual impact on one another. 

Keystroke dynamics 

As one more kind of behavioral biometrics, it compares the way someone types particular key patterns, taking into account the time it takes to press a key and the duration of that key being pressed. 

Platforms for user identification with biometrics

Persona

With over 77M of students, Coursera is a bright example of an education marketplace platform which uses a customized Persona solution for user identification. They have implemented face recognition for registration of new users. It is based on comparison of a user's ID photo with their automatically captured selfie. 

For each further login to Coursera, keystroke dynamics analysis is used to assure that the person receiving a certificate about a course completion, and a person who has been submitting the home tasks and tests are the same individual.

Persona has official iOS and React Native SDKs.

Pricing. A free starter package with the maximum number of verifications equal to 500 and limited functionality. There is an offer of paying only for successful identifications. The price of more advanced or customized solutions is not indicated on a website and can be found out by contacting Persona directly.

TypingDNA offers keystroke dynamics solutions for user authentication and credit card payments, and has expertise in financial services and education. They provide two-factor authentication (2FA), when both unique text patterns (like password or credit card number) and the keystroke dynamics are analysed to allow completion of the particular action.

The two main solutions available are

• Authentication API:
  • Developer package is free, with open-source React Native SDK, and NodeJS client
• keystroke dynamics analysis while the user interacts with the app 
• no one-time password (OTP) functionality
• pricing: 

Pro package provides e-commerce functionality for payments and costs $0.2/month per userEnterprise package - the pricing has to be discussed for each particular case.

• Verify: 
• intelligent one-time passwords (OTP)+typing behavior analysis
• free starter kit for up to 100 000 users, pro package pricing is 1 cent per active user.

IDR&D

Established in 2016, IDR&D team has achieved significant progress in voice recognition implementation. To make sure not only that it's the right person, but that it's a real person at all, their solutions are enhanced with liveness check running in the background that is invisible to the user. 

A possible user experience may consist of

• enrollment 
• repeat a phrase written on a screen 3 times
• take 2 selfies

• log in
• read a phrase written on a screen (it usually takes just 1.3 - 1.5 seconds)
• take one selfie

IDR&D has developed a solution called SafeChat for “invisible” user authentication during their interaction with bots and virtual assistants. Some of its outstanding features include:  

• no traditional log in procedure: immediate start of interaction
• no more wasting time for entering passwords
• multi-factor biometric identification that runs in a background and is completely passive for the user 
• the authentication process runs not just for logging a user in, but during their whole interaction with the voice assistant or chatbot.

Pricing is not available on the website.

Live video streaming

Samsung Live  is a perfect example of an app where live video streaming functionality benefited the devices sales beyond the expectations, bringing 127% more purchases.

A real-time demonstration of product features helps the customer to make a reasoned decision whether the particular item suits their needs. Samsung have used Bambuser platform's functionality for live video streaming implementation. 

It also provides such great features as 

• recording and storing of live videos so that they still can bring some value to customers
• leaving comments under the videos and a real-time chat.

Recently we at Apiko have achieved a rather ambitious goal of implementing custom broadcasting functionality within an app for online wedding ceremonies. Why was that goal ambitious? 

The main challenge was to provide full HD streaming, high-quality sound, and ability to renew video streaming even during disruptions like incoming calls or internet connection outages, since everything was to be done just with a mobile phone. Such a custom module for real-time video streaming can be also beneficial for contactless retail.

Check out a more detailed overview of live video streaming platforms and our own experience of such functionality implementation in the article Going Live, Wanna Come? Live Video Streaming Platforms vs Custom Development .

Point of sale (POS)

According to the definition, a point of sale (POS) is a place where the customer pays for the purchase and completes the transaction. Besides the traditional functionality, modern POS systems provide the features required for ecommerce and remote contactless operations.

 

Shopreme won 2020 Best Retail Cases Awards for their innovative approach for the in-store checkout process during Covid 19 pandemic. BILLA supermarket network was the first to implement their Scan & Go application, which enabled the customers to scan the product barcodes with their smartphones, and pay via app whenever they have picked everything they need.

The app also provided indoor navigation to make it easier to find the items one is looking for, and payment assistant to smoothly guide the client through the payment process. No more waiting in lines, and minimized time spent at the store provided more safety and convenience to the customers, and made shopping less stressful.

For retailers Shopreme offers integration with their existing systems to minimize the set-up effort, along with customization, inventory management, analytics tools, hardware to omit product theft, and more. Native mobile SDKs provide enhanced user experience.

The pricing depends on the use case peculiarities and should be discussed with Shopreme sales department.

 

Square offers functionality for running your retail business indoor and online, and provides convenient tools for syncing these both options. The other features include (but are not limited to)  

• easy website customization and setup 
• inventory management
• payment solutions (credit card transactions, Google pay, paychecks, etc.)
• QR code ordering
• indoor pickup and delivery setup
• real-time reports
• analytics tools and CRM to better know your customers
• team management
• abandoned cart notifications, etc.

Pricing depends on the size of your business, starting with 2.9% +30 cents of each online transaction or $60 monthly. It can be reconsidered and customized for businesses with over $250.000 yearly card payments and the average paycheck larger than $15. Square also provides readers for contactless card transactions for free.

Lightspeed

Besides providing POS services for retail in general, Lightspeed has specialized products for restaurants and golf courses management. Sony and Five Guys are among their well-known clients, benefitting from the wide range of the platform's functionality.

Lightspeed offers over 60 integrations with popular tools for payments, invoicing, inventory management, employee scheduling, hospitality management, rental and reservations, etc., eliminating the necessity of using external tools.

Along with the common POS functionality, Lightspeed offers plenty of progressive features to facilitate your business, e.g.:

• Google ads integration for promoting your products to local customers
• SEO tools
• shipment and in-store pickup arrangement
• customizable gift cards, including eCards
• purchase history, analytics, marketing tools to know your customers better, etc.

Pricing ranges from $69 to $199 per month (if paid annually) depending on the features included in the package.

AR & VR, AI

AR & VR allow demonstrating the product that the customers are interested in in a way closest to what it would look like in real life. Since VR requires specialized equipment, it is not a widespread feature in retail these days. However, it has a huge potential, e.g. frequent VR use cases include virtual tours. 

Real Estate has implemented VR functionality to demonstrate houses and apartments, saving time and money required for actual visiting of multiple places. Such experience may be not enough to instantly make a decision about buying a new home, but it definitely helps to select the options one prefers.

Converse and Gap have successfully launched AR virtual fitting rooms, providing their customers with a chance to see how the clothes and shoes would look on them when shopping online. 

Gap virtual fitting room requires customer's key body measurements and builds a 3D model of a mannequin with such parameters. It automatically “dresses” it into the chosen garments, providing a 3D view with all the colors and fabric textures maintained.

Converse fitting app requires a video stream of customer's feet, breaks it into frames and determines key fitting points. Having 3D models of all shoes, it's easy to superpose a selected pair onto the feet, taking into account their angle and lighting in the video, to provide a nearly real-life look.      

IKEA has developed an AR app that allows customers to see how the selected furniture would look in their home. All you have to do is 

• pick a desired piece of furniture 
• direct phone camera to the place where you'd like to put it
• AR will place a 3D model of that piece of furniture in the chosen place on your phone or tablet screen, and you are all set to decide whether you like the look or not.

AR and VR functionality increase the order accuracy by minimizing the gap between customers' expectations and reality. This results in higher clients' satisfaction, and decrease of returns.

There are multiple ways to use AR in retail:

• in-store navigation
• in-store marketing tools (e.g. providing information about products on sale or with the discounts)
• virtual makeup try out, etc.

Besides enabling biometric recognition and AR & VR functionality, artificial intelligence (AI) is capable of analyzing customers' interaction with the ecommerce service, their orders and search data, facilitating further personalization of customer experience. Thoroughly selected suggestions on product items lead to increase of retail revenue and customer satisfaction.

Let's sum it up!

Although many useful high-tech features for retail are emerging, it doesn't mean you have to implement them all at once for your ecommerce app. Select what will add the most value to your business solution and to user experience, and grow the functionality gradually. 

Have any questions about upgrading your ecommerce app's functionality, or need to develop a new application from scratch? Feel free to contact Apiko , we are always eager to help.