Next Level Shopping Experience - Adapt NAO Humanoid Robot Features In Customer Care

Humanoid Robot ?

Humanoid Robot is a machine which is a combination of hardware and software and it can interact with humans, but for a specific purpose or a domain since it only programmed for one specific area.

NAO is also a humanoid robot which is a product of Aldebaran Robotics and it includes many advanced features like   wireless capable and programmable with abstract interfaces. It introduces specific tasks such as face recognition & detection, voice recognition & detection, voice to text translation, text to speech with some advanced gestures.

Click this to find features of NAO humanoid robot.

Integrate a Display to NAO

Here I tried to display product description with some product related attributes, views and some recommendations through a product catalog when a customer requests a product search (voice based request) from NAO. When a customer appears in front of NAO, It greats (waves hands and greet according to the time) the customer and then asks for a product search. When that client performs a voice based product search, NAO extracts the product name, product specific attributes like color, product name, size etc using the pattern matching approach with a help of set of predefined grammar rules.

Then extracted product related attributes are sent to a back end server. Products details (Image, description etc.) displays on a tab - when NAO gets a SUCCESS response from the back end server and then it points the display to the customer.

1. NAO robot detects voice request and extracts product attributes using grammar rule extractor.
2. Send product attributes through a web socket connection  to the tablet application and tablet application forwards the request to the Product API.
3. Captures the product search request and performs indexed search on given attributes.
4. Sends product result response back to tablet application ( List of products with images and descriptions). Check the response state.
5. Displays the responses on the display.
6. Sends back SUCCESS response to NAO ( if tab application timeout / FAIL state for product search request - application sends FAIL state response back to NAO and terminate socket connection)

In addition to the product display, application captures customer selected (clicked) product name and sends it back to the NAO as a product select event. In order to keep the interaction with customer and provide a good expereince, NAO responses and do some activities  based on the selected products (Give some promotion codes etc.) .

I have used NAO’s sonar sensor to identify whether the customer still stand nearby if the customer does not engage with the NAO. if the customer no longer engage with NAO, it waits some time and end the customer session while coming back to the default position and setting the display to main page.

NAO programming : Python +  NAO Python API

Backend Server : Node Js

Tablet Application : Node Js + Angular Js

NOTE: Aldebaran Robotics has introduced a new humanoid robot called Pepper targeting retail industry with a inbuilt display and Romeo with more advanced features.