Smart shopping Cart

While shopping, customers always grab a shopping cart, put merchandise into the cart, and finally proceed to the check-out stand during the whole shopping process in the market. Our smart shopping cart system will bring remarkable ease to our customers by providing various services such as collecting customer ID, recognizing merchandise, and payment automation within a single shopping cart. The system will be implemented on each shopping cart which is placed in front of the market. The system will be activated to serve the user during shopping once it detects a person, and the user will automatically login if his/her face ID is recognized. During shopping, the system provides various interactive services for the customers. When checkout, the smart shopping cart will scan through the cart and send the list of purchased items to the customer. The user only needs to give his/her password to the cart through the touch panel for check-out.

We implement this system with the idea of edge computing. We use google cloud as our cloud for user authorization, sales statistics, and data storage and management. The fog runs on a computer for data transmission between edge and the Cloud, and between edge and edge. For the edge devices, we use two Raspberry Pi, one using a camera to detect items and the other using a camera, a microphone and a touch screen to interact with the user.

System Concept Graph

Detailed Diagram of the Network and Layers

Libraries and Packages We Are Using

For edge devices, we use the two trained models from github, TensorFlow Lite Object Detection on Android and Raspberry-Pi and a common face-recognition library, to do object detection and face recognition.

For the voice recognition technique, we use the package called SpeechRecognition to record voice and then translate speech to text. The specific function we are using is recognizer_instance.recognize_sphinx

For a more refined GUI mockup, we use PyQt5 for buttons drawing, image embedding, and windows setup.

We implement the system communication based on ZeroMQ, an embeddable networking library.

We use Python-RSA library to support the encryption and decryption on the password.

We use Quick Easy SMS API to send SMS to users after they successfully register and checkout.