Smarter Door

Links

Smarter_Door_Log
Amazon Reckognition Tutorial
GitHub

Team Members

• Jonathan Strek
• Katie Cardwell
• Andrew Koltz
• Ethan Shry (TA)
• Professor Feher

Overview

We are building a door security system that sends the user a picture when an unknown person tries to open the door. The user will receive a text and be able to interact with the door by responding over text. Our security system includes a red light that illuminates when the user says the visitor is not welcome, or when the user has not authorized entry after a set period of time.

Goals

• Photograph anyone who tries to open door
• Recognize owner from an uploaded photo collection
• Text owner if unrecognized face is seen
• Respond to owner's text with security light
• Design plastic housing to make unit presentable

The goal of this project is to build a facial recognition security system for a door. The system will increase security by taking a picture of anyone who tries to open the door. If the person is not recognized by the system, a text will be sent to the user informing him/her that someone has tried to open the door. If the user does not recognize the person in the picture, they can respond via text and a security light will go on. This light should discourage people from trying to open a random door, and it also indicates to law enforcement that someone unknown has tried to enter the door.

Gantt Chart

Challenges

• Learning Python
• Gaining familiarization with Twilio
• Security of user interface
• Building a 3D casing for raspberry pi detector
• Ensuring motion detector only signals to camera to take a picture when there is truly someone in front of the door rather than when people walk by or there is motion in the background

Security Concerns

• Who can upload a picture to be recognized
• Who receives the text from the Raspberry Pi
• Who can text the Raspberry Pi

Budget

• Raspberry Pi: provided ($35)
• Raspberry Pi Camera: provided ($12)
• Infrared PIR motion Sensor: ($12) (https://www.adafruit.com/product/189?gclid=CjwKCAiAqaTjBRAdEiwAOdx9xo34FxU9DTSkwjdc8BoTUGuFJe6SMUgkO7Ussy_YMhy5Z-2xehN72xoCVYsQAvD_BwE)
• Twilio Messaging ($10)
• Amazon Web Services Recognition: ($5)
• 3D Printing software and materials: ($10)
• Small breadboard ($6)
• a LED
• a 330 ohm resistor

Total: $90

Why this project?

Security in dorms or houses is always a concern, and we hope to alleviate such worries. The increase in dependence on online retailing means more strangers than ever are using your address to find your place of residence. With Smarter Door you will not have to worry about who is at the door. Dorms can also be safer as a red light outside of a dorm will be a sign that an RA or even WUPD should be called.

Design and Solutions

Setup

• Download development environment (we used Visual Studio and Atom)
  • Download Python, Node, CSS, and HTML extensions
• Download MySQL
• Create AWS instance
• Create AWS S3 bucket
  • Connect to raspberry pi
• Create AWS database instance
  • Connect to MySQL
• Purchase Twilio number that can send and receive messages

Modules

General Project Diagram:

• Door

Our project uses a raspberry pi to combine the inputs and outputs of a PIR motion sensor, a Raspberry Pi Camera and various colored LEDs. If motion is detected (one of the LED bulb lights up), the camera takes a picture (while the other blue LED is on) and sends the picture to S3 to undergo facial recognition (a yellow light is shown to indicate the picture being processed). If Amazon Rekognition recognizes the user, a green light is displayed for 7 seconds before running again. If the user is not recognized, the picture is sent to the owner of the door asking for approval. If the owner texts back a positive response, a green light is displayed. If they text back a negative response, the blue, yellow and green light all go on, indicating that this person is an intruder. A button is placed into our system for demoing purposes, acting as a backup for the motion sensor. Below is the circuit schematic for our physical system and a flowchart to display the process of recognition at the door.

• Server
  • AWS Lightsail Server alerts the door owner that there is someone at the door, sends picture to user, and receives reply
    • These objectives are achieved by utilizing routes and Twilio. The server is continuously listening on an open port, when the Pi sends a picture to the bucket it gives the server the name of the photo. The server then uses the name along with a static ip for the public s3 bucket to send the picture using the Twilio number. The user is prompted to reply to the photo, and the text is sent back to Twilio, which uses a webhook to send the content of the reply back to the server. The server can send that information to the Pi, which changes lights based on the reply.
• Database
  • Stores information and pictures of who tried to enter

Database:

• S3
  • Bucket holds a "list" of known users that the unknown user is tested against
  • Bucket holds pictures of who tried to enter with filename indicative of the time of occurence
• Web UI
  • Allows the user to access the information from the database
    • Currently includes a landing page (buttons are for learning HTML, will be removed), and a link to a page which displays the visitors to the door.

• New picture

• AWS recognition
  • Detects a face in the taken image and compares various facial features to determine a confidence number of the face being known
  • Determines if the person at the door is the owner of the door by comparing the confidence number to a predetermined confidence threshold
• API
  • Communicates with the door owner using Twilio

Features & Design Choices

We used Visual Studio to code in HTML, CSS, Node, and Python to run our server, and our Raspberry Pi. The Amazon Web Service Lightsail server allowed us to communicate between the door, the facial recognition, and the end user. The Raspberry Pi controls the motion sensing and picture taking, and uploads the pictures to AWS to be used in facial recognition.

The majority of the project is comprised of the Raspberry Pi, AWS server, facial recognition, bucket, and Twilio. We chose the Pi because it is easy to use, yet complex enough to handle communications with the server. Amazon Web Service was chosen because it provided us with an affordable and easy to use suite of web services to handle communications. The Lightsail server listens to the pi, contacts the user, receives the decision, and sends the decision back to the Pi. The facial recognition was an easy way to check the new pictures against previously uploaded images of the user. The s3 bucket allowed us to store the new images from the Pi and send them with the server to the user. We chose Twilio because it was a simple way to communicate from a third party to the user. For a small amount anyone can purchase a Twilio number with the ability to send texts, and with minimal console work can write a script to send anything from that message to any number in the United States or Canada.

Learning Raspberry Pi

• Made blinking LED

References

• Twilio set up PI https://www.twilio.com/docs/sms/quickstart/node#send-an-outbound-sms-message-with-nodejs
• Proposal presentation Proposal Presentation
• Twilio add on installation [1]