Mobile Security Robot

by Paul Wilson, Zhen Lui, Haitian Sun, and James Knowles, III.

Overview

Mobile Security Robot is a security solution that extends the limits of stationary at home security. The idea is that when the security system is powered, a mobile robot can examine an extended range of areas. If it detects any motion the robot triggers an image capture. Image analysis is subsequently performed to validate the alarm. If substantial changes in the images are confirmed, the owner is then notified of the alert. Ideally, this alert would include an image of what was detected.The iRobot Create's movement is controlled by an mbed device that gathers data about its surroundings. When the security system is being calibrated for setup it should examine the distances around it (at specific locations). If those values are changed past a threshold in real-time, the alarm will trigger causing a sequence of events to follow. This project was created to satisfy the final design project component of our Embedded Systems Design course.

Goals

The main objectives for the Mobile Security Robot project are below.

• Control the Create's movement using an mbed
• Detect motion or changes in the space around the robot
• Capture and store image frames of the motion that is detected
• Wirelessly communicate this data with a server
• Perform image analysis on the image to validate the alarm trigger
• Notify the security owner

Interfacing with Create

See the snippet below for rotating the Create and capturing a photo.

void moveAndPhoto(int dir) {
    // dir: 1 stands for counter-clockwise, 2 for clockwise
    led2 = 1;
    if (dir == 1) {
        left(); // rotate counter-clockwise
        wait(1.3);
        stop(); // stop
    } else if (dir == 0) { // moving clockwise
        right();
        wait(1.3);
        stop();
    }
    led2 = 0;
    picture(); // take picture
}

// Left - drive motors set to rotate to left
void left() {
    device.printf("%c%c%c%c%c", DriveDirect, char((speed_right>>8)&0xFF),  char(speed_right&0xFF),  
    char(((-speed_left)>>8)&0xFF),  char((-speed_left)&0xFF));
}

// Right - drive motors set to rotate to right
void right() {
    device.printf("%c%c%c%c%c", DriveDirect, char(((-speed_right)>>8)&0xFF),  char((-speed_right)&0xFF),  
    char((speed_left>>8)&0xFF),  char(speed_left&0xFF)); 
}

Components

The major components to this project are categorized by device and cloud. The essential modules are below.

What's on the Device?

Sonar modules

Sonar modules (MB1210) are used to detect changes in the space around the robot. After calibration, the system expects a range of values. If the sonar modules return any values outside of the threshold, an alarm is triggered.

Image Sensor

Following the detection, the mbed rotates the Create camera side front to capture the image. The image sensor used is LS-Y201 Camera. The data is then stored on an SD card.

Wireless module

The mbed will then send this data to a web server for image analysis. To implement the communcation, a Wifly module is being used to connect to a wireless network for data transmission. As a backup we have verified that we can transmit data via Ethernet.

What's in the Cloud?

A web server is used for image analysis as well as emailing notifications to the security owner. Both scripts are written in python. The web server storage and processes are via AWS.

Image Analysis

A quick, but effective way to detect changes in images is using a histogram of the hue saturation values. When the Create is calibrated, it stores images in an array based on position (index). This system creates a histogram of the original image and compares it to a histogram of the space where the detection occurred. If the histograms differ outside of a threshold detection is confirmed. We are using OpenCV's Histogram Comparison for the image analysis on the data.

Code below shows snippets for running the histogram comparison.

import cvs
import numpy as np

# Convert BGR to HSV
hsvOld = cv2.cvtColor(oldImage, cv2.COLOR_BGR2HSV);
hsvNew = cv2.cvtColor(newImage, cv2.COLOR_BGR2HSV);

# Red, green, blue channels for histogram computations
int channels [] = {1,2,3};

# Take histogram of old and new
histOld = cv2.calcHist([hsvOld],[channels],None,[256],[0,256]);
histNew = cv2.calcHist([hsvNew],[channels],None,[256],[0,256]);

# Compare histograms using correlation method
double check = compareHist(histOld,histNew,cv2.cv.CV_COMP_CORREL);

# Threshold for correlation
if check < .85

# return detection via email script

Alerting the user

An SMTP package was installed on the web server. This gives the security system the ability to send email alerts to the owner.

Code below shows the linux commands for installing the SMTP package on the web server.

Current Status

Our project currently works as 2 individual programs. The embedded program controls our Create, detects motion, captures images, and sends the image data to the server. Our second program is hosted on the server. It processes the image data and sends email notifications to the owner. While we were unsuccessful in sending our data wirelessly, our mbed can successfully transmit data via Ethernet.

Moving forward

The image analysis code has not yet been fully vetted. Moving forward we would combine it with our server code to validate our motion detection. As well as determine the reason our wireless attempts were unsuccessful.

Other Important Snippets of Code

Capturing an image via mbed

void picture() {
    char fname[64];
    snprintf(fname, sizeof(fname) - 1, FILENAME, imgNum);
    int r = capture(&cam1, fname);
    if (r == 0) {
        DEBMSG("[%04d]:OK.", imgNum);
        NEWLINE();
    } else {
        DEBMSG("[%04d]:NG. (code=%d)", imgNum, r);
        NEWLINE();
        error("Failure.");
    }
    wait(3);
    imgNum++;
}

int capture(Camera_LS_Y201 *cam, char *filename) {
    /*
     * Take a picture.
     */
    if (cam->takePicture() != 0) {
        return -1;
    }
    DEBMSG("Captured.");
    NEWLINE();
 
    /*
     * Open file.
     */
    work.fp = fopen(filename, "wb");
    if (work.fp == NULL) {
        return -2;
    }
 
    /*
     * Read the content.
     */
    DEBMSG("%s", filename);
    NEWLINE();
    if (cam->readJpegFileContent(callback_func) != 0) {
        fclose(work.fp);
        return -3;
    }
    fclose(work.fp);

    /*
     * Stop taking pictures.
     */
    cam->stopTakingPictures();
    return 0;
}

Repository

Solution for robot controls, motion detection, and image capture

Solution for sending image data to server

Server Code - used for notifications /media/uploads/pwilson39/finalprojectpython.py

Result