Linetracking with opencv

Overview

This Tutorial will show how to make a line detection and find a center of line for a line following pi car.

Materials/Prerequisites

You will need a Raspberry pi 3 with python installed and Pi camera (Webcam works too). you also need to install OpenCV onto your Raspberry Pi. We can follow instructions to install OpenCV. If it is the first time to use pi camera, to be able to access the Raspberry pi camera with OpenCV and Python, we recommend to look at this instruction. For interfacing with the Raspberry Pi camera module using Python, the basic idea is to install picamera module with NumPy array support since OpenCV represents images as NumPy arrays when using Python bindings.

Process

If you are using Webcam, you can simply follow this link to realize line detection. The code is given below:

import sys
import time
import cv2
import numpy as np
import os

Kernel_size=15
low_threshold=40
high_threshold=120

rho=10
threshold=15
theta=np.pi/180
minLineLength=10
maxLineGap=1

Initialize camera
video_capture = cv2.VideoCapture(0)

while True:
    # CAPTURE FRAME-BY-FRAME
    ret, frame = video_capture.read()
    time.sleep(0.1)
    #Convert to Grayscale
    gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
    #Blur image to reduce noise. if Kernel_size is bigger the image will be more blurry
    blurred = cv2.GaussianBlur(gray, (Kernel_size, Kernel_size), 0)
    
    #Perform canny edge-detection.
    #If a pixel gradient is higher than high_threshold is considered as an edge.
    #if a pixel gradient is lower than low_threshold is is rejected , it is not an edge.
    #Bigger high_threshold values will provoque to find less edges.
    #Canny recommended ratio upper:lower  between 2:1 or 3:1
    edged = cv2.Canny(blurred, low_threshold, high_threshold)
    #Perform hough lines probalistic transform
    lines = cv2.HoughLinesP(edged,rho,theta,threshold,minLineLength,maxLineGap)
    
    #Draw cicrcles in the center of the picture
    cv2.circle(frame,(320,240),20,(0,0,255),1)
    cv2.circle(frame,(320,240),10,(0,255,0),1)
    cv2.circle(frame,(320,240),2,(255,0,0),2)
    
    #With this for loops only a dots matrix is painted on the picture
    #for y in range(0,480,20):
            #for x in range(0,640,20):
                #cv2.line(frame,(x,y),(x,y),(0,255,255),2)
    
    #With this for loops a grid is painted on the picture
    for y in range(0,480,40):
            cv2.line(frame,(0,y),(640,y),(255,0,0),1)
            for x in range(0,640,40):
                cv2.line(frame,(x,0),(x,480),(255,0,0),1)
                
    #Draw lines on input image
    if(lines != None):
        for x1,y1,x2,y2 in lines[0]:
            cv2.line(frame,(x1,y1),(x2,y2),(0,255,0),2)
            cv2.putText(frame,'lines_detected',(50,50),cv2.FONT_HERSHEY_SIMPLEX,1,(0,255,0),1)
    cv2.imshow("line detect test", frame)

    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

# When everything is done, release the capture

video_capture.release()
cv2.destroyAllWindows()

Our group is using pi camera so the code needs to be modified. Otherwise it will show Nonetype error due to an image not being read properly from disk or a frame not being read from the video stream. You cannot use cv2.VideoCapture when you should instead be using the picamera Python package to access the Raspberry Pi camera module.

There are two ways to modify the code to replace USB Webcam with pi camera.

One is swapping out the cv2.VideoCapture for the VideoStream that works with both the Raspberry Pi camera module and USB webcams. more details

We did another way. Firstly we import some pi camera packages

from picamera.array import PiRGBArray
from picamera import PiCamera

when initializing camera, we comment out video_capture = cv2.VideoCapture(0). Instead, we give code below:

#Initialize camera
camera = PiCamera()
camera.resolution = (640, 480)
camera.framerate = 10
rawCapture = PiRGBArray(camera, size=(640, 480))

We also change the "While True" loop into:

for f in camera.capture_continuous(rawCapture, format="bgr", use_video_port=True):

Finally, we comment out "ret, frame = video_capture.read()" and add:

frame = f.array

Now you should be able to realize line detection with pi camera. However, you might find it is not satisfying because sometimes the video window suddenly disappears and the terminal gives you a Nonetype error again. In addition, the lines appearing in the window are more likely some sparse line segments or even dots. In fact, those problem stem from HoughLineP function. For more understanding of HoughLineP function, you can check This Link. Basically, cv2.HoughLinesP returns us an array of array of a 4-element vector(x_1, x_2, y_1, y_2) where (x_1,y_1) and (x_2, y_2) are the ending points of each detected line segment.

The sample of cv2.HoughLinesP return value:

print(cv2.HoughLinesP) = [[[a b c d]]]

When we there is no line shown is the camera, cv2.HoughLinesP returns null(or None?) so that is the reason why the video window suddenly disappears. The code cv2.imshow("line detect test", frame) is in the if statement in which (lines != None). To maintain the window when there is no line detected, we could modified the code as below:

if(lines is not None):
        for x1,y1,x2,y2 in lines[0]:
            cv2.line(frame,(x1,y1),(x2,y2),(0,255,0),2)
            cv2.putText(frame,'lines_detected',(50,50),cv2.FONT_HERSHEY_SIMPLEX,1,(0,255,0),1)
    cv2.imshow("line detect test", frame)
else:
        cv2.putText(frame,'lines_undetected',(50,50),cv2.FONT_HERSHEY_SIMPLEX,1,(0,255,0),1)
        cv2.imshow("line detect test", frame)

After we understand the meaning of cv2.HoughLinesP function, we can modified the loop in the if statement since the old one "for x1,y1,x2,y2 in lines[0]:" only give us sparse line segments. The modified loop is:

for line in lines:
            for x1,y1,x2,y2 in line:

Authors

Pi Car Discovery Group

Group Link

group project page

Weekly logs