Linetracking with opencv
This Tutorial will show how to make a line detection and find a center of line for a line following pi car.
You will need a Raspberry pi 3 with python installed and Pi camera (Webcam works too). you also need to install OpenCV onto your Raspberry Pi. We can follow instructions to install OpenCV. If it is the first time to use pi camera, to be able to access the Raspberry pi camera with OpenCV and Python, we recommend to look at this instruction. For interfacing with the Raspberry Pi camera module using Python, the basic idea is to install picamera module with NumPy array support since OpenCV represents images as NumPy arrays when using Python bindings.
If you are using Webcam, you can simply follow this link to realize line detection. The code is given below:
import sys import time import cv2 import numpy as np import os Kernel_size=15 low_threshold=40 high_threshold=120 rho=10 threshold=15 theta=np.pi/180 minLineLength=10 maxLineGap=1 Initialize camera video_capture = cv2.VideoCapture(0) while True: # CAPTURE FRAME-BY-FRAME ret, frame = video_capture.read() time.sleep(0.1) #Convert to Grayscale gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY) #Blur image to reduce noise. if Kernel_size is bigger the image will be more blurry blurred = cv2.GaussianBlur(gray, (Kernel_size, Kernel_size), 0) #Perform canny edge-detection. #If a pixel gradient is higher than high_threshold is considered as an edge. #if a pixel gradient is lower than low_threshold is is rejected , it is not an edge. #Bigger high_threshold values will provoque to find less edges. #Canny recommended ratio upper:lower between 2:1 or 3:1 edged = cv2.Canny(blurred, low_threshold, high_threshold) #Perform hough lines probalistic transform lines = cv2.HoughLinesP(edged,rho,theta,threshold,minLineLength,maxLineGap) #Draw cicrcles in the center of the picture cv2.circle(frame,(320,240),20,(0,0,255),1) cv2.circle(frame,(320,240),10,(0,255,0),1) cv2.circle(frame,(320,240),2,(255,0,0),2) #With this for loops only a dots matrix is painted on the picture #for y in range(0,480,20): #for x in range(0,640,20): #cv2.line(frame,(x,y),(x,y),(0,255,255),2) #With this for loops a grid is painted on the picture for y in range(0,480,40): cv2.line(frame,(0,y),(640,y),(255,0,0),1) for x in range(0,640,40): cv2.line(frame,(x,0),(x,480),(255,0,0),1) #Draw lines on input image if(lines != None): for x1,y1,x2,y2 in lines: cv2.line(frame,(x1,y1),(x2,y2),(0,255,0),2) cv2.putText(frame,'lines_detected',(50,50),cv2.FONT_HERSHEY_SIMPLEX,1,(0,255,0),1) cv2.imshow("line detect test", frame) if cv2.waitKey(1) & 0xFF == ord('q'): break # When everything is done, release the capture video_capture.release() cv2.destroyAllWindows()
Our group is using pi camera so the code needs to be modified. Otherwise it will show Nonetype error due to an image not being read properly from disk or a frame not being read from the video stream. You cannot use cv2.VideoCapture when you should instead be using the picamera Python package to access the Raspberry Pi camera module.
There are two ways to modify the code to replace USB Webcam with pi camera.
One is swapping out the cv2.VideoCapture for the VideoStream that works with both the Raspberry Pi camera module and USB webcams. more details
We did another way. Firstly we import some pi camera packages
from picamera.array import PiRGBArray from picamera import PiCamera
when initializing camera, we comment out video_capture = cv2.VideoCapture(0). Instead, we give code below:
#Initialize camera camera = PiCamera() camera.resolution = (640, 480) camera.framerate = 10 rawCapture = PiRGBArray(camera, size=(640, 480))
We also change the "While True" loop into:
for f in camera.capture_continuous(rawCapture, format="bgr", use_video_port=True):
Finally, we comment out "ret, frame = video_capture.read()" and add:
frame = f.array
Now you should be able to realize line detection with pi camera. However, you might find it is not satisfying because sometimes the video window suddenly disappears and the terminal gives you a Nonetype error again. In addition, the lines appearing in the window are more likely some sparse line segments or even dots. In fact, those problem stem from HoughLineP function. For more understanding of HoughLineP function, you can check This Link. Basically, cv2.HoughLinesP returns us an array of array of a 4-element vector(x_1, x_2, y_1, y_2) where (x_1,y_1) and (x_2, y_2) are the ending points of each detected line segment.
The sample of cv2.HoughLinesP return value:
print(cv2.HoughLinesP) = [[[a b c d]]]
When we there is no line shown is the camera, cv2.HoughLinesP returns null(or None?) so that is the reason why the video window suddenly disappears. The code cv2.imshow("line detect test", frame) is in the if statement in which (lines != None). To maintain the window when there is no line detected, we could modified the code as below:
if(lines is not None): for x1,y1,x2,y2 in lines: cv2.line(frame,(x1,y1),(x2,y2),(0,255,0),2) cv2.putText(frame,'lines_detected',(50,50),cv2.FONT_HERSHEY_SIMPLEX,1,(0,255,0),1) cv2.imshow("line detect test", frame) else: cv2.putText(frame,'lines_undetected',(50,50),cv2.FONT_HERSHEY_SIMPLEX,1,(0,255,0),1) cv2.imshow("line detect test", frame)
After we understand the meaning of cv2.HoughLinesP function, we can modified the loop in the if statement since the old one "for x1,y1,x2,y2 in lines:" only give us sparse line segments. The modified loop is:
for line in lines: for x1,y1,x2,y2 in line:
It would have a better result if we set both minLineLength and maxLineGap to be 0.
Now, we should be able to detect lines. How about the center of the lines? How do we find the center of it?
What we did is declare two variable x and y initialized to be 0. Then in the innermost loop of drawing lines from images, we set x=x+x1+x2 and y=y+y1+y2. Next we xa=x/(2*len(lines)*len(lines))
Pi Car Discovery Group