CV Chess Logs

Link to project page

Weekly log:

Week 1 Sept. 9 - 15

Robert Goodloe

• (0.25hr) September 12: Created project page and weekly logs.

Team

• (1.5hr) September 13: Built night light circuit and designed print-able case. Unsure which dimensions to use exactly on case until we get the perfboard.

Week 2 Sept. 16 - 22

Robert Goodloe

• (0.75hr) September 18: Flashed raspbian wheezy operating system onto SD card to set up Raspberry Pi (rpi). Researched various ways to install/use openCV on rpi.
• (0.5hr) September 19: Updated Wiki to include the two weeks of work done up to this point.

Nhut Dang

• (1hr) September 19: Soldered night light circuit to newly obtained perfboard

Team

• (2hr) September 20: Attempted OpenCV compilation and install. An error was made by trying to compile on all 4 of the rpi's CPU cores. This led to overheating and a frozen installation.
• (1.5hr) September 22: TA Ethan Shry introduced us to python installation manager 'Pip'. Pip should allow a smaller/simpler install of OpenCV components without long compile time. Ran into issues with OpenCV modules

Week 3 Sept. 23 - 29

Robert Goodloe

• (2.5hr) September 24: Compiled OpenCv on rpi. The pip install was abandoned because to many issues arose trying to install cv2 module. OpenCV traditional compile and

install was done instead. Being ~4GB in size, a new 16GB SD card was used rather than the original 8GB. To avoid the overheating problem, the I let the rpi compile overnight using 1 CPU core. Total install time: ~4 hours. Looking to try face detection tutorial in the next few days to familiarize myself with the software.

• (1.5hr) September 26: Research: how to calibrate rpi camera, Corner detction algorithm vs line detection algorithm, related projects that already exist
• (2hr) September 27: Set up ssh for rpi. SSH was working after 15 mins, rest of time spent unsuccessfully attempting to get pi to automatically email IP address upon boot or setting a static IP address.
• (3hr) September 28: Compiled a list of tutorials for OpenCV on rpi. See refereces section of Project Page. Set Up VNC Viewer and added the execution of a python script to mail the IP address upon start. Managed to complete the equivalent of a "Hello World" tutorial for OpenCV. At its completion, the Pi could recognize faces in an image. I also discovered how to display the input from the camera to the desktop.

Nhut Dang

• (2hr) September 24: Compile the list of goods needed. Thinking about the problem with the chessboard if viewing from the bottom
• (1hr)September 26: Research about light up dance floor, designing the final product, finalizing the budget.
• (1hr) September 28: redesigning the powerpoint, updating the budget, researching about the line detection and corner detection, trying to understand the math behind the transformation.

Week 4 Sept. 30 - Oct. 6

Robert Goodloe

• (2hr) Oct. 3: Attempted Keras Convolutional neural net training on the constructed datasets. Unsuccessful. It does not seem efficient to train the neural net

on the rpi. Not powerful enough. Furthermore, the Microsoft Bing API scraped images that were mostly from the side angle of pieces while our current design is from the top down.

• (3hr) Oct. 4: Abandoned neural net idea with the Bing API. Attempted to get image difference to work unsuccessfully. This is without piece recognition so it should be simpler. Will try again in the next few days. Will also compile a directory of pictures of chess pieces from the top and will train on these.

Nhut Dang

• (1.5hr) Sep 30: Researching about existing code about line detection and corner detection. The board can be detected easier using the line detection.

• (1hr) Oct. 3: Finalizing the list of material. Brief research on existing projects which uses Corner detection.

• (4 hr) Oct 4: Researching about Harris Corner detection, Hough line transform, Fast corner detection. Reading existing code and try to understand it. Researching the math behind it to figure out which one is better for our project.

Week 5 Oct 7 - Oct. 13

Robert Goodloe

• (1hr) Oct. 11 : Watched tutorials and compiled OpenCV commands to be used at a later date.
• (4hr) Oct. 12 : Spent most of time solving rpi camera module problems. Have compiled a list of solutions for this. Ultimately, looks like the cable is damaged at the connector. Purchased a new one. Worked on image difference and corner detection without success.
• (4hr) Oct. 13 : Tried to use the code at Pyimagesearch to find shapes (squares in this case) in the code. The code did not work well, but I leaned about contours and their features. This will prove to be useful later. I also explored resizing, image smoothing, and thresholding. All of the image operations will make the analysis quicker and easier. In order to find the squares, I have been trying corner detection functions, but it does not seem to be accurate enough, nor does the smoothing get rid of enough noise to make it efficient.

Nhut Dang

• (1hr) Oct. 9 : Having all the material ready. Having the tripod design ready.

• (3hr) Oct. 11 : Researching about the chess engine, chessboard corner detection. Finding the source code for the youtube video. Analyzing the code.

https://pypi.org/project/python-chess/; https://github.com/jayhack/CVChess

• (3hr) Oct. 12 : Testing out the code. Researching on Corner detection. Attempting to adjust the code to fix the problem but no avail. Trying different approach by using a different code. Spending time diagnosing the code. Learning how to use Github on OpenCV.

https://www.meccanismocomplesso.org/en/opencv-python-harris-corner-detection-un-metodo-per-rilevare-i-vertici-in-unimmagine/ https://github.com/jayhack/CVChess https://www.edwardrosten.com/work/fast.html

Week 6 Oct 14 - Oct. 20

Robert Goodloe

• (6hr) Oct. 14 : In order to properly find the contours, the image needs to be properly smoothed and a a mask must be added to the image around the region of interest, this being the chess board. The mask blacks out all unnecessary background in the picture yielding only the chess squares. I was able to implement the mask by resizing, smoothing and converting the image to a gray, colorless version. Then an adaptive thresholding was applied to convert the image to strictly black and white. I then found the largest square contour in the image. Then I converted all pixels outside this largest contour to black using a black image copy. This cleaned up the image nicely.

• (10hr) Oct.15 : Able to find edges using canny edge detection. Found lines using Hough lines transform. lastly, found the corners of the chess board squares by finding the intersections of these lines. The next step is to remove duplicate corners (many were found since the lines intersect across many pixels)and to bound rectangles around the found corners, this will effectively map all 64 squares of the chess board and complete the board recognition objective set for this week.Began implementing some object-oriented programming into the project as it moves from image analysis to the game analysis. Code available at https://github.com/rjgoodloe/ESE205-CVChess under image_Analysis.py

Nhut Dang

• (3 hr) Oct. 13: trying to tell pi to distinguish the chessboard and trying to find the corners of the chessboard. Having some fail experiments.

• (8 hr) Oct 14: After a lot of research, coming up with the suggestion that we should use the edge detection, finding the biggest square blob and then use the Hough lines transform to determine the corner. The reason for using the Hough Line detection is because that detection is better at detecting lines than the Corner detection. The name of the detection should be kind of self explanatory. The reason the Harris Corner detection fail is that the square in the chessboard are not perfectly square. There also exists square which does not have distinguishable corner; some square have no corner. So by using the Hough line, we will try to find the lines in the y-axis and x-axis by using the kernel and then link the y and x axis together to determine the corners of each square. The idea is inspired from the work of another person on sudoku(https://stackoverflow.com/questions/10196198/how-to-remove-convexity-defects-in-a-sudoku-square).

Starting to implementing the color detection, and identify grid. (testing: http://www.aishack.in/tutorials/sudoku-grabber-opencv-detection/; http://nerdclub-uk.blogspot.com/2016/04/opencv-python-and-colour-tracking.html).

Week 7 Oct 20 - Oct. 26

Nhut Dang

• (1.5 hr) Oct. 23: Designing the tripod. Planning to finalize and start printing by next week.
• (1 hr) Oct.25: Testing the chess engine.
• (3 hr) Oct.26: Successfully test out the chess engine. Familiar oneself with all the method and determine what functions are needed to use the chess engine.

Robert Goodloe

• (3 hr) Oct. 23: Refactored code completely. Created new files to reorganize functions and implement more object-oriented programming. Code is now easier to read. Files created: Square.py, Line.py, Main.py. Image_Analysis.py is now renamed board_recognition.py which will serve to initialize board at start of game. Main.py will be used as the main call to run the program.
• (2 hr) Oct.25: could not solve duplicate corner problem.

Week 8 Oct. 27 - Nov. 3

Robert Goodloe

• (4 hr) Oct. 28: After Ethan helped us solve the duplicate corner issue at our weekly meeting, I was able to construct the squares. The corners were sorted by their x values and placed into corresponding columns. Then the four corners of each square were determined. Next, the square objects were created and drawn. The board is effectively mapped. The next step will be piece recognition which will most consist of an image difference algorithm.

• (4 hr) Oct. 30: Calculated the moment (center) of each square and defined a circle of radius 15 as a Region of Interest. In this ROI we will be averaging the color of the pixels. The difference in color between two photos will determine whether a given square is occupied. The program is now able to determine if a square is occupied and the give that square's position. Below is a picture of the chess board with all ROI circles drawn. The second picture shows a piece placed on the board that has been detected and circled. The terminal output also read "Square occupied: e5" which is the correct position for the piece.

Nhut Dang

• (2 hr) Oct. 27: Finish the design for the tripod.
• (2 hr) Oct. 30: Make some adjustment on the design to improve the tripod. Cut the design into smaller component so that the 3D printer can print them. Read the chess engine and continue testing the chess engine. Familiarizing the syntax of the chess engine.

Week 9 Nov. 4 - Nov. 10

Robert Goodloe

• (2 hr) Nov. 6: Set up PiCamera to take pictures and pass image to main.py file. Began to set up board initialization in which starting positions are properly marked black, white, or empty. Looking at color difference algorithm next to determine piece movements.
• (1 hr) Nov. 7: Cleaned up code a bit as I work in new functions for PiCamera and piece Recognition files.
• (3 hr) Nov. 8: Implementing color difference to determine piece movement. There is more difficulty in determining a piece "taking" another piece than a simple movement on one piece to an empty square.

Nhut Dang

• (4 hr) Nov. 7: Creating functions which will synchronize with the chess engine by taking color difference and occupied square. Setting up the touchscreen.

Week 10 Nov. 11 - Nov. 17

Robert Goodloe

• (3 hr) Nov. 11: Successfully tracked piece movement. This included determining which squares changed, updating square state (Black, White, or Empty), and determining the actual move ("e2e4")

250px