To ensure accurate piece recognition, we first calibrated the camera using nav2.org’s camera calib-
ration. A prerequisite for using this is having either linux or windows with WSL, and
ROS2 on your device. Link to tutorial: https://docs.nav2.org/tutorials/docs/camera_calibration.html

As seen in the first image on the right, the calibration button is not highlighted. By taking the calibration grid and
moving it in all direction, including moving it upwards and skewing it, inside the camera field of
view (FOV) the program will be satisfied. The second image on the right illustrates when the program is satisfied and
calibration can commence.

The calibrated data is stored as a YAML file on the system when save is pressed after calibration, which we
imported into the program.

From line 21 shown in the image on the right, we undistort the images captured by the camera to correct
any lens distortion. The cv2.getOptimalNewCameraMatrix calculates a new camera matrix that is
used to undistort images, providing a balance between retaining as much from the original image
and removing unwanted pixels (black edges).cv2.undistort corrects the distortion in an image
by using the camera matrix and distortion coefficients.The combination of these functions ensures that image processing tasks, such as chess piece detection, can be performed accurately.

For more information regarding the cv2 functions: https://docs.opencv.org/4.x/d9/d0c/group__calib3d.html#ga7a6c4e032c97f03ba747966e6ad862b1

To distinguish between different chess pieces, we used HSV filtering.To adjust the HSV ranges
dynamically we created trackbars.

In this code snippet we create the taskbar, setters and getters for the HSV values. We dynamically
change these values to find the best values that filters out the unwanted colors.

The two images shown on the right, shows the orange pieces on the left, and the black pieces the right.
The trackbar below on both of the images shows the parameters that are set too achieve the color
filtering in this setup.

We defined the coordinates of the chessboard grid based on the pixel coordinates and used these
coordinates to draw the grid on the undistorted image as shown in the image on the right.

The next step involved detecting the chess pieces based on the HSV masks. We used contours to
identify the positions and sizes of the pieces.

The code snippet shown in figure 18 first initialize the position of the pieces as an empty dic-
tionary to store the positions of detected pieces. Next the code iterates over two masks, orange
and black masks, which represent the different types of chess pieces based on color. For each
mask, cv2.findContours is used to find the contours in the binary mask image. Contours are the
detected objects boundaries in the mask. The code loops through each detected contour, and with
cv2.contourArea we calculate the area. To filter out noise we set a minimum area of 500. For
contours that are bigger than 500, the smallest rectangle that can enclose contour is calculated
13using cv2.boundinRect. This provides us with (x,y) coordinates and dimensions of the rectangle.
Using this we calculate the center of the bounding rectangle.

Because of the angle of the camera the pieces becomes skewed when moving further away from
the camera, making the pieces inhabit more than one grid. To determine if a chess piece is in the
predefined grid coordinates, we check if the center of the bounding rectangle is within the bounds
of a grid square.We then assign position of the piece and store both the color and rectangles
coordinates to the grid. If more than one piece is in a grid, it checks the piece the piece with the
largest area (assuming that the bigger one is inside the grid) and assign the position to it.

For more information on the cv2 functions, see OpenCV documentation: https://docs.opencv.org/4.x/d3/dc0/group__imgproc__shape.html

Detect Movement