find_quadrangle improvements

[simonMoisselin]

Hi,

Great job !
Everything works fine for my dataset.
But I am wondering if you have some ideas on how to improve find_quadrangle function.
Sometimes I have really good board mask but this one fails ( no contour found).

Simon

[gudbrandtandberg]

Nice! Thanks!

Yes it's a bit silly isn't it to have a failure-prone computer vision algorithm stuck in the middle of two neural nets! I just kinda landed on it since it works most of the time. I have two ideas for how to improve this.

1. Make find_quadrangle less faulty. This can be done by not giving up so early, insist on finding a suitable quad before returning. That means if filter_contours fails, try again with different parameters until it works, or if cv2's poly-approximate returns a triangle or a pentagon just try again with a slightly different epsilon parameter. It's a bit ugly but this is a relatively quick and easy way of fixing it.

2. Better idea (perhaps) is to make the board_extractor NN return coordinates instead of a bitmask. We could probably use many of the layers/weights we already have and instead have a (1, 8) output of 4 x,y coordinates. The downside to this is that it will always find a quadrangle, even when there are none. I guess this can be caught quite easily in a filter function, but something to be aware of..

To repeat that point: it would be nice if the program not only finds chessboards when they are there but also returns "no chessboard" when there is no chessboard there.

Hope this helps and that you are motivated to help out! I would really like to keep improving ChessVision, but I just got a job so time flies! But feel free to PR or share if you get some nice results!

And if you have some more data for the project I will gladly integrate it in the quilt repo, more data will almost certainly give better results!

Gudbrand

[gudbrandtandberg]

Oh and by the way, there is already an abandoned start with a coordinate-finding NN there. There's a notebook and a tran_board_extractor_coord or something. I didn't quite get it to work, something wrong with my architecture I guess. Also there is a script for generating 1x8 numpy labels for the images called make_labels.py.

[simonMoisselin]

My guess is that training a neural network from original image for finding coordinates leads to overfitting. I think this is your case:
Here, you have small dataset.
By predicting mask, you have a good ratio of number of prediction / images (each pixel is a prediction), you can work with small dataset. With coordinate, you have 8 predictions only, you need more images.

But I think create a third neural network for finding coordinate is still the way to go. I see 2 paths :

• Augment the size of the dataset.
• Maybe using the predicted mask as input . It will be more difficult to overfit using the mask image than the original image.

I am working on this project I'll send you some news !

Simon