My nephew against machine learning / Geektimes

My four-year-old nephew Yali was deeply attracted to Pokemon. He has many of these toys and several cards from the collectible card game (TGG). Yesterday he found I had a large collection of TGG cards, and now he has so many cards that he just can not cope with them.

The problem is that Yali is too small to figure out how to play the game, and He invented his version of the game. The purpose of the game is to disassemble the cards by categories (pokemon, energy and training map).

He did not ask how I know what type of card. He just took a few cards and asked what type they were. After receiving several answers, he managed to divide several cards by type (having made several mistakes). At that moment I realized that my nephew is, in fact, an algorithm for machine learning, and my job as an uncle is to label the data for him. Since I’m an uncle-geek and an enthusiast of machine learning, I started writing a program that can compete with Yali.

This is how a typical Pokemon card looks:

]
For an adult who can read, it’s easy to understand what kind of card this is-it’s written on it. But Yali is 4 years old, and he does not know how to read. A simple OCR module would quickly solve my problem, but I did not want to make any extra assumptions. I just took this card and gave it to study the MLP-neural network. Thanks to the site pkmncards, I was able to download pictures already sorted by category, so there was no such problem.

Machine learning algorithm needs features, and my features were pixels of images. I converted 3 RGB colors into one whole number. Since I came across pictures of different sizes, they needed to be normalized. After finding the maximum height and width of the pictures, I added zeros to smaller pictures.

Quick QA run before the program itself. I randomly took two cards of each type and started the prediction. When using 100 cards from each category, the fit was very fast, and the predictions were terrible. Then I took 500 cards from each category (excluding energy types, which were only 130), and ran the fit.

The memory is over. It was possible to run the fit code in the cloud, but I wanted to come up with a way to save memory. The largest picture was 800×635, it was too much, and changing the size of the pictures solved my problem.

For a real check, in addition to the usual maps, I added maps on which I gave a little calyx, cards cut in half with Painted over the contours, photographed them with a phone (with a bad camera), etc. For training, these cards were not used.

I used 1533 models. Different sizes of pictures, several hidden layers (up to 3), layer length (up to 100), image colors, image reading methods (whole, top, every second pixel, etc.). After many hours of fitting, the best result was 2 errors of 25 cards (few of which models had such a result, and each of them was mistaken on different cards). 9 models from 1533 worked with the result in 2 errors.

A combination of models gave me the result with 1 error if I raised the threshold above 44%. For the test I used a threshold of 50%. I waited a month while Yali played with the cards, and tested.

Errors occur when recognizing energy cards. The pkmncards of such cards were only 130, in contrast to thousands of other types of cards. Less examples for training.

I asked Yali about how he recognized the cards, and he told me that he saw some Pokémon in TV shows or books. This is how he recognized Raichu’s ears or learned that Vaporone is the watery Ivi. My program did not have such data, only maps.

After defeating Yali in the game with the Pokemon and receiving the award, our machine intelligence goes to meet new adventures.