AI for Matti
By Ward Dehairs, May 1, 2020
Corona lockdown was a chance for me to undertake a few tasks that where being kicked down the road for a long time. One of which was working out an algorithm for detecting footprints on the sensor input of the Matti.
This would prove to be quite a challenge right off the bat, since input from the Matti isn't spectalurly reliable.
Heatmap disgram showing dead rows/columns of sensors, as wel as random smidges of noise
The first step then would be to process the input generally, rescaling sensor values and filtering out noise. If necessary bridging over dead sensors to get a more complete picture.
With better input a basic clustering algorithm would be possible. The key insight here is to treat the sensor inputs as a sort of weighted 2D datapoints. This opens the door to existing clustering algorithms, while having a grid built in, making a neighbour distance based algorithm ideal.
This weighted statistics approach is a trick used a lot in AI, which seems only appropriate for this sort of task. It also provides a means of detecting the "direction" of a cluster, since the data points can be taken together as belonging to a normal distribution, yielding a covariance matrix, which in term will have primary and secondary eigenvector, pointing in the long and short direction of an ellipsoidal cluster.
With the clusters in place, the only remaining job is to classify the clusters as a left and/or right foot print and others. This required several iterations, though I ended up with a fairly reliable approach. Selecting the right clusters is the easy part, since weight, aspect ratio and size of the clusters are enough to distinghuish footprints from most other signals, while noise has already been dealt with earlier.
They key strategy though, was to compare a given footprint to a reference print. In practice, this means recording many footprints, and using them to generate averaged out and optimized discriminant images.
Discriminant images are optimized to show the difference between a forward/backward facing foot or a right/left foot, displaying the most differing areas of pressure with white/black respectively.
These images are used to calculate responses and scores for every possible orientation, based on how well it overlaps with the discriminant images. To finish up the algorithm, a few score penalties are added depending on the situation;
- Medium score bonus for retaining the same arrangement, while the same clusters persist
- A massive score penaly for having 2 left or 2 right feet
- A large score penalty for having feet pointing in opposite directions
- A medium score penaly for having feet cross (left foot right, right foot left)
- A small score penalty for having feet point backwards
The result is as reasonable as could be expected, the points where the algorithm struggles are the same points where a human would typically struggle to distinghuish what they are seeing. Further improvements would have to come from the hardware end.