Image Classification on satellite imagery has become an important and pertinent challenge in the domain of machine learning. As more data is sourced through satellite imagery it has become an important task to accurately identify important hotspots and targets within these images so as to classify them for practical use.
With hopes to solve the parking problem for a 258 space parking lot in a Chicago based retail store, we decided to create an app that lets consumers get a real time view of the parking lot and a visualization of the aggregated parking densities over a time series. Just as important are the benefits real estate companies may gain from the aggregate data they may receive from consumers using the app which includes an estimation of lost revenues from full parking lots and real time series data on actual parking lot densities. This could prove to have a lasting and highly beneficial impact on the business.
SIFT Points and OpenCV
The OpenCV package in Python which was specifically developed for Image Classification was extremely useful in helping us identify cars within our images. Specifically we used the SIFT package (Scale Invariant Feature Transform) which was created to identify edges of objects in a given image and create a dataset with the coordinate of each point, as shown below.
SIFT Point Identifiers
By adjusting the radius, edge and contrast threshold we were able to re-engineer SIFT points for vehicle identification under different weather conditions and especially to adjust for the colors of the cars. While grayscale images are generally recommended for Sift point identification, the large number of black cars posed a big challenge as SIFT could barely recognize darker colors and was found to be failing to identify car edges in these cases. On the whole each SIFT classification generated between 1500-2500 SIFT points per image.
Spatial Classification Algorithm Selection
Both the KD-Tree and Ball Tree spatial classification algorithms seemed to be highly pertinent to our analysis since they had the capability to create a pre defined geometric space around a given point in order to count the overall point density. Specifically, both algorithms implemented k-neighbor and bounded neighbor searches for each tree branch at each parking lot space so as to understand overall densities. The algorithms implemented a fast kernel density (KDE) system to identify and count the SIFT points around each parking spot which minimized the processing time per image to almost 0.0003 seconds. The only difference between the two algorithms was that the Ball tree allowed users to set a predefined radius around each car spot for KDE selection, making it slightly more versatile than the KD-tree which creates a 3-D hexagonal object. After we set our threshold on the number of SIFT points that would identify a car from an empty spot we set a 1 or 0 classification for that given spot in order to test it against the actual dataset.
Ball Tree Algorithm Understanding
In the image above, figure(a) shows us the multiple circles the Ball tree creates on each coordinate point so as to count all available SIFT points while figure(b) represents the branching and aggregated count of the points from a spatial perspective.
Model Classification and Overfitting
In order to classify our model we had to manually search and identify the actual parking spots that contained cars in a given image in order to create an actual test set for model comparison. After running a manual search on approximately 30 images we were able to gain a good estimate of the accuracy, precision and recall scores of our model.
In our case, accuracy and recall scores were extremely high at 95.3% and 93.4% respectively. However, precision scores were slightly lower at 72.2%. If we think of the formula for precision TP/(TP+FP) we can conclude that the model led to a higher number of False positives identified compared to False negatives. This translates to the model predicting a higher number of cars in the parking lot than there actually are (approximately 19.7% more on average) in a given image. This is actually beneficial because consumers who are looking for parking will never be told there are spots when they don’t exist in the case of a full parking lot minimizing consumer discontent with the app, however at the same time the real estate company may lose some revenue from cars choosing to park at alternative lots when there are spots available.
Flask App and Final Product
The first part of the app gives a big picture view of the aggregated parking densities by day and hour. The visualization changes for each week in a month by clicking the respective buttons.
While the aggregated view is great for customers to understand average densities when they do plan to park at the parking lot, they may learn more from a real time image sent to the app with actual empty and full parking spots highlighted, as well the overall percentage of occupied spots at that time.