Deep learning models are utilized for various research tasks, one of which is image classification. Our research in AI-Assisted Pattern Discovery in Car Parts Data has leveraged this technology for recognizing patterns. Given the sales of stolen car parts on online marketplaces is a significant issue in today’s society, we wished to investigate deeper and identify any patterns of interest.
Dataset
The dataset used comprised of several TAR (Tape Archive) files, each consisting of thousands of images from various online Craigslist posts advertising car parts for sale. Utilizing various Python scripts, each image was parsed with significant features being noted for use in classification with the deep learning model.
Model
ViT Base, the pre-trained model that was used, proved useful for this classification task. Employing a vision transformer architecture, this model was designed specifically for processing image data. Each image is handed in by patches of 16×16 pixels, and then subsequently processed by the model. This model was pre-trained on a vast dataset, ImageNet-21k, which contains over 14 million images.
Process
Utilizing the dataset and deep learning model, this facilitated the requirement for embeddings. Embeddings are distinct features of image data that are converted into a vector, allowing the model to process information. Each image had embeddings extracted and were subsequently compiled in a CSV (Comma-Separated Values) file to be passed in for UMAP (Uniform Manifold Approximation and Projection) visualization. By the end, nearly 85,000 images/embeddings were obtained.
Upon extraction of these embeddings, the K Means Algorithm was utilized in order to generate 20 clusters for the given dataset. Through this, the script produced 3 different scores: Silhouette, Calinski-Harabasz Index, and Davies-Bouldin Index. Silhouette measures how similar an object is to its own cluster compared to all other clusters, Calinski-Harabasz Index measures the variance within clusters compared to the variance between clusters, and Davies-Bouldin Index measures the average similarity ratio of each cluster with the cluster that is most similar to it. Using these scores, it was now possible to interpret an optimal low dimensional space for the clustering.
Among the dimensions that were tested—16, 32, 64, and 128—64 dimensions were the best suited. Furthermore, 128 dimensions proved to be optimal while 16 dimensions and 32 dimensions fell short of expectations. However, it was through these tests that the silhouette score revealed itself to be quite small, sitting at a value of 0.015150264836847782. UMAP visualization for 64 dimensions also revealed several outliers to corroborate this fact. With 64 dimensions, K Nearest Neighbors was run to obtain all the cluster centroids and cluster assignments for all points in the dataset. The top 10 closest posts to each cluster centroid were compiled into a document that displayed all the images along with their corresponding post ID.
Findings
Upon completion, 20 different clusters were evaluated each representing a different set of features in the images. As a result, we had acquired sets of posts that have commonalities, allowing for interpretation of the dataset. For reference, Cluster 0 displayed images that were primarily of exteriors of various vehicles, Cluster 1 displayed images of parts that attach to the exterior, Cluster 2 displayed images of powertrains (engines, transmissions, differentials), Cluster 3 displayed individual body panels (doors, trunks, hoods), and Cluster 4 displayed images of accessories that are used to pull trailers and carry bikes.
As stated previously, the silhouette score for the dataset proved to be notably small, which was supported by the fact that the visualization contained numerous outliers in the data. This might have been attributed to the fact that the clusters lacked clear definition. Essentially, this means that several of these posts contained images that did not display many distinguishable features. This makes sense considering that although the clusters emphasized focus on a specific set of car parts, several still displayed various other vehicle components. For instance, although Cluster 2 primarily featured images of powertrains, the posts in this cluster also exhibited efforts to showcase the exterior and body panels of the vehicle. This is logical as posters would likely aim to showcase multiple facets of the vehicle when selling it, explaining the lack of focus on specific car parts.