How to Use Machine Vision to Calculate Palletized Boxes in a Warehouse
Editor’s note: Serge overviews machine vision techniques used to automate warehouse inventory counting and explains how to ensure they bring accurate results. If you consider powering your warehouse management with a similar solution, have a look at ScienceSoft’s offering for image analysis software development.
According to the recent report by Mordor Intelligence, the global warehouse robotics market was valued at $9.88 billion in 2021 and is projected to reach $23.09 billion by 2027, growing at a CAGR of 15.33%. The solid growth rate is mainly driven by the increasing competition in ecommerce and the need for faster and more accurate order delivery, which forces supply chain and logistics companies to continuously optimize and speed up their warehouse operations. In recent years, many large companies like Amazon and Quiet Logistics have successfully employed robots to operate their warehouses. However, most of the wholesale distributors still rely on manual labor to some extent and only start looking more closely at logistics automation solutions.
The idea to make machines do at least some of warehouse work is tempting, because managing a huge storage is a demanding task, and human workers often fall short of that. Doing dull and monotonous work, such as counting boxes on a pallet, people easily get tired and start making mistakes. Besides that, human operators, unfortunately, are not completely reliable. This results in losses both in money and time, which is something everyone prefers to avoid.
So, what if we decide to introduce automation into this process and count the cartons of each pallet using a camera and image analysis software? When appropriately programmed, machines work quickly and efficiently. They neither get tired nor try cheating. All they need is an instruction to count the number of cartons from the picture mode or video mode, either programmed directly or derived from a machine learning process. And if we want this instruction to be accurate, the images to analyze should be of reasonable quality and contain all the information required.
Possible machine vision solutions
The basic image analysis strategy, in the case of counting boxes, will include several iterations of image segmentation: first, to separate a pallet with boxes stacked on it from the surrounding scene (which may in its turn include other packages irrelevant to this particular pallet), then to separate each box from all others on the pallet and from the pallet itself. To perform this task successfully, the software should find all the edges of cartons correctly and do not mistake for box edges any other features found.
Training a neural network
Today, several quite accurate edge detecting techniques exist, such as Canny edge detector, Sobel operator, Marr-Hildreth algorithm, as well as their modifications and combinations. These and other algorithms can be used as layers of a convolution neural network (CNN). Deep neural networks represent a set of state-of-the-art techniques used for data analysis and classification, and CNNs show particularly great results in image analysis. Neural networks are trained with huge sets of images similar to those they will eventually interpret. This information allows them to build a pattern of features corresponding to the objects they are learning to recognize.
Training a CNN to detect and count pallets being loaded on a truck and calculate boxes on the pallet will require thousands of images to make its predictions as accurate as a human's, and the training set can be continuously extended to increase the output accuracy.
Calculating boxes based on the volume
Another possible approach requires adding one more axis to the image. It assumes that the boxes on the pallet are of the same type and occupy the same amount of space. It also involves 3D reconstruction of the scene and accordingly requires a stereo camera and infrared proximity sensors.
The idea is based on calculating the volume occupied by the entire loaded pallet, then the boxes only (pallet and in-between volume extracted), and finally the volume of a single box. Knowing the volume occupied both by all the boxes and one box, the machine can easily find the number of boxes on a pallet:
Depending on the chosen approach, development of the image analysis software MVP may take 2–4 months. End-to-end solution implementation requires 7+ months.
Financial outcomes of implementing image analysis software
Automated visual inspection for warehouse management helps logistics companies and distributors achieve a range of tangible improvements, including:
- Up to 20% reduction in warehouse operating costs, including labor costs and equipment maintenance costs.
- 50%+ faster counting workflows.
- 99%+ inventory object detection accuracy.
- Increased productivity and safety of employees involved in inventory audit and dispatch procedures.
Challenges that can affect the performance of image analysis software
In the perfect warehouse world where all pallets are stacked according to existing TI-HI standards, image analysis algorithms show their best. Unfortunately, it is not always the case. In real life, image-based, automated counts of products, cartoons, and pallets face several challenges that can affect their performance:
- Inaccurate palletizing: if there is no order in a way boxes are stacked, some of them may be impossible to detect even when through both top and lateral views.
- Illumination conditions: the insufficient amount of light produces noise and makes edge detection more difficult.
- Shadows: they can be misinterpreted as box edges.
- Wrapping film: it reflects light and obscures box shapes.
- Different types of packaging: variations in shape, size, and color of packages should be minimal for software to recognize and interpret them as objects of the same type.
Accuracy is everything
Making machine vision a part of the warehouse management system can bring a significant return on investment in automation. But to maximize the profit, a logistics company should revise the existing palletizing operations in a way that would improve the efficiency of the automated visual inspection software.
The more standardized warehouse operations become, the more of them can be performed by machines. In the future, we can expect the advent of fully automated warehouses, where all the loading, sorting and counting is performed by robots.