Designing and implementing a machine vision system as a student project

An interesting student project resulting in a billiard ball handling robot cell was carried out under the instruction of Timo Suvela, lecturer in automation technology, at Satakunta Univeristy of Applied Sciences. The cell will be on show at the premises of Satakunta University of Applied Sciences Faculty of Technology and Maritime Management Pori as a part of a larger technology exhibition. The project was implemented by students of degree programmes in automation technology, electrical engineering, mechanical and manufacturing engineering, and information technology.

THESIS AS INSPIRATION

The idea of a billiard ball handling cell was first introduced in a thesis which looked into possible objects at a technology exhibition. It was decided that the main exhibit would be a robot-controlled game of billiards which consists of a billiard table and a robot cell. The essential controllers of the robot cell are a programmable logic controller, PLC, a user interface, a smart camera, and a robot arm. Billiards is played in a normal fashion, but the sunken balls are transferred through a pipe into a robot cell in which the balls are gathered into a triangle.

The conveyor inside the cell lifts the balls from the floor level to the bumper where the robot can reach them. The PLC coordinates the cell activities based on the information received from the sensors on the devices. It drives the conveyor to run when the ball enters the cell. The PLC drives the camera to take a picture of the ball when the robot has brought the ball to the camera. The robot arm in the cell is also slave to the PLC. Task rotations, opportunely activated by PLC, have been programmed in the robot.

The camera identifies the number of the ball based on its colour. The camera returns the ball to the PLC, which first finds the next free spot on the bumper, then activates the task “take ball to bumper spot x”, and moves the ball’s information to that spot once the robot has left the ball at the bumper. The same is repeated with the rest of the balls excluding the white one, which is immediately returned from the cell by the robot. After the game the PLC drives the robot to move the balls from the bumper into the triangle and arrange them according to their numbers. The ball triangle is in a box which a player pulls out of the cell when all the balls are in the triangle.

The essential devices and interfaces of the robot cell.

ROBOT CELL IS AN EXTREMELY GOOD PRACTICAL EXAMPLE

The project has taken over 3,000 working hours of which the most come from separate student projects and about a third from work done during study periods. Also one thesis covering the project has been written. The electrical engineering students completed the electrical design of the cell and the manufactured the control box as a student project. The design and manufacturing documents of the cell, the conveyor, the ball feeder, the grabber, and the bumpers are the result of the mechanical and manufacturing engineering students’ project. They were also responsible for organising the project. The students were also largely responsible for the manufacture and the assembly of the devices. Other cell devices, such as the box in which the ball triangle is put, the programmes for the PLC and the robot, and the user interface pages, were produced during study periods. A memory and a speed game were designed for the cell’s operating panel in a thesis. The work exploited the web server property of the panel, enabling browser-based play on any computer in Satakunta University of Applied Sciences network.

Engineering students Hannu and Ilkka Luoma finish the robot installation to its final position.

IN CO-OPERATION WITH THE ENTERPRISES OF THE FIELD

The camera application was designed by the students with the help of Satakunta University of Applied Sciences researchers and device deliverers. Identifying the colour of the billiard ball turned out to be more difficult than expected. After the right identifying tool was found it was easy to make the application for the smart camera. At first it was difficult to create the right lighting. After many experiments the ring-shaped led light with sequences of blue, green and red leds was selected for the purpose. The leds of the led light caused reflections on the ball, which were clearly discernible in the camera’s image. This problem was solved by blocking outside light from the imaging spot, which enabled the lowering of the led-light’s capacity.

Initially only bright light was used in colour identification. The similarity of certain colours seen through the camera turned out to be a problem. The camera found almost equal amounts of blue and green colour on the blue ball. The same problem recurred with the yellow and orange balls as well as some other colours. This problem was solved by taking a new picture of the balls whose colour the camera could not identify accurately. The new image must be captured in lighting that best fits the case. For example, if the camera identifies the ball as blue and green, a new image is captured in blue lighting, which enables the camera to easily separate between blue and green, Suvela explains. The Finnish Society of Automation has also granted funds for the project.

>> Machine vision solutions for various wavelength areas

Text and photos: Jari Vihervirta