Journal of Multimedia Information System
Korea Multimedia Society
Section A

Design, Development and Testing of the Modular Unmanned Surface Vehicle Platform for Marine Waste Detection

Josip Vasilj1, Ivo Stančić1,*, Tamara Grujić1, Josip Musić1
1Faculty of Electrical Engineering, Mechanical Engineering and Naval Architecture, University of Split, Split, Croatia, Josip Vasilj: jvasil00@fesb.hr; Tamara Grujic: tamara.grujic@fesb.hr; Josip Musić: josip.music@fesb.hr
*Corresponding Author: Ivo Stančić, Ruđera Boškovića 32, 21 000 Split, Croatia, ++385-21305879, ivo.stancic@fesb.hr

© Copyright 2017 Korea Multimedia Society. This is an Open-Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/4.0/) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.

Received: Dec 14, 2017 ; Revised: Dec 18, 2017 ; Accepted: Dec 19, 2017

Published Online: Dec 31, 2017

Abstract

Mobile robots are used for years as a valuable research and educational tool in form of available open-platform designs and Do-It-Yourself kits. Rapid development and costs reduction of Unmanned Air Vehicles (UAV) and ground based mobile robots in recent years allowed researchers to utilize them as an affordable research platform. Despite of recent developments in the area of ground and airborne robotics, only few examples of Unmanned Surface Vehicle (USV) platforms targeted for research purposes can be found. Aim of this paper is to present the development of open-design USV drone with integrated multi-level control hardware architecture. Proposed catamaran - type water surface drone enables direct control over wireless radio link, separate development of algorithms for optimal propulsion control, navigation and communication with the ground-based control station. Whole design is highly modular, where each component can be replaced or modified according to desired task, payload or environmental conditions. Developed USV is planned to be utilized as a part of the system for detection and identification of marine and lake waste. Cameras mounted to the USV would record sea or lake surfaces, and recorded video sequences and images would be processed by state-of-the-art computer vision and machine learning algorithms in order to identify and classify marine and lake waste.

Keywords: control system; mobile robotics; marine waste detection; unmanned surface vehicle

I. INTRODUCTION

The pollution of the sea, lakes and other water surfaces is a global ecological problem of modern society. Approximately 70% of the planet is covered with seas and lakes. Marine and lake wastes (especially plastics) can be found almost everywhere and endanger not only the health of our seas, lakes and coasts but also the economy as a whole. The question is how can we prevent the waste from ending in our seas and lakes? The prevention is almost impossible, since the sources of waste are numerous. As example, Jambeck et al. [1] calculated that 275 million metric tons (MT) of plastic waste was generated in 192 coastal countries in 2010, with 4.8 to 12.7 million MT entering the world’s seas and oceans. Plastic packaging waste such as drinking bottles and disposable bags, is by far the most common type of waste in the marine and lake environment.

Presence of microplastics in aquatic environments is of particular importance as these pollutants represent an increasing threat to marine organisms and ecosystems [2, 3]. Microplastics are small plastic particles in the environment classified as less than 5 mm in diameter. Some researchers reported that microplastic has become ubiquitous in aquatic environment with diverse concertation due to industry, traffic and natural movement of sea [2, 4]. Most of microplastics originate as secondary microplastics or fragments derived from the breakdown of larger plastic debris, both at sea and on land. Culmination of physical, biological and chemical processes can reduce the structural integrity of debris, resulting in its fragmentation into smaller and hard to detect particles. Because plastics do not disintegrate for many years, they can be ingested and incorporated into bodies of many organisms, and finally find its way into human body, if that organism is part of a human food chain.

Detecting and removing larger plastic objects in water before it breaks into smaller parts may significantly reduce its concentration in long term. Scientist have investigated concentration of larger plastic particles (>2.5 cm) in several locations collected with trawls, where concentration varied between 5 and 17 items / km2 [5]. Literature review showed that advanced sensors and methods are mostly used for detecting accumulation of litter, their convergence zones and migration due to sea currents, in order to be manually collected [6]. Similar systems that can individually detect and track single plastic particle in large open water surfaces are nonexistent, despite most of needed procedures based on computer vision and machine learning are highly developed and utilized on daily basis for similar ground-based tasks. Some automated systems that detect and tracks underwater and surface objects based on computer vision are developed [7, 8], where similar approach may be utilized in future litter detection systems.

Regarding mobile robots, in recent years we are witnessing improvements in research and introduction of new applications for unmanned aerial vehicles, especially considering today ubiquitous multicomputer drones. As a result of reduction in size and advancement in motor and battery technologies, drones are becoming even more popular, and can be found in various areas such as traffic and environment surveillance, structure inspection, aerial mapping and as a novelty toy. Development of various multirotor drones like the one produced by DJI [9] and Parrot [10] enables enthusiasts and researchers in academic community to modify and implement new control algorithms. There have been number of attempts to improve functionalities of those drones [11, 12], by modifying their control algorithms. There are also alternative approaches where researchers completely overwrite existing (stock) firmware with costume one in order to achieve newly desired performance [13, 14]. Similar observation can be done while considering ground based mobile robots, which are also employed for various tasks, including military applications, structure inspection, and as service robots.

Considering the development of unmanned surface vehicles, their application is mainly aimed toward military and security task [15] or hydrographic research [16]. Only few examples of USV can be found that aim primarily on academic community, and which are meant to be utilized as an education tool [17] or platform for seaborne experiments [18]. Some researchers developed algorithms for obstacle detection embedded on both slow and high speed unmanned surface vehicles [19-22]. Main intention was obstacle avoidance rather than identification and removal of potential marine waste, which is our final goal.

Contemporary mobile robots usually employ single microcontroller or onboard computer to control all robot tasks, which include communication, navigation and control of the actuators. Innovation presented in education robots like ArduinoRobot [23], are representative of a simple differential drive education robot which has two microcontrollers, one on each of its two boards. The Motor Board controls the motors, and the Control Board reads sensors and decides how to operate. This separation of motor control from other tasks enables users to modify and develop only one robot functionality at a time, while other core functionalities are intact. This in turn enables easier development of pathfinding algorithms or implementation of alternative communication interfaces, while control of robot drive and readings from ground-based sensors is completely independent of modifications performed on robot’s control level, and unaffected by their computing requirements.

Similar approach is applied in our prototype of USV, presented in the paper, where control over robot’s functionalities is separated in three control levels, all implemented on physically separate modules (microcontrollers) connected together by a shared data link. Proposed USV is designed in the form of a catamaran vehicle, with two propulsion units. Each propulsion unit is controlled by its own microcontroller, which is in charge of optimal and secure operation. Propulsion module is controlled by middle-level navigation level, which is responsible for navigation planning and forwarding commands to the drive control microcontrollers. At top, command level is responsible for communication with the ground base station and forwarding commands to the navigation level. This design also enables improved performance of manual radio controllers by implementing drive-by-wire algorithms, where pilot is no longer responsible for adapting UAV to current environment situation (e.g. large waves, strong currents, wind) which is now of loaded to the navigation level.

We believe that the developing of a system for detection and reliable identification of marine and lake waste could be possibly significant contribution to the preservation of water resources and, consequently, to the development of ecology in general. Such system would include an autonomous vessel, equipped with sensors (cameras) for underwater and water-surface recording, and detection of waste using state-of-the-art computer vision and machine learning algorithms. The system would implement important multimedia elements such as processing and analysis of digital images and video sequences. With the introduction of capable and affordable communication devices, enlarged capacity of batteries and advancements in brushless motors technology, development of small and affordable USV has become possible. In this paper we present the design, development and testing of our first prototype of the catamaran-type autonomous vessel intended to be a part of the planned system for detection and identification of marine and lake waste.

The rest of the article is structured as follows: in Section II used materials and methods are presented. In Section III three-level design of control and communication between USV modules is described in detail. Section IV describes testing of the developed catamaran prototype. Finally, conclusion and plans for future work are presented in Section V.

II. MATERIALS AND METHODS

Basic design of the proposed vessel resembles a catamaran boat with two interconnected hulls. Conceptual USV design with labeled main components is depicted in Figure 1. This design enables USV to be utilized as a stable platform for future research applications (intended waste detection) which require stabilization of the payload. Each 70 cm long hull contains its own drive module. As vessel is driven by two independent drive modules, such design offers more possibilities for implementation of various control algorithms in order to achieve desired vehicle behavior, especially in challenging sea conditions. Proposed USV must be capable for autonomous operation, which is enabled by integration of navigation sensors, and direct control over a radio link. Powerful motors and small mass (~3 kg) would enable USV to achieve speeds up to 20 km/h (11 knots) and to cope with high seas with autonomy over 30 minutes and estimated 10 km range. Innovation presented in this paper, as mentioned in Section I, is splitting of main vessel task on separate hardware microcontrollers and modules, as described in more detail in the following Section(s).

jmis-4-4-195-g1
Fig. 1. Conceptual 3D design of the proposed catamaran-type USV with labeled main modules.
Download Original Figure
2.1. Control and Communication Module

Main module is a Control and Communication module (C&C), which holds Silicon Labs Si4463 radio communication modem, EM506 GPS module and Siemens BMP 280 chip for reading the basic weather condition parameters. This module is responsible for receiving commands from the ground station, continuously parsing location data from GPS module (with 1 Hz refresh rate) and collecting of status data from other modules, if ground station requests status report or self-diagnostics. Control and communication module algorithms are completely implemented on Arduino Mini Pro microcontroller board with Atmega 328P MCU running at 8 MHz (3.3 V version compatible with most radio modems). C&C module can be seen in Figure 2.

jmis-4-4-195-g2
Fig. 2. Top view of the C&C and navigation modules, placed into common housing.
Download Original Figure
2.2. Navigation Module

Navigation module is responsible for driving of the USV to a desired location, as commanded by C&C module. It holds advanced Ardufruit BN0055 IMU sensor which continuously feeds integrated MiniPro microcontroller board (5V version running at 16 MHz) with the orientation data. Navigation module is continuously calculating desired speed and direction of rudder for both motor drive modules. As all components are modular and replaceable, future development in navigation level (module) could be done by replacing MiniPro microcontroller with by a more powerful on-board computer like Raspberry Pi that runs ROS (Robot Operating System) [24]. More processing power would enable advanced path planning and advanced real-time control of motor drives in order to overcome heavy seas and achieve system’s maximum performance.

Control and communication module and navigation module are placed into common housing, as presented in Figure 2.

2.3. Drive Module

The drive module, whose main components are depicted in Figure 3 a), is responsible for moving the USV on a water surface. Each of the two drive modules consists of a single water cooled B2040 3000 KV brushless motor controlled by water cooled reversible ESC (Electronic Speed Controller). Motor is attached over a flexible shaft to a 3 fin 36 mm propeller. Propulsion is directed over a three connected rudders by a small 9 g servo motor, as shown in Figure 3 b).

jmis-4-4-195-g3
Fig. 3. a) 3D model of a Drive Module, b) Back view of propulsion mechanism with rudder and propeller, c) Drive Module top view with components.
Download Original Figure

Propeller is enclosed in a plastic tube housing, which ensures safe operation and protects propeller from seizing nearby objects or inflicting injury. Whole system is water cooled with (sea) water pumped using a single 4.5 W water pump, where cooling liquid is first directed over the ESC controller and then over the brushless motor. Exiting temperature of water is measured by analog LM35 sensor, which ensures that both ESC and motor are inside safe operating temperature ranges. All electronic parts are controlled by a single Arduino MiniPro microcontroller board, running at 16 MHz. Drive module is placed in two compartments, one holding the motor, ESC and water pump (wet compartment) and second one that holds 11.2 V Lithium-Polymer 3 Ah battery, microcontroller and servo system (dry compartment).

2.4. Construction of the Catamaran Prototype

Drive modules are inserted into the hulls made from Styrofoam, which can also hold additional payload, according to current mission. C&C and navigation modules are placed on a small extruded tower at front of the USV, as shown in Figure 1.

All mechanical parts are designed by Google SketchUp software and 3D printed using a PLA filament and Ultimaker 2 3D printer. Figure 4. presents example of 3D designs of printed mechanical parts. Figure 5. presents the final look of the constructed catamaran. In Figure 5 a) top view of the catamaran with C&C and navigation modules placed in the same housing, and drive modules inserted into the hulls is shown. In front of the vessel camera for sea surface recording is mounted, Figure 5 a). Bottom view of the vessel with propulsion mechanisms is shown in Figure 5 b).

jmis-4-4-195-g4
Fig. 4. Example of 3D designs of printed mechanical parts of the catamaran.
Download Original Figure
jmis-4-4-195-g5
Fig. 5. The final look of the catamaran prototype: a) Top view b) Bottom view.
Download Original Figure

III. THREE – LEVEL DESIGN OF CONTROL AND COMMUNICATION BETWEEN USV MODULES

As mentioned in Section I, the main goal of the paper is to develop an open-architecture modular USV which could be used for research purposes such as, in our case, detection and identification of marine and lake waste. Researchers and users can work with each individual module, without compromising functionalities of other modules. Three – level design of control and communication between USV modules (C&C, navigation and drive) is depicted in Figure 6. This approach ensures that core functionalities, such as smooth operation of the propulsion system and communication with ground station are not affected by implementation of various navigation and path planning algorithms.

jmis-4-4-195-g6
Fig. 6. Three – level design of control and communication between USV modules, with connected sensors, actuators and communication devices.
Download Original Figure

C&C module controller is continuously parsing serial data received from GPS module in NMEA (National Marine Electronics Association) format, holding the most recent readings in memory. At the same time it is listening for any data available on radio channel, in case ground station issues new command or requests sensor readings. If new command is received, in form of a new waypoint, or direct command from operator’s joystick, it redirects data to a navigation module over an I2C line, following the I2C package format described in Table 1. Also, C&C module is sending in 1 Hz intervals the most recent location data to the navigation module. Possible additional sensors and payloads would be connected directly to the C&C module.

Table 1. Description of I2C communication between C&C module and navigation module.
Type I2C Data Bytes
1 2 3 4 5 6 7
Direct command Direct commands(7 buttons)[0-127] USV Speed USV Heading Camera Pitch Camera Yaw Camera Zoom 0
Waypoint Geo Position(N,S,E,W)[128-131] NMEA Latitude NMEA Latitude NMEA Latitude NMEA Latitude NMEA Latitude NMEA Longitude
Location Geo Position(N,S,E,W)[132-135] NMEA Latitude NMEA Latitude NMEA Latitude NMEA Latitude NMEA Latitude NMEA Longitude
Download Excel Table

Navigation module receives commands from C&C module (Waypoint or Direct command over an operators joystick), and the most recent USV location. With additional data received from integrated IMU, navigation module decides how to operate motor drives. This level is the most open for future development, as some new/improved control algorithms could be implemented. At this moment only direct control over USV is completely implemented, while waypoint navigation or autonomous trajectory planning is under development.

Navigation microcontroller is constantly recalculating new motor drive parameters and redirecting them to the drive modules using an I2C link. Commands are consisted of 3 bytes containing motor speed, direction and rudder angle, while drive module is returning its status, as described in Table 2. Control unit of the drive level is designed “as it is”, without plans for future improvements, as it controls cooling of ESC and brushless motor, and in case of overheating turns of the motor drive while still pumping cold water. In case of any malfunction it sends error status to navigation and C&C modules. When control command or navigation level malfunctions, USV securely stops until self-diagnosis is performed, and the whole system reboots by WatchDog Timer (WDT). Therefore, in case of faulty algorithm is loaded, USV would not be lost due to exiting communication range by continuing to execute the last received valid command. Furthermore, it wouldn’t be damaged due to overheating of drive components.

Table 2. Description of I2C communication between navigation module and drive module.
COMMAND I2C Data Bytes
1 2 3
Receiving Direction Motor Thrust Rudder Angle
Return Battery Voltage Drive Status Code Temperature
Download Excel Table

IV. TESTING THE PROTOTYPE OF UNMANNED CATAMARAN

Upon the construction of the unmanned vessel prototype, it was tested in the laboratory conditions, the freshwater pool, and at the open sea, Figure 7.

jmis-4-4-195-g7
Fig. 7. Open sea testing of the USV prototype with the camera for recording objects in catamaran surroundings: a) Front view of the vessel, b) Side view of the vessel, c) Trajectory of the vessel during one of the performed tests
Download Original Figure

Laboratory tests included measurements of the quality of communication between the control unit and the unmanned vessel, and the interruption of the radio link. Measurement results of the communication between the control unit and the vessel, i.e. the response of the vessel to the steering commands were more than satisfactory, since the response time values were measured in milliseconds. During the interruption of the radio link, the vessel’s engines were turned off, and the rudder is set to zero (neutral) position, as was programmed.

Maneuvering capabilities and stability of the vessel, cooling system, and the navigation speed were tested in the freshwater pool. The results of maneuvering capabilities and stability were very good since the catamaran was very stable and could rotate almost around its axis. The cooling system also worked as intended. The vessel’s autonomy was about 14 minutes, while the engine is running steadily. However, the navigation speed was on average 5 knots, while 180° turn could be achieved in less than 1 second.

During tests at the open sea, camera was mounted at the top of the vessel for recording objects in catamaran surroundings, Figures 7 a), b), and Figure 8. The range of radio link, stability, and behavior of the boat at small sea waves were tested. Tests showed that the radio link range in the open sea is about 2000 m. The stability and rotation of the boat are the same as in the pool (i.e. satisfactory), while the navigation speed is reduced compared to the speed in the freshwater pool conditions.

jmis-4-4-195-g8
Fig. 8. Sea surface as recorded by the camera mounted at the top of the vessel.
Download Original Figure

V. CONCLUSION AND FUTURE WORK

In the paper the design, development and testing of modular catamaran - type USV is presented. The realized USV is planned to be used as a part of a system for detection and reliable identification of marine and lake waste by processing video sequences and digital images recorded by underwater and above sea surface cameras mounted to the vessel.

Key innovation of the proposed USV prototype is multi-level modular control design, which enables the implementation of new algorithms on desired control level/module, while other functionalities remain intact. This enables development and implementation of various navigation algorithms and adaptation of USV for various missions or weather/sea conditions. As a most important part, drive module is devised as a separate modular part that controls USV movements, performs self-diagnosis and prevents failure.

Developed prototype of USV is still under trials at the Faculty of Electrical Engineering, Mechanical Engineering and Naval Architecture, University of Split, Croatia. As a future improvement of the proposed and developed USV prototype, navigation level controller will be replaced with more capable based on - board computer running ROS framework, which would enable optimal path planning when USV is set to sweep over desired sea area. In order to increase the navigation speed which is at the moment insufficient, incorporation of more powerful engines, and improvement of hydrodynamic properties of the vessel by applying naval engineering techniques are planned. Also, additional trials are planned in autonomous mode and in open waters, in order to receive seaworthy certificate and assure vehicles safety for all participants in marine traffic.

At the moment, the presented USV prototype has only one, above sea surface camera, so we plan to mount underwater camera to obtain more valuable data for waste detection and identification. Also, implementation of gyro-stabilized camera is planned, where operator could receive direct visual feedback and manually direct USV. Finally, algorithms based on deep neural networks will be used for detection and classification of waste from video streams provided by on-board underwater and above sea surface cameras.

REFERENCES

[1].

J. R. Jambeck, R. Geyer, C. Wilcox, T. R. Siegler, M. Perryman, A. Andrady, R. Narayan and K. L. Law, “Plastic waste inputs from land into the ocean,’’ Science, vol. 347, no. 6223, pp. 768-771, Feb. 2015.

[2].

W. Zhang, S. Zhang, J. Wang, Y. Wang, J. Mu, P. Wang, X. Lin and D. Ma, “Microplastic pollution in the surface waters of the Bohai Sea, China,” Environmental Pollution, vol. 231, pp. 541-548, Dec. 2017.

[3].

C. G. Avio, S. Gorbi and F. Regoli, “Plastics and microplastics in the oceans: From emerging pollutants to emerged threat,” Marine Environmental Research, vol. 128, pp. 2-11, Jul. 2017.

[4].

J. Q. Jiang, “Occurrence of microplastics and its pollution in the environment: A review,” Sustainable Production and Consumption, In Press, Nov. 2017.

[5].

U. Kammann, M. O. Aust, H. Bahl and T. Lang, “Marine litter at the seafloor – Abundance and composition in the North Sea and the Baltic Sea,” Marine Pollution Bulletin, In Press, Oct. 2017.

[6].

T. H. Mace, “At-sea detection of marine debris: overview of technologies, processes, issues, and options,” Marine Pollution Bulletin, vol. 65, no. 1-3, pp. 23-27, 2012.

[7].

D. Walther, D. Edgington and C. Koch, “Detection and tracking of objects in underwater video,” in Proceedings of the 2004 IEEE Conference on Computer Vision and Pattern Recognition, Washington, June 2004.

[8].

D. L. Rizzini, F. Kallasi, F. Oleari and S. Caselli, “Investigation of vision-based underwater object detection with multiple datasets,” International Journal of Advanced Robotic Systems, vol. 12, no. 6, pp. 1-13, Jun. 2015.

[9].

DJI multirotor drones, http://www.dji.com/matrice-200-series, December, 2017.

[10].

Parrot multirotor drones, https://www.parrot.com/fr/drones/parrot-ardrone-20-elite-edition, December, December, 2017.

[11].

M. Ericsson, “Visual tracking and control of a quadrocopter,” M.S. thesis, Lund University, Lund, Sweden, 2015.

[12].

J. Pestana, J.L. Sanchez-Lopez, I. Mellado-Bataller, C. Fu and P. Campoy, “AR Drone identification and navigation control,” presented at the XXXIII Jornadas Nacionales de Automatica Conference, Vigo, Spain, Mar. 2012.

[13].

A. Koszewnik, “The parrot UAV controlled by PID controllers,” Acta, Mechanica et Automatica, vol. 8, no. 2, pp. 65-69, 2014.

[14].

J. J. Lugo and A. Zell, “Framework for autonomous on-board navigation with the AR.Drone,” Journal of Intelligent and Robotic Systems, vol. 73, no. 1-4, pp. 401-412, Jan. 2014.

[15].

A. Motwani, “A survey of uninhabited surface vehicle,” Marine and Industrial Dynamic Analysis, School of Marine Science and Engineering, Plymouth University, Tech. Rep., 2012.

[16].

A. Romano and P. Duranti, “Autonomous unmanned surface vessels for hydrographic measurement and environmental monitoring,” in Proceedings of FIG Working Week 2012, Rome, Italy, 6-10 May 2012.

[17].

J. Holler, S. Longfield, K. Murphy, A. Striz and B. Bingham, “Unmanned surface vehicles for undergraduate engineering education,” in Proceedings of the OCEANS 2008 Conference, Quebec City, pp. 1-8, Sept. 2008.

[18].

M. Bibuli, G. Bruzzone, M. Caccia, E. Fumagalli, E. Saggini, E. Zereik, E. Buttaro, C. Caporale and R. Ivaldi, “Unmanned surface vehicles for automatic bathymetry mapping and shores’ maintenance,” in Proceedings of the OCEANS 2014 Conference, Taipei, pp. 1-7, Apr. 2014.

[19].

J. Larson, M. Bruch, and J. Ebken, “Autonomous navigation and obstacle avoidance for unmanned surface vehicles,” in Proceedings of SPIE - The International Society for Optical Engineering, pp. 1-13, 2006.

[20].

M. Kristan, V. S. Kenk, S. Kovacic and J. Pers, “Fast image-based obstacle detection from unmanned surface vehicles,” IEEE Transactions on Cybernetics, vol. 46, no. 3, pp. 641-654, Mar. 2016.

[21].

H. Niu, H. Oh, A. Tsourdos and A. Savvaris, “Development of collision avoidance algorithms for the C-enduro USV,” IFAC Proceedings Volumes (proceedings of 19th IFAC World Congress), vol. 47, no. 3, pp. 12174-12181, Aug. 2014.

[22].

D. Hermann, R. Galeazzi, J. C. Andersen and M. Blanke, “Smart sensor based obstacle detection for high-speed unmanned surface vehicle,” IFAC-PapersOnLine (proceeding of 10th IFAC Conference on Maneuvering and Control of Marine Craft MCMC 2015), vol. 48, no. 16, pp. 190-197, Aug. 2015.

[23].

[24].

M. Quigley, K. Conley, B. Gerkey, J. Faust, T. Foote, J. Leibs, R. Wheeler and A. Y. Ng, “ROS: an open-source Robot Operating System,” presented at the ICRA Workshop on Open Source Software, Kobe, May 2009.