1. INTRODUCTION

The main focus of the RoboCup competitions is the game of football/soccer, where the research goals concern cooperative multi-robot and multi-agent systems in dynamic adversarial environments [1]. In the field of RoboCup, self-localization technique is important to estimate own position including goal and other robot positions and to decide strategy. Basically, we estimate the self-position with the image information, the environment information and the field information. In this paper, we describe a real-time self-localization method that applies a genetic algorithm (GA) for the RoboCup middle size league (MSL, Figure 1), which has the widest field size (12 × 18 m). In Section 2 is hardware structure, which is overview of our past robot, driving module, ball handling module and kicking module. In Section 3, vision systems, which are omni-directional camera module and self-localization method are described. And we verify the effectiveness of the proposed self-localization method using GA in Section 4.

Figure 1

RoboCup MSL.

2. HARDWARE STRUCTURE

We have restructured the most part of hardware structure every year since our team was founded in 2008. We developed the robot based on recent MSL concepts that are high torque driving module, ball handling module, electrical kicking module using solenoid and USB3.0 camera system. We have called the platform as “Mugen” series [2], and “Mugen III (M-III)” shown in Figure 2, which is improved and based on “Mugen” model, participated in RoboCup Japan Open 2017.

Figure 2

“Mugen III’ mechanical layout.

This high torque driving module equips 4-wheels and each wheel is omni-directional wheel. We use ball handling module for rotating a ball with natural direction. This kicking module can shoot a ball by solenoid.

3. VISION SYSTEM

3.1. Hardware of Vision System

The omni-directional vision system of our robot is consisted of the camera (FLIR^® Systems, Inc. Flea3 [3]), a varifocal lens (Vstone) and a hyperboloidal mirror (Vstone). We developed vision system shown in Figure 2 for RoboCup MSL robot by combining with above elements. The image captured by this vision system is shown in Figure 3a, and the image size and frame rate are 512 × 512 (pixels) and 30 (fps) respectively.

Figure 3

Process of making search model. (a) Original image from camera. (b) White detection image. (c) Orthogonalized image. (d) Search model.

3.2. Self-localization

We use a white line of MSL field for self-localization. We have proposed the self-localization method which generates the searching space based on a model-based matching using white line information, and which recognizes the robot position by optimizing the fitness function which has the correct robot position as the maximum value of the function. And this proposed self-localization method uses Genetic Algorithm (GA) [4] for optimization of the fitness function.

3.2.1. Searching model

Figure 3 shows the process of making the searching model of the proposed method. At first, we need the detection image of the white line for making the searching model based on the white line. We obtain the white detection image by employing the converting method of color space from RGB to HSV and to YUV like Figure 3b. And we generate the field information by orthogonalizing the white line information like Figure 3c. Moreover, we determine the searching model by thinning down the field information based on white line like Figure 3d. Therefore we use thinned model as searching model for the self-localization.

3.2.2. Model-based matching

The proposed self-localization method generates the searching space by model-based matching between geometric information of the white line in the MSL field and above-mentioned searching model. Here, the matching area Ω is MSL field, and white line in the field is expressed as set of pixels like Figure 4. Therefore, the matching area Ω is defined by following Equation (1).

Ω={r=(x,y)|0≤x≤xmax,0≤y≤ymax} (1)

Here, x_max and y_max are positive constant values corresponding to the image plane boundaries in x and y directions. Now let us consider the set of pixels with above-mentioned searching model shape inside Ω as moving model S_f like Figure 5. The position r˜=(x˜,y˜) and orientation θ˜ of moving model is represented as ϕ˜=[x˜,y˜,θ˜]T . Then the extent of the existence of the moving model is given by:

Γ={ϕ˜=(x˜,y˜,θ˜)∈𝔕3|r˜∈Ω,-π<θ˜≤π} (2)

Figure 4

Matching area.

Figure 5

Moving model.

The movement of the moving model S_f in the matching area is expressed as Sf(ϕ˜) . And, if the pixel value of field image corresponding to the area of the moving model is expressed as p(r˜) , r˜∈Sf(ϕ˜) , then the evaluation function F(ϕ˜) of the moving model is given as follows.

F(ϕ˜)=∑r˜∈Sf(ϕ˜)p(r˜) (3)

The fitness function F(ϕ˜) obtains the maximum value when the position and orientation of the moving model corresponds to the correct position and orientation that robot exist in the MSL field. Then, the problem of detection of robot position and orientation is changed to the problem of searching ϕ˜ that maximizes F(ϕ˜) [5,6], and can be expressed as:

find ϕ˜ to maximize F(ϕ˜)subject to ϕ˜∈Γ (4)

The calculation result of whole matching area shown in Figure 6 shows Figure 7. In this Figure 7, the vertical axis represents the fitness value, and the horizontal axes represent the field plane. Here, we select only one depending on the value of an electric compass, because two maximum value exist in the function value caused by revolution symmetry of geometric characters of the MSL field.

Figure 6

Model matching.

Figure 7

Calculation result of the fitness function.

3.2.3. Verification of model-based matching

We performed an experiment to verify the effectiveness of the proposed model-based matching. Figure 8 shows the error between the correct position and the detected position in the quarter area of the MSL field at an interval of 1 m. The severity of the error amount is indicated by the brightness of the gray scale. The average error was 12.7 (cm), so the self-localization method is accurate enough for playing soccer.

Figure 8

Error of the self-localization using whole searching.

However, the processing time of whole searching for matching area Ω is 3000 (ms). So, we have needed the efficient searching method for real-time self-localization.

4. REAL-TIME SELF-LOCALIZATION METHOD

4.1. Genetic Algorithm

In the proposed self-localization method, we use GA for searching the maximum value of the fitness function F(ϕ˜) . A GA is an example of an artificial intelligence program and is well known as a parallel search and optimization process that mimics natural selection and evolution (Figure 9). In the proposed method, an elitist model of a GA that preserves the best individual in the population at every generation is utilized and genetic coding using gray code, roulette selection and one-point crossover are used. And, the parameters of the GA process are determined by previous experiments. Figure 10 shows the result of the convergence of GA in case of self-localization experiment using actual image that the robot captured at voluntary position. In this figure, the vertical axis represents fitness value of fitness function, and the horizontal axis represents the generation number of GA. The GA converged the maximum value, which means current position of robot in the MSL field, at about 60th generation in real-time.

Figure 9

GA’s process.

Figure 10

Result of the convergence of GA.

4.2. Verification Experiment using GA

We performed the self-localization experiment to verify the effectiveness of the proposed method using GA. Figure 11 shows the result of the verification experiment that checked the self-localization error using GA as same as Figure 8. Average error of this experiment was 12.9 (cm), and the accuracy of the self-localization by the proposed method is roughly the same as the above-mentioned whole searching.

Figure 11

Error of the self-localization using GA.

Moreover, we performed the experiment using five players shown in Figure 12 on the assumption of MSL game. Laptop PC displays the position sent by each robot in real-time, and the positions of five players described in the result image corresponded with the actual position of the robots.

Figure 12

Verification experiment in the real world.

5. CONCLUSION

In this paper, we have proposed the self-localization method which generates the searching space based on a model-based matching with white line information of RoboCup MSL soccer field, and which recognizes the robot position by optimizing the fitness function using genetic algorithm. Moreover, we verified the effectiveness of the proposed self-localization method using GA. Furthermore, we confirmed that the accuracy of the self-localization by the proposed method is enough to play soccer.

CONFLICTS OF INTEREST

The authors declare they have no conflicts of interest.

ACKNOWLEDGMENT

This work was supported by “FY2016 MEXT Private University Research Branding Project”.

AUTHORS INTRODUCTION

Ms. Kaori Watanabe

She received her Master’s degree from the Department of System Electronics and Information Science, Tokyo Polytechnic University, Japan in 2013. She is currently a Doctoral course student in Tokyo Polytechnic University, Japan.

Mr. Yuehang Ma

He received his B.S. degree in Engineering in 2018 from the Faculty of Engineering, Tokyo Polytechnic University in Japan. He is acquiring the M.E. in Tokyo Polytechnic University.

Dr. Hidekazu Suzuki

He is an Associate Professor of Faculty of Engineering at Tokyo Polytechnic University in Japan. He graduated from the Department of Mechanical Engineering, Fukui University, in 2000. He received his D. Eng. degree in System Design Engineering from Fukui University in 2005. His research interest is Robotics.