1. INTRODUCTION

In factory environment, industrial robots are expected to assemble a product. However, robotic assembly is difficult since a robot has to assemble a variety of objects. Sometimes, a robot has to realize a very skillful and complex motion.

On the other hand, this research aims to fully automate the product assembly process and proposes a novel robotic grasp/assembly planner. In conventional grasp/assembly planners [2,3,6,7[, we simultaneously planned the grasping posture of a part, the assembly sequence and the motion of a robot. However, conventional grasp/assembly planners can just realize a series of pick-and-place motion of a robot. For the purpose of realizing more complex and skillful motion using a grasp/assembly planner, we consider incorporating the database of skillful motion. The motion data can be generated using several methods such as human motion capture and robotic motion generation methods. Especially, our motion generation tool utilizes the previously proposed three-layered structure of motion data. In our proposed grasp/assembly planner, if the solution path includes complex and skillful motion, such motion is realized using the database of skillful motion.

This paper briefly outlines our novel grasp/assembly planner using the motion data and shows a simple numerical example. The rest of this paper is organized as follows; we first explain the three-layered data structure in Section 2. The robotic grasp/assembly planner is explained in Section 3. Application of motion data to the grasp/assembly planner is explained in Section 4. A numerical example is shown in Section 5.

2. THREE-LAYERED DATA STRUCTURE

In this section, we explain the structure of the motion data and method for storing skillful motion to the database [1]. An assembly task is composed of multiple task elements where each task element is composed of a set of motion sequence. According to this classification, the first, the second and the third layers are respectively named as “work”, “task” and “action” layers. Figure 1 shows an example of assembly of a toy airplane. Since a single action usually includes multiple objects to be assembled, the object held by a robot is named as the “main object” and the object assembled to the main object is named as the “sub object”. The following is a set of information included in the database:

• •

  Motion strategy (A sequence of task/motion)

• •

  Human (Identified name etc.)

• •

  Robot (Name, type, hand)

• •

  Object Name

• •

  Object category (e.g., JICFS code)

• •

  Object shape

• •

  Object parameters (Mass, CoG, material, stiffness, etc.)

• •

  Grasping configuration (Main object)

• •

  Fixture configuration (Sub object)

• •

  Time series of the main object’s pose w.r.t the sub object

• •

  Control method and its parameters

• •

  Force information applied by the hand

Figure 1

Three layered data structure and data expression using YAML format.

The data can be obtained by several methods such as a human motion capture and a motion generation tool. The data is stored in the YAML format where some examples are shown in Figure 1.

On the other hand, we have constructed a software tool manually constructing a skillful motion and store it to the database [1]. Among the three-layered structure of database, our proposed grasp/assembly planner especially uses the action data corresponding to the edge of the AND/OR graph of assembly which will be explained in the next section.

3. GRASP/ASSEMBLY PLANNER DEFINITION

This section shows the overview of the grasp/assembly planner which has proposed in Harada et al. [2] and Wan and Harada [3]. Let us consider a product composed of m parts P = (P₁, · · ·, P_m). Let us also consider assembly of parts by C = (C₁, · · ·, C_r). For example, the assembly C_l composed of the parts P_u, P_v, and P_w can be defined as Equation (1)

where T_uv and n_uv denote the homogenous matrix and the approach vector, respectively, of P_v relative to P_u used for an assembly task.

To assemble a product, a gripper has to grasp an assembly of parts. We consider a set of n grippers H = (H₁, · · ·, H_n) where H_j denotes the j-th gripper. We consider using a grasp planner to calculate a grasping posture where G_ljk denotes the k-th grasping posture of the assembly C_l by the gripper H_j. By iteratively calculating grasping posture with changing the approach position, we consider constructing a set of grasping postures G_lj = (G_lj1, · · ·, G_ljk) of the assembly C_l grasped by the gripper H_j.

Every possible assembly sequence can be expressed using the AND/OR graph G(V, E) where the node V is composed of the assembly of parts C_l, l = 1, · · ·, r (Figure 2). In the grasp/assembly planner introduced in this section, we assume that the AND/OR graph is defined in advance. To obtain robot motion to assemble a given product, we first solve the AND/OR graph [4,5] and obtain a possible assembly sequence. Then, we consider constructing the manipulation graph from the assembly sequence including a set of grasping posture. By solving the manipulation graph, we simultaneously plan grasping posture and robot motion. The method is detailed in our previous papers [2,3].

Figure 2

AND/OR graph of our assembly task where the red lines denote the edges with which the action data is associated.

4. APPLICATION OF ASSEMBLY MOTION DATA

The grasp/assembly planner explained in the previous subsection just can realize a series of pick-and-place motion. We consider extending this grasp/assembly planner for more complex tasks using the database of skillful assembly motion data. In the AND/OR graph shown in Figure 2, the wing is picked by the gripper and put on the body, and then the cockpit can be picked and put on the wing. However, the bolt cannot be assembled to the wing just using a pick-and-place motion. The edges marked in red shows that the assembly motion data is required for realizing this assembly sequence. We consider associating the action Yaml file corresponding to this assembly sequence. To obtain an assembly sequence, we consider solving the AND/OR graph. If an action Yaml file is associated to the solution edge, we consider using the action Yaml file to generate the robot motion. Otherwise, we consider constructing the manipulation graph and solving it to obtain the robot motion.

5. RESULT

We consider performing the robotic assembly of a toy airplane. We prepared the skillful motion of the robot rotating the bolt stored in the database by the YAML format. We first obtained the solution path of the AND/OR graph. Since the solution path includes the screwing motion of the bolt, we used the action YAML data of the screwing motion. The result is shown in Figure 3. As shown in Figure 3, when the robot assembles the bolt to the wing, the robot tries to rotate the bolt using the task motion data.

Figure 3

Simulation result of toy airplane assembly. (a) Picking the body, (b) Placing the body to the assembly position, (c) Picking the wing, (d) Fitting the wing to the body, (e) Picking the cockpit, (f) Fitting the cockpit to the body, (g) Picking the bolt, (h) Screwing the bolt 1, (i) Screwing the bolt 2, (j) Screwing the bolt 3.

6. CONCLUSION

In this paper, we proposed a motion-data driven robotic grasp/assembly planner. A motion data includes an information on skillful motion which cannot simply be realized using conventional grasp/assembly planners. This paper first explained the structure of motion data. Then, we explained the proposed grasp/assembly planner. Finally, we showed a numerical example of a toy airplane assembly.

ACKNOWLEDGMENT

This research was conducted under the support of NEDO AI-Robotics Project.

Authors Introduction

Prof. Kensuke Harada

He received his Doctoral degrees in Mechanical Engineering from Kyoto University in 1997. From 2016, he has been working as a Professor at Graduate School of Engineering Science, Osaka University.

Dr. Natsuki Yamanobe

She received the PhD Degree from the University of Tokyo in 2007. She is currently a Senior Research Scientist with Intelligent Systems Research Institute of Advanced Industrial Science and Technology (AIST).

Prof. Weiwei Wan

He received his PhD degree from the University of Tokyo in 2013. From 2017, he has been working as an Associate Professor at the Graduate School of Engineering Science, Osaka University.

Dr. Kazuyuki Nagata

He received his PhD in Engineering from Tohoku University in 1999. He is currently a Senior Research Scientist with Intelligent Systems Research Institute of Advanced Industrial Science and Technology (AIST).

Prof. Ixchel G. Ramirez-Alpiar

She received the PhD degree in Mechanical Engineering from Osaka University, Osaka, Japan, in 2013. Since 2016, she has been working as an Assistant Professor at the Graduate School of Engineering Science, Osaka University.

Prof. Tokuo Tsuji

He received his Doctoral degrees from Kyushu University in 2005. From 2016, he has been working as an Associate Professor at Institute of Science and Engineering, Kanazawa University.