1. INTRODUCTION

Food and recipes are one of the preferred content categories found in YouTube channels. Most of user’s comments for how-to clips are in question forms related to Ingredients, Cooking tools and techniques. Even though the platform allows the communication among viewers but the comments has not yet been organized. The clip owner needs to interact with those comments however there are quite a number of comments asking the same questions whilst the owner must take his time to keep answering these comments. If we can manage the comment in such a way that the same category of question is grouped with the response. It will also benefit other viewers to explore frequently asked question based on category.

In this work, we collect posted comments from cooking video clips using YouTube API then we use ensemble learning method to learn from the training set and classify the comments into predefined class. The comment can be categorized into four classes which are: (1) How-to, (2) Ingredients, (3) Cooking tools and (4) Others. Since the dataset is imbalanced therefore, the imbalance class problem is solved by oversampling method.

The rest of paper is organized as follows. Section 2 gives the brief background of algorithms and related works. Section 3 provides the detail of our proposed method. Then all Experimental Results are shown in Section 4. Section 5 shows detail about application and the conclusion is given in the final section.

2. BACKGROUND

Begin

    H ← ∅

    for i ∈ 1, …, B do

      S(i) ← A bootstrap sample from S

      hi ← Randomized Tree Learn (S(i), F)

      H ← H ∪ {hi}

    endfor

  return H

end

Function: Randomized Tree Learn(S, F)

  At each node:

    f ← very small subset of F

    Split on best feature in f

  return The learned tree

end function

2.2. SMOTE

We found that our data set is imbalanced in terms of number of instances in each class see Table 2. The imbalanced data set affects the classification performance. The classification on imbalanced data causes problems because learning algorithm is sensitive to the number of training instances. The classification performance of majority class is higher than that of minority class. The problem is that the majority class is not the target class to be learned by the algorithm. In this research, class Others contains general comments such as admiring comments or comments that need no feedback from the clip owner. This class has higher number of comments than the target classes (How-to, Ingredients, Tools). Therefore we need the algorithm to manipulate the training set by increasing the size of the minority class either duplicates or interpolates minority instances. One of the famous oversampling method is the SMOTE algorithm [5]. SMOTE increases the number of minority class instances using interpolation method. The algorithm starts from searching for the k-nearest neighbors for each minority instance, then for each neighbor, it randomly selects a point from the line connecting the neighbor and the instance itself. This algorithm is able to generate synthetic instances rather than duplicate minority class instances; therefore, the overfitting problem can be avoided (see Table 3 for detail).

Begin

  for each instance in dataset do

    comment = preprocessing()

    vector: = featureExtract(comment)

    add vector and label to vectors

  endfor

model: = Random Forest with SMOTE (SMOTE(trainset))

result: = model.predict(testset)

end

Begin

  if N < 100 then

    Randomize the T minority class samples

    T = (N/100 ) * T

    N = 100

  endif

N = (int)(N/100) // The amount of SMOTE is assumed to be in integer multiples of 100.

k = Number of nearest neighbors

numattrs = Number of attributes

Sample[ ][ ]: Array for original minority class samples

newindex: Keeps a count of number of synthetic samples generated, initialized to 0

Synthetic[ ][ ]: Array for synthetic samples

// Compute k-nearest neighbors for each minority class sample only //

  for i ← 1 to T

    Compute k-nearest neighbors for i, and save the indices in the nnarray

    Populate(N, i, nnarray )

endfor

end

Populate(N, i, nnarray)

//Function to generate the synthetic samples//

  while N ≠ 0

    nn = Random()

    for attr ← 1 to numattrs

      dif = Sample[nnarray[nn]][attr] − Sample[i][attr]

      gap = random number between 0 and 1

      Synthetic[newindex][attr] = Sample[i][attr] + gap * dif

    endfor

  newindex++

  N = N − 1

  endwhile

return //End of Populate//

DBSCAN(D, epsilon, min_points):

  C = 0

  for each unvisited point P in dataset

    mark P as visited

    sphere_points = regionQuery(P, epsilon)

    if sizeof(sphere_points) < min_points

      ignore P

    else

    C = next cluster

    expandCluster(P, sphere_points, C, epsilon, min_points)

expandCluster(P, sphere_points, C, epsilon, min_points):

    add P to cluster C

    for each point P′ in sphere_points

      if P′ is not visited

      mark P′ as visited

      sphere_points’ = regionQuery(P′, epsilon)

      if sizeof(sphere_points’) ≥ min_points

      sphere_points = sphere_points joined with sphere_points’

      if P′ is not yet member of any cluster

      add P′ to cluster C

regionQuery(P, epsilon):

    return all points within the n-dimensional sphere centered at P with radius epsilon (including P)

3. FRAMEWORK

The framework of the comment management system (Fig. 2) consists of the feature extraction process, the classification task that applies SMOTE with Random Forest and the comment clustering. After the stopword removal and word stemming are completed, the feature selection is performed by chi-square test (χ²) [Equation (4)]. The objective of using chi-square is to remove words that are unlikely to be significant. By calculating the chi-square scores for all words, we can rank the words by the chi-square scores, then choose the top ranked words for model training. This yield a dataset containing 800 word features that are further created as a feature vectors in the training set. Note that the higher value of the chi-square score, the more likelihood the feature is correlated with the class, thus it should be included for model training.

Given the feature vectors attached with the class label, the oversampling is performed on the minority classes using SMOTE. Then the ensemble learning algorithm, Random Forest starts learning from the training set. After the comment classification is finished, all comment in each class are further clustered in order to organize the comment into hierarchical structure (see Fig. 1). Note that comments that are classified as class Others are filtered out.

Figure 1

Hierarchical structure obtained from the comment management system

4. EXPERIMENTAL RESULTS

We retrieve comment posts on cooking video clip via website YouTube. There are 16,277 comments written in Thai language each of which is labeled into four classes. Class How-to consists of content regarding cooking methods or techniques whereas comments of class Cooking tools mainly concern about cooking utensils. Class Ingredients focus on cooking recipes but class Others are miscellaneous contents such as admiring comment. Since Thai language has no word boundary, therefore Thai word segmentation tool namely Lexto that uses longest matching method is applied in the preprocessing step. The tool use Lexitron as a corpus. Stopword removal and word stemming are done as the preprocessing task.

Figure 2

The comment management framework

To explore the learning algorithm’s performance, two machine learning algorithm are evaluated, Random Forest, Naïve Bayes. The imbalanced problem is taken into consideration using synthetic minority class oversampling method namely SMOTE. We measure performance using precision, recall and F₁ (Equation 5–7), all experiments are performed based on 10-fold cross-validation.

The experimental result show that applying SMOTE with Random Forest provides the best performance compared to other algorithms (see Fig. 3 and Table 6). We found that F₁ score obtained from SMOTE + RF is 0.87 for Ingredients and Cooking tools class. For How-to class, F₁ score obtained from SMOTE + RF is 0.72 which is higher than that of SMOTE + NB and NB algorithm. We found that for imbalanced dataset using synthetic oversampling method with ensemble can enhance the classification performance. SMOTE + NB cannot improve the precision value for every class.

Figure 3

Average performance comparison

We found that NB performs better than SMOTE + NB since NB considers all features to calculate the conditional probability for each given class label. Although SMOTE algorithm increases the number of instances in minority classes but their feature values are synthesis to form the new instances. All of these feature values (a_i) are needed for conditional probability calculation that might affect classification of NB. Note that NB classify the instance using Equation (8).

We know that Random Forest algorithm creates multiple learning models by randomly selects feature subset to create different decision trees and combines the final decision using majority vote. The advantage of ensembles with respect to single models show the robustness since there is no side effect on synthetic instances created by SMOTE.

5. APPLICATION

Our application has three main features as follows:

1. (1)

   Clip video feeder obtains user query to retrieve clip video and comments via YouTube API (see Figs. 4 and 5).

2. (2)

   Comment management system classifies all comments into four classes (see Fig. 6).

3. (3)

   The incremental learning facilitate administrator to update model’s knowledge by adding new training set or relabel the misclassification instances (see Fig. 7).

Figure 4

Main user interface

Figure 5

Video Clip retrieval

Figure 6

Comment management system organizes the posts into hierarchies

Figure 7

Training set updating facilities

Figure 7 shows the output of incremental learning feature that allows user to update new instances into the training set. User can do class labelling via the system and see the performance of the new model (see Fig. 8).

Figure 8

The new model performance

6. CONCLUSION

In this work, we have studied and applied machine learning algorithms for the comment post in social media. The imbalanced problem is solved using SMOTE algorithm. There is no effect from minority class instances oversampling since the ensemble create multiple learning model using different feature sets.

The framework is implemented as the comment management system based on cooking video clips. The system provides options for the user to update the learning model in incremental style.