Machine Learning – Labeling Datasets – Possible Approaches

In a Machine Learning work, Data Scientists and Data Engineers can adopt the following approaches to label their datasets.

Firstly, the task of labeling could be because of engineers and scientists 

  • have a dataset related to a domain but without any labels, or
  • they need to enrich the dataset with additional labels

Possible approaches are:

  1. Public Knowledge bases
    1. like StackOverflow, Wikipedia, Github
    2. DevOps teams who know how systems are deployed, interacting, accessible. They do this by creating alerts on systems, observing systems logs and deploying business applications on various global environments
    3. Of course, subject matter experts (both functional and technical)
  2. Implicit / Explicit User Behavior
    1. Look at what the user is searching for i.e.
      1. explicit user interaction and
      2. implicit user interaction
    2. Think that every user thinks themselves as special, so find out what they are accessing and at what time
  3. Inter-User similarities
  4. Heuristics
  5. cQA sites
    1. the number of questions answered on social/community QA (cQA) sites

Example Labels:

Some of the example labels in typical log analytics using Machine Learning are:

  • Connected Successfully
  • Unable to Connect
  • Connected
  • Error
  • Provider Connected