Evaluating algorithm performance
Evaluating our tools is an essential part of our scientific approach. We follow a methodology that involves self-assessing the performance of our algorithms: after having taught an algorithm to differentiate between individuals based on their position or company, we test this differentiation capacity and calculate the reliability odds.
Interpreting the results
Our algorithms not only allow us to appreciate an individual's sense of belonging to a group (company or position), but also to analyze the group in and of itself. We can actually analyze the structure of the psychometric data that defines a group of individuals; in other words, the psychological characteristics that define that group.
The graph below corresponds to an algorithm with a high degree of reliability, showing a very good prediction score and associated area under the ROC curve (AUC).
The area under the ROC curve represents the ability of our algorithm to distinguish between individuals who do or do not belong to a specific group. An area of 50 means that our algorithm will not predict any better than at random. On the other hand, an area of 100 means a perfect prediction: no false positives (what seems to be positive, but is in fact negative) and no false negatives (what seems to be negative, but is in fact positive). The area under the ROC curve can never be less than 50.
Here, an accuracy of 95% means that the algorithm is capable of detecting individuals belonging to a reference group (for example, a company) from new data 95% of the time.
The graph below corresponds to an algorithm with poor performance - it is difficult to identify the behavioral invariants. A random candidate is not greatly differentiated from a candidate belonging to a reference group.