All 'LOF' for the team.

Sorry for the mix of Dutch and English; What I intend to say is: "praise the team". They're doing really good! But moreover LOF stands for local outlier factor. An outlier detection algorithm they implemented themselves from scratch in Spark.

But first things first! They have more and more to show. After uploading a file, one can analyse it and the first results are shown. In an overview:

With boxplots:

And an anomaly percentage:

They have as well already set up Spark streaming via Kafka. Next sprint they will show the results of streaming outlier detection algorithms too. It goes so well, that we gave them some extra work: next Thursday is InfoFarm at the office day: everyone is working from the office and the students get the opportunity to show what they already have. And on there final day at InfoFarm, the students can practice their sales talk, while showing their prototype.

But we're not there yet! For now the static anomaly detection part is done. The most intelligent, hence hardest to implement, algorithm is the Local Outlier Factor algorithm. Calculate the Local Reachability Distance for a points neighbours (based on the knn algorithm) and then actually calculate the LOF score. At this weeks retrospective, we asked the team to give a "flower" to someone who did something really good for him or the team. The first flower was: "thank you for implementing the LOF algorithm, I for sure did not want to start on that".

And let's praise the team a few times more: they are highly motivated and aiming for the best possible performance (always asking for performance goals, which is in Big Data and Data Science hard to set). They are hard workers: every retrospective they are telling that the day work at home is really good (in order to get it again), but when you see them updating JIRA in the weekend, or having architecture or algorithm disscussions at Slack at night, then they deserve it. Why not let them tell themselves how good they are with a Confluence extract from the meeting notes: