Elasticsearch można lubić lub nie. Fakty są takie, że robi robotę. Razem z Kibana, Logstash i Beats pozwalają w prosty sposób zbierać logi, metryki i przeprowadzać analizy w czasie rzeczywistym. Gdy potrzebujemy więcej, możemy chwycić za inne narzędzia. W tym wpisie przyjrzymy się jak połączyć Apache Spark i Elasticsearch. Psst! Repo z kodem na dole wpisu!😎Czemu Spark? Ostatnio zwrócił moją uwagę HELK, czyli rozwiązanie do analizy cyber oparte na Elastic Stack. Patrząc na jego architekturę, widzimy wy...

In this post, I will show you how to build a self-maintenance mechanism for ELK stack using curator together with cron scheduler.

Let’s assume that your system consists of a few microservices. Everything must have high availability so each microservice has at least two active instances on separate machines and everything must be multiplied by the number of testing and production related environments. When there is a situation that requires log analysis you have to skip from server to server looking for the file with desired information. You browse each file using some kind of notepad-based editor and if the files weight hundreds of...

Reading time ~16 minutes While I’m working on the next angle on how to speed up calculating similarities I started investigating how to get better telemetry from cookit. Getting telemetry is easy - making sense of it is the hard part. This also brought another pain point of current setup - logging and monitoring. Since cookit is my pet, nonprofit project it was time to do something. There is a comparison table at the end, and what I’ve choose.The current state of logging and monitoringRequirementsThe ...