IntelMQ 3.0.2 improves the performance of high-load data collection
Shortly after the 3.0.1 release, an IntelMQ user reported general performance issues, which we investigated over the following days. We identified two causes for them at the heart of IntelMQ: As it turned out, they just affected the data feed-collection components "collectors" with high load (streaming data or just a few big reports were not a problem) or those which use threading (receiving API collector and STOMP collector).
Due to the nature of these bugs they were imminent only in bigger and/or more complex setups. Both issues stemmed from complex overhauls in IntelMQ's core in IntelMQ 3.0.
This blog post is part of a series of blog posts related to our 2020-AT-IA-0254 project, which also support our participation in the CSIRTs Network.