Just the other day (Okay, it was a few weeks ago) I was having a discussion about logging with a “small” Fortune 50 company. Their problem was… They wanted a more intelligent way to analyze the information they are logging so they could help troubleshoot or understand problems in their environment easier. This is obviously a capability we all would love, intelligence out of our data collection, systems and event log subsystems.
Oh but logging intelligence doesn’t come without its challenges, you tell me if you experience some of the same challenges they expressed because this really throws a wrench into the works.
- Only collecting logs from some systems not every single one of them
— Not collecting Windows Event Logs, Syslog, or detailed logging from every server or device
- Inability to ingest the information of the existing logs which are being collected
- Unable to keep long collections of information in accord with compliance due to lack of allocated storage
Now let’s not even bring compliance or regulatory requirements into this, because imagine the above challenges, at scale and then retention over the course of 7 to 10 years depending upon who’s “rules” you need to follow.
You might be asking yourself just as I was asking while we were discussing this; If your unable to collect the data fast enough, without enough space to store it for a long enough duration, from an incomplete picture of your entire infrastructure… What’s the point? I mean what if we were trying to more than merely troubleshoot a problem and had to react or respond to a breach which seems all the rage these days?
With breaches like the ones which are making all the news having some elements of intelligence to analyze, interpret and act upon the data would be ideal, however without a complete picture of the environment, or only selectively logging it gives us an incomplete ability to react and respond to incidents.
The challenges we all face when it comes to logging collection can be paramount to a successfully defended and understood infrastructure.
Are there other challenges you see organizations face?
Do you find logging to be more of a ‘set it and forget it’ never to look at unless troubleshooting or responding to an incident?
I know it’s difficult to ask these questions without implicitly exposing your environment by saying, “Yes we have an incomplete logging solution” which is why it can be a sensitive topic to discuss.
What are your thoughts, is this off the mark and these issues are few and far between? I’d love to hear your thoughts on this matter.
