Users Logged Out
Incident Report for MachineMetrics
This incident has been resolved.
Posted Feb 06, 2018 - 10:14 EST
This morning at approximately 6:05am Eastern Time, the server that maintains our user sessions (whether a user is logged in or not) failed. This resulted in all users being logged out. It is important to note that all important data (machine data, production data, settings, etc) are stored in very resilient systems that are both redundant and backed up regularly. Session information, however, is not.

Log your users back into the tablets. The system is operating normally.

Why? This session information needs to be accessed very frequently and must be fast. Using an all in-memory database for storing session information allows for this data to be quickly accessible and can handle a significant number of requests. The downside is that it is susceptible to loss if the server ever is powered down. Because this data is easily recoverable (by logging a user back into a computer, phone, or tablet), the risk is low.

One important note is that we will likely be upgrading the server that maintains session information in the near future to prevent this issue from occurring again as we grow. When we do, users will be logged out again until you log them back in.
Posted Feb 06, 2018 - 07:02 EST