SIEM (Security Information and Event Management) systems are comprehensive security solutions designed to collect, analyze, and correlate log data from various sources across an organization’s network. By monitoring and identifying security incidents in real-time, SIEM systems help organizations detect and respond to threats, enhance compliance with regulatory requirements, and improve overall security posture by providing centralized visibility and actionable insights into potential security incidents.
There are many SIEM vendors in the market as shown in this study by Gartner.
Microsoft has its own SIEM implementation through its Defender product line, it is also offered as part of some Microsoft 365 plans.
Problem :
Organizations using Office 365 may not have a plan with Defender included, and may have invested in a different product. In such a case the customer would like to have the data from Office 365 flow into the SIEM product through an integration mechanism.
Office 365 exposes an API called Management API , this aggregates actions and events that users perform on various Office 365 services like SharePoint, Exchange, Teams, including system generated logs from Threat Protection and Threat Intelligence services into a log Microsoft refers to as “Unified Audit log”. This is described in detail here. There are couple of challenges here with the integration:
-> The API only provides data for the last 7 days. A typical SIEM system would want to have access to data for a larger duration.
-> To be able to successfully extract data from the API, a client has to perform some steps, for example register an App in Azure AD for AuthN and AuthZ with appropriate permissions, subscribe to the content types needed and establish a solution to continuously receive the data on a schedule.
-> The 3rd party SEIM system may need custom extensions to be written to integrate with an API this involves working with SIEM vendor.
-> Many SIEM solution have mechanism to ingest data from a storage system like Azure Storage.
Solution :
We have built a solution to read the data from the log and save to an Azure storage. Some of the features are as below:
-> Saves the audit logs to Azure blob storage. The data would be saved in multiple files in JSON format.
-> First time the background service is run on initial deployment/ restart, the service would retrieve the last 7 days of data first and then proceed to retrieve the data as per the defined schedule.
-> Trace logs would be written to a telemetry system to monitor the application execution.
-> Email based alerts would be configured to notify on failures.
-> Solution can be extended to provide search capabilities
The solution is built on Azure utilizing Azure PaaS services with simple architecture as shown here:
Solution highlights from operations point of view:
-> Most of the solution deployment is automated using Infrastructure as code.
-> Where possible, the solution utilizes Azure Managed Identities to minimize storage of credentials, keys in configuration.
Summary :
We built an accelerator to integrate Office 365 audit logs to a 3rd party SIEM system, reducing your time and effort to deploy such a solution.