Adlumin Secure Data Collector Application
August 28, 2018
By: Dan McQuade
One of the challenges we face as a cloud-based SIEM platform is the process of collecting data from a variety of disparate sources on a local network, and securely transmitting that data into our platform over the internet. These sources can include end-user PCs, Windows/UNIX/Linux servers, firewalls, VPN servers, network security monitoring devices, and more. For traditional end-user desktops and servers, Adlumin has addressed this problem with custom applications that monitor activity and securely transmit the data into our platform for analysis. For hardware devices such as firewalls and VPN servers, the problem is a bit more challenging, as there is usually no easy way to install custom software on such devices.A common feature amongst firewalls and other network-based hardware devices, is the ability to forward log data in syslog format to an external source. One of the benefits of dealing with syslog data is that it usually conforms to one of a handful of standards (RFC 3164, RFC 5424, etc.), and can therefore be easily parsed for analysis on the receiving end. However, the transmission generally occurs over TCP or UDP as unencrypted plain-text, and therefore transmitting such data over the public internet to the Adlumin platform is not an option. We needed a way to capture syslog data, and securely forward it into our platform for analysis. Enter the Adlumin Syslog Collector.
The Adlumin Syslog Collector is a custom application written in Python, which runs on a Linux-based virtual machine as a systemd service. The application listens on numerous pre-defined TCP and UDP ports, securely forwarding all incoming data over an encrypted TLS connection to the Adlumin platform for collection and analysis. Once ingested, syslog data is immediately available to be viewed and searched through using the Adlumin dashboard. Powerful visualizations are generated in real-time, giving users the ability to spot patterns and identify threats as they occur.
We designed the syslog collector with ease-of-use in mind, and in less than 15 minutes it can be fully up and running, ready to receive and forward data. It offers a user-friendly GUI, which allows it to be installed and configured even if the end-user isn’t proficient with Linux or the command line. The application is shipped as a single-file OVA (Open Virtualization Format Appliance) and is capable of running under most modern hypervisors (VMWare, VirtualBox, etc.). The configuration required to deploy the Adlumin Syslog Collector is very straightforward. The only steps required to get up and running are as follows:
- Load the OVA into the hypervisor and boot the system
- Change the default password
- Enter the client-specific Adlumin endpoints
- Configure the network interface
- Set the time zone on the virtual machine
- Verify the configuration
- Route syslog traffic to the forwarder
Once the initial setup is completed, no further intervention is required of the end-user. As long as the virtual machine is running, the application will securely forward all received data to the Adlumin platform. Out of the box, the application has eight built-in listeners for a variety of syslog data sources. These include: firewall, VPN, network security device (i.e. FireEye NX), endpoint security, Carbon Black, and two miscellaneous listeners. Each listener resides on a unique TCP or UDP port (specified in the documentation). Support for additional listeners and data sources is constantly being added, based on requests and feedback we receive from our clients.
To keep up with the dynamic threat landscape, modern SIEMs must be able to interpret massive amounts of log data from a wide variety of applications and devices that reside on an enterprise network. Traditional on-premise SIEMs can become overloaded with this data, and it may take the user hours to sort through it all. The Adlumin Syslog Collector filters and normalizes syslog data in our cloud-based platform at unparalleled speed, in order to paint a more complete picture of the activities occurring on a network and to alert on anomalous events as they occur in real-time.