Background
Identifying which activities adversaries have performed in a compromised AWS environment is not an easy task. Therefore together with @BertJanCyber we have performed research to investigate if Sigma rules can provide first-response capabilities in a post-compromised environment.
We tested a total of 32 different attacks on an AWS environment and compared the detection capabilities of CloudTrail, GuardDuty and Sigma. In which all activities were logged, but more importantly Sigma did identify more malicious activities than GuardDuty. This blog will discuss how we investigated this and explain the results in more detail. The dataset as well as the Sigma, Athena and Splunk rules are available in our Github repository.
Experiments
The first stage of our research was to create an AWS environment that contained malicious activities. This was done using the Stratus Red Team tool, which simulates 32 different attack techniques. The image below shows the setup of our test environment.
An external attack machine was used to simulate the attacks via the AWS CLI. The AWS environment had GuardDuty enabled, which enabled us to compare Sigma and GuardDuty. The activities are logged in CloudTrail and those logs are streamed to S3. The streaming of CloudTrail logs is an incident response best practice, because it enables you to have longer retention periods and to query all logs from all regions at once. Lastly, AWS Athena was used as an SIEM solution, to query the CloudTrail events.
SIGMA -> AWS Athena
You might think, can I just execute the Sigma rules in AWS Athena? The answer is unfortunately no, they first need to be translated to SQL (Query Language of AWS Athena). This was done using the steps below:
We’ve translated all Sigma AWS rules to Athena and Splunk queries. We leveraged uncoder.io to convert the rules from Sigma to SQL to use it in Athena. The example below shows the AWS CloudTrail Disable Logging rule. However, there’s a translation error between Sigma →Athena where non existing field names are used (e.g. api.service.name). The image below shows the difference in the fields, the eventSource and eventName fields are not translated correctly.
To automatically fix this translation error, we build a script that you can run in the directory which contains all preprocessed Sigma rules, resulting in working SQL queries.
Example: output AWS CloudTrail Disable Logging rule:
SELECT * FROM cloudtrail_logs WHERE (eventSource = ‘cloudtrail.amazonaws.com’ AND eventName IN (‘StopLogging’, ‘UpdateTrail’, ‘DeleteTrail’))
Results
In this section, we will discuss the results in more detail. All 32 simulated Attack Techniques were logged by CloudTrail, which is positive. This means that incident responders at least have a way to identify malicious activities.
The table below shows, the EventName mapped to the simulated Stratus attack technique.
GuardDuty only triggered for just three of the simulated attacks, which is very limited. Especially, because these attacks are all well-known. In our opinion a GuardDuty alert should be triggered for most of these attacks.
Sigma performed much better than GuardDuty and identified 11 out of the 32 simulations. It was already expected that Sigma would not score 100%, since 100% coverage does not exists. Why not more detections? Partially because some of the simulated attacks look very legit from a CloudTrail log entry perspective. It is the task of the incident responder to add context to an event and assess whether this is malicious.
Sigma for Incident Response in AWS
So, can we use Sigma for IR in AWS environments?
The answer for is not yet! At the moment it’s not mature enough in the following aspects:
- Current rules/structure is too limited, at the moment you cannot use aggregation or statistics in Sigma rules (e.g. where ConsoleLogin > 10). To be able to identify suspicious activity we need to have advanced filtering capabilities.
- Sigma rules are everywhere, there is a central repository but rules are posted all over the internet. This can be a challenge especially for new users to figure out what ‘good’ rules are and what you should use.
- Structure of rules can be improved. Let’s take Brute Force attacks as an example at the moment there’s Sigma rules for each specific cloud (AWS, Azure, GC). However, they’re not build out from a threat perspective. A proposed structure of rules is shown below:
In this model you start with the threat and specify the treshold, from there on for each vendor you specify the log source and field for the rule to alert on. An added benefit is also that you can add additional technologies later.
- Last but not least, conversion of rules from Sigma to Athena doesn’t work out of the box, this is not necessarily a Sigma shortcoming, but something that needs to be addressed before you can use this solution.
Overall this research shows that there is a lot of promise for Sigma in combination with cloud incident response. The idea is that for Sigma to be used as a first response capability in cloud environments further development is required. Ultimately we want to have a Chainsaw like tool for cloud incident response. From a high level it would look like this:
Where a script or tool runs a set of Sigma rules against available CloudTrail logs using AWS Athena queries. This approach works in live environments or you can choose to acquire the logs and load them in another AWS account for querying.
More on this another time..