At a high level the following steps need to be followed for threat intelligence uploaded from a url to generate threat activity or create notables from matches in Splunk Enterprise Security:
Administrators configure threatlist sources.
Threatlist.py downloads intelligence
File parser parses the intelligence file
Threat intelligence audit dashboard shows latest status and changes
Lookup generating searches extract and deduplicate threat data
Threat generating searches compare lookups to events
Matches found populate the threat_activity index
Threat activity detected correlation search creates notable events from matches
Source:
Configuring Threat Intelligence in Splunk Enterprise Security by Joe Abraham, PluralSight Course
https://www.pluralsight.com/courses/configuring-threat-intelligence-splunk-enterprise-security
This is also drawn out in the diagram from the Splunk Developer Guide detailing the Splunk Threat Intelligence Framework
The process starts with the administrator adding the data from a reputable threat feed. This is done in Configure >> Enrichment >> Threat Intelligence Management >> New >> Line Oriented
For this example a generic threat feed will be used, not one in STIX or OpenIOC formats. This means that additional care will need to be shown towards proper field extraction.
In the Parsing screen enter a transforms.conf expression to extract the fields corresponding to the threat lookups you’re looking to populate.
To see the complete list of fields that can be used in the Fields expression, we'll need to look in the corresponding KVstore collection. Here we're uploading a list of urls which will be uploaded to the http_intel KVstore collection.
You can also get the list of fields by looking in the datasets of the Threat_Intelligence datamodel
Once the threat feed has been saved, verify if the lookup gen searches de-duplicated the threat intelligence data and entries made it into the KVstore collection. You will find the corresponding Lookup Gen searches if you look for "- Lookup Gen" in Searches, Reports, and Alerts. Use quotes to limit result set to relevant knowledge objects.
| `http_intel`Next we verify if the data made it into the Threat Intelligence datamodel
| datamodel Threat_Intelligence HTTP_Intelligence search | table *If you’re having trouble downloading the url feed, check in index _internal for hints on why it may have failed.
index=_internal sourcetype="threatintel:download"After a while verify if threat activity was detected based on the uploaded intelligence
| datamodel Threat_Intelligence Threat_Activity search | table *
Threat activity is generated based on “Threat Gen” scheduled searches. You will find the corresponding Threat Gen searches if you look for "- Threat gen" in Searches, Reports, and Alerts. Use quotes to limit result set to relevant knowledge objects. In the case of url threats the threat generating search is Threat - URL Matches - Threat Gen.