Event

An event represents individual piece of data processed by IntelMQ. It uses JSON format.

Example Event:

{
    "source.geolocation.cc": "JO",
    "malware.name": "qakbot",
    "source.ip": "82.212.115.188",
    "source.asn": 47887,
    "classification.type": "c2-server",
    "extra.status": "offline",
    "source.port": 443,
    "classification.taxonomy": "malicious-code",
    "source.geolocation.latitude": 31.9522,
    "feed.accuracy": 100,
    "extra.last_online": "2023-02-16",
    "time.observation": "2023-02-16T09:55:12+00:00",
    "source.geolocation.city": "amman",
    "source.network": "82.212.115.0/24",
    "time.source": "2023-02-15T14:19:09+00:00",
    "source.as_name": "NEU-AS",
    "source.geolocation.longitude": 35.939,
    "feed.name": "abusech-feodo-c2-tracker"
  }

Minimum Requirements

Below, we have enumerated the minimum recommended requirements for an actionable abuse event. These keys should be present for the abuse report to make sense for the end recipient. Please note that if you choose to anonymize your sources, you can substitute feed.name with feed.code. At least one of the fields ip, fqdn, url or account should be present. All the rest of the keys are optional. This list of required fields is not enforced by IntelMQ.

Field	Terminology
feed.name	Should
classification.type	Should
classification.taxonomy	Should
time.source	Should
time.observation	Should
source.ip	Should*
source.fqdn	Should*
source.url	Should*
source.account	Should*

* at least one of them

Classification

IntelMQ classifies events using three labels: classification.taxonomy, classification.type and classification.identifier. This tuple of three values can be used for deduplication of events and describes what happened.

The taxonomy can be automatically added by the taxonomy expert bot based on the given type. The following classification scheme loosely follows the Reference Security Incident Taxonomy (RSIT):

Classification Taxonomy	Classification Type	Description
abusive-content	harmful-speech	Discreditation or discrimination of somebody, cyber stalking, racism or threats against one or more individuals.
abusive-content	spam	Or 'Unsolicited Bulk Email', this means that the recipient has not granted verifiable permission for the message to be sent and that the message is sent as part of a larger collection of messages, all having a functionally comparable content.
abusive-content	violence	Child pornography, glorification of violence, etc.
availability	ddos	Distributed Denial of Service attack, e.g. SYN-Flood or UDP-based reflection/amplification attacks.
availability	dos	Denial of Service attack, e.g. sending specially crafted requests to a web application which causes the application to crash or slow down.
availability	misconfiguration	Software misconfiguration resulting in service availability issues, e.g. DNS server with outdated DNSSEC Root Zone KSK.
availability	outage	Outage caused e.g. by air condition failure or natural disaster.
availability	sabotage	Physical sabotage, e.g cutting wires or malicious arson.
fraud	copyright	Offering or Installing copies of unlicensed commercial software or other copyright protected materials (Warez).
fraud	masquerade	Type of attack in which one entity illegitimately impersonates the identity of another in order to benefit from it.
fraud	phishing	Masquerading as another entity in order to persuade the user to reveal private credentials.
fraud	unauthorized-use-of-resources	Using resources for unauthorized purposes including profit-making ventures, e.g. the use of e-mail to participate in illegal profit chain letters or pyramid schemes.
information-content-security	data-leak	Leaked confidential information like credentials or personal data.
information-content-security	data-loss	Loss of data, e.g. caused by harddisk failure or physical theft.
information-content-security	unauthorised-information-access	Unauthorized access to information, e.g. by abusing stolen login credentials for a system or application, intercepting traffic or gaining access to physical documents.
information-content-security	unauthorised-information-modification	Unauthorised modification of information, e.g. by an attacker abusing stolen login credentials for a system or application or a ransomware encrypting data.
information-gathering	scanner	Attacks that send requests to a system to discover weaknesses. This also includes testing processes to gather information on hosts, services and accounts. Examples: fingerd, DNS querying, ICMP, SMTP (EXPN, RCPT, ...), port scanning.
information-gathering	sniffing	Observing and recording of network traffic (wiretapping).
information-gathering	social-engineering	Gathering information from a human being in a non-technical way (e.g. lies, tricks, bribes, or threats). This IOC refers to a resource, which has been observed to perform brute-force attacks over a given application protocol.
intrusion-attempts	brute-force	Multiple login attempts (Guessing/cracking of passwords, brute force).
intrusion-attempts	exploit	An attack using an unknown exploit.
intrusion-attempts	ids-alert	IOCs based on a sensor network. This is a generic IOC denomination, should it be difficult to reliably denote the exact type of activity involved for example due to an anecdotal nature of the rule that triggered the alert.
intrusions	application-compromise	Compromise of an application by exploiting (un)known software vulnerabilities, e.g. SQL injection.
intrusions	burglary	Physical intrusion, e.g. into corporate building or data center.
intrusions	privileged-account-compromise	Compromise of a system where the attacker gained administrative privileges.
intrusions	system-compromise	Compromise of a system, e.g. unauthorised logins or commands. This includes compromising attempts on honeypot systems.
intrusions	unprivileged-account-compromise	Compromise of a system using an unprivileged (user/service) account.
malicious-code	c2-server	This is a command and control server in charge of a given number of botnet drones.
malicious-code	infected-system	This is a compromised machine, which has been observed to make a connection to a command and control server.
malicious-code	malware-configuration	This is a resource which updates botnet drones with a new configuration.
malicious-code	malware-distribution	URI used for malware distribution, e.g. a download URL included in fake invoice malware spam.
other	blacklist	Some sources provide blacklists, which clearly refer to abusive behavior, such as spamming, but fail to denote the exact reason why a given identity has been blacklisted. The reason may be that the justification is anecdotal or missing entirely. This type should only be used if the typing fits the definition of a blacklist, but an event specific denomination is not possible for one reason or another. Not in RSIT.
other	dga-domain	DGA Domains are seen various families of malware that are used to periodically generate a large number of domain names that can be used as rendezvous points with their command and control servers. Not in RSIT.
other	other	All incidents which don't fit in one of the given categories should be put into this class.
other	malware	An IoC referring to a malware (sample) itself. Not in RSIT.
other	proxy	This refers to the use of proxies from inside your network. Not in RSIT.
test	test	Meant for testing. Not in RSIT.
other	tor	This IOC refers to incidents related to TOR network infrastructure. Not in RSIT.
other	undetermined	The categorisation of the incident is unknown/undetermined.
vulnerable	ddos-amplifier	Publicly accessible services that can be abused for conducting DDoS reflection/amplification attacks, e.g. DNS open-resolvers or NTP servers with monlist enabled.
vulnerable	information-disclosure	Publicly accessible services potentially disclosing sensitive information, e.g. SNMP or Redis.
vulnerable	potentially-unwanted-accessible	Potentially unwanted publicly accessible services, e.g. Telnet, RDP or VNC.
vulnerable	vulnerable-system	A system which is vulnerable to certain attacks. Example: misconfigured client proxy settings (example: WPAD), outdated operating system version, etc.
vulnerable	weak-crypto	Publicly accessible services offering weak crypto, e.g. web servers susceptible to POODLE/FREAK attacks.

Meaning of source and destination identities

Meaning of source and destination identities for each classification.type can be different. Usually the main information is in the source.* fields.

The classification.identifier is often a normalized malware name, grouping many variants or the affected network protocol.

Examples of the meaning of the source and destination fields for various classification.type and possible identifiers are shown here.

Classification Type	Source	Destination	Possible Identifiers
blacklist	blacklisted device
brute-force	attacker	target
c2-server	(sinkholed) c&c server		zeus, palevo, feodo
ddos	attacker	target
dga-domain	infected device
dropzone	server hosting stolen data
exploit	hosting server
ids-alert	triggering device
infected-system	infected device	contacted c&c server
malware	infected device		zeus, palevo, feodo
malware-configuration	infected device
malware-distribution	server hosting malware
phishing	phishing website
proxy	server allowing policy/security bypass
scanner	scanning device	scanned device	http, modbus, wordpress
spam	infected device	targeted server
system-compromise	server
vulnerable-system	vulnerable device		heartbleed, openresolver, snmp, wpad

Examples:

If an event describes IP address that connects to a zeus command and control server, it's about the infected device. Therefore the classification.taxonomy is malicious-code, classification.type is infected-system and the classification.identifier is zeus.
If an event describes IP address where a command and control server is running, the event's classification.type is c2server. The malware.name can have the full name, eg. zeus_p2p.

Additional Information

Information that do not fit into any of the event fields should be placed in the extra namespace.Therefore the keys must be prefixed extra. string. There are no other rules on key names and values for additional information.

Fields Reference

Here you can find detailed information about all the possible fields used in an event.

`classification.identifier`

Type: String

The lowercase identifier defines the actual software or service (e.g. heartbleed or ntp_version) or standardized malware name (e.g. zeus). Note that you MAY overwrite this field during processing for your individual setup. This field is not standardized across IntelMQ setups/users.

`classification.taxonomy`

Type: ClassificationTaxonomy

We recognize the need for the CSIRT teams to apply a static (incident) taxonomy to abuse data. With this goal in mind the type IOC will serve as a basis for this activity. Each value of the dynamic type mapping translates to a an element in the static taxonomy. The European CSIRT teams for example have decided to apply the eCSIRT.net incident classification. The value of the taxonomy key is thus a derivative of the dynamic type above. For more information about check ENISA taxonomies <http://www.enisa.europa.eu/activities/cert/support/incident-management/browsable/incident-handling-process/incident-taxonomy/existing-taxonomies>_.

`classification.type`

Type: ClassificationType

The abuse type IOC is one of the most crucial pieces of information for any given abuse event. The main idea of dynamic typing is to keep our ontology flexible, since we need to evolve with the evolving threatscape of abuse data. In contrast with the static taxonomy below, the dynamic typing is used to perform business decisions in the abuse handling pipeline. Furthermore, the value data set should be kept as minimal as possible to avoid type explosion, which in turn dilutes the business value of the dynamic typing. In general, we normally have two types of abuse type IOC: ones referring to a compromised resource or ones referring to pieces of the criminal infrastructure, such as a command and control servers for example.

`comment`

Type: String

Free text commentary about the abuse event inserted by an analyst.

`destination.abuse_contact`

Type: LowercaseString

Abuse contact for destination address. A comma separated list.

`destination.account`

Type: String

An account name or email address, which has been identified to relate to the destination of an abuse event.

`destination.allocated`

Type: DateTime

Allocation date corresponding to BGP prefix.

`destination.as_name`

Type: String

The autonomous system name to which the connection headed.

`destination.asn`

Type: ASN

The autonomous system number to which the connection headed.

`destination.domain_suffix`

Type: FQDN

The suffix of the domain from the public suffix list.

`destination.fqdn`

Type: FQDN

A DNS name related to the host from which the connection originated. DNS allows even binary data in DNS, so we have to allow everything. A final point is stripped, string is converted to lower case characters.

`destination.geolocation.cc`

Type: UppercaseString

Country-Code according to ISO3166-1 alpha-2 for the destination IP.

`destination.geolocation.city`

Type: String

Some geolocation services refer to city-level geolocation.

`destination.geolocation.country`

Type: String

The country name derived from the ISO3166 country code (assigned to cc field).

`destination.geolocation.latitude`

Type: Float

Latitude coordinates derived from a geolocation service, such as MaxMind geoip db.

`destination.geolocation.longitude`

Type: Float

Longitude coordinates derived from a geolocation service, such as MaxMind geoip db.

`destination.geolocation.region`

Type: String

Some geolocation services refer to region-level geolocation.

`destination.geolocation.state`

Type: String

Some geolocation services refer to state-level geolocation.

`destination.ip`

Type: IPAddress

The IP which is the target of the observed connections.

`destination.local_hostname`

Type: String

Some sources report an internal hostname within a NAT related to the name configured for a compromised system

`destination.local_ip`

Type: IPAddress

Some sources report an internal (NATed) IP address related a compromised system. N.B. RFC1918 IPs are OK here.

`destination.network`

Type: IPNetwork

CIDR for an autonomous system. Also known as BGP prefix. If multiple values are possible, select the most specific.

`destination.port`

Type: Integer

The port to which the connection headed.

`destination.registry`

Type: Registry

The IP registry a given ip address is allocated by.

`destination.reverse_dns`

Type: FQDN

Reverse DNS name acquired through a reverse DNS query on an IP address. N.B. Record types other than PTR records may also appear in the reverse DNS tree. Furthermore, unfortunately, there is no rule prohibiting people from writing anything in a PTR record. Even JavaScript will work. A final point is stripped, string is converted to lower case characters.

`destination.tor_node`

Type: Boolean

If the destination IP was a known tor node.

`destination.url`

Type: URL

A URL denotes on IOC, which refers to a malicious resource, whose interpretation is defined by the abuse type. A URL with the abuse type phishing refers to a phishing resource.

`destination.urlpath`

Type: String

The path portion of an HTTP or related network request.

`event_description.target`

Type: String

Some sources denominate the target (organization) of a an attack.

`event_description.text`

Type: String

A free-form textual description of an abuse event.

`event_description.url`

Type: URL

A description URL is a link to a further description of the the abuse event in question.

`event_hash`

Type: UppercaseString

Computed event hash with specific keys and values that identify a unique event. At present, the hash should default to using the SHA1 function. Please note that for an event hash to be able to match more than one event (deduplication) the receiver of an event should calculate it based on a minimal set of keys and values present in the event. Using for example the observation time in the calculation will most likely render the checksum useless for deduplication purposes.

`extra`

Type: JSONDict

All anecdotal information, which cannot be parsed into the data harmonization elements. E.g. os.name, os.version, etc. Note: this is only intended for mapping any fields which can not map naturally into the data harmonization. It is not intended for extending the data harmonization with your own fields.

`feed.accuracy`

Type: Accuracy

A float between 0 and 100 that represents how accurate the data in the feed is

`feed.code`

Type: String

Code name for the feed, e.g. DFGS, HSDAG etc.

`feed.documentation`

Type: String

A URL or hint where to find the documentation of this feed.

`feed.name`

Type: String

Name for the feed, usually found in collector bot configuration.

`feed.provider`

Type: String

Name for the provider of the feed, usually found in collector bot configuration.

`feed.url`

Type: URL

The URL of a given abuse feed, where applicable

`malware.hash.md5`

Type: String

A string depicting an MD5 checksum for a file, be it a malware sample for example.

`malware.hash.sha1`

Type: String

A string depicting a SHA1 checksum for a file, be it a malware sample for example.

`malware.hash.sha256`

Type: String

A string depicting a SHA256 checksum for a file, be it a malware sample for example.

`malware.name`

Type: LowercaseString

The malware name in lower case.

`malware.version`

Type: String

A version string for an identified artifact generation, e.g. a crime-ware kit.

`misp.attribute_uuid`

Type: LowercaseString

MISP - Malware Information Sharing Platform & Threat Sharing UUID of an attribute.

`misp.event_uuid`

Type: LowercaseString

MISP - Malware Information Sharing Platform & Threat Sharing UUID.

`output`

Type: JSON

Event data converted into foreign format, intended to be exported by output plugin.

`protocol.application`

Type: LowercaseString

e.g. vnc, ssh, sip, irc, http or smtp.

`protocol.transport`

Type: LowercaseString

e.g. tcp, udp, icmp.

`raw`

Type: Base64

The original line of the event from encoded in base64.

`rtir_id`

Type: Integer

Request Tracker Incident Response ticket id.

`screenshot_url`

Type: URL

Some source may report URLs related to a an image generated of a resource without any metadata. Or an URL pointing to resource, which has been rendered into a webshot, e.g. a PNG image and the relevant metadata related to its retrieval/generation.

`source.abuse_contact`

Type: LowercaseString

Abuse contact for source address. A comma separated list.

`source.account`

Type: String

An account name or email address, which has been identified to relate to the source of an abuse event.

`source.allocated`

Type: DateTime

Allocation date corresponding to BGP prefix.

`source.as_name`

Type: String

The autonomous system name from which the connection originated.

`source.asn`

Type: ASN

The autonomous system number from which originated the connection.

`source.domain_suffix`

Type: FQDN

The suffix of the domain from the public suffix list.

`source.fqdn`

Type: FQDN

A DNS name related to the host from which the connection originated. DNS allows even binary data in DNS, so we have to allow everything. A final point is stripped, string is converted to lower case characters.

`source.geolocation.cc`

Type: UppercaseString

Country-Code according to ISO3166-1 alpha-2 for the source IP.

`source.geolocation.city`

Type: String

Some geolocation services refer to city-level geolocation.

`source.geolocation.country`

Type: String

The country name derived from the ISO3166 country code (assigned to cc field).

`source.geolocation.cymru_cc`

Type: UppercaseString

The country code denoted for the ip by the Team Cymru asn to ip mapping service.

`source.geolocation.geoip_cc`

Type: UppercaseString

MaxMind Country Code (ISO3166-1 alpha-2).

`source.geolocation.latitude`

Type: Float

Latitude coordinates derived from a geolocation service, such as MaxMind geoip db.

`source.geolocation.longitude`

Type: Float

Longitude coordinates derived from a geolocation service, such as MaxMind geoip db.

`source.geolocation.region`

Type: String

Some geolocation services refer to region-level geolocation.

`source.geolocation.state`

Type: String

Some geolocation services refer to state-level geolocation.

`source.ip`

Type: IPAddress

The ip observed to initiate the connection

`source.local_hostname`

Type: String

Some sources report a internal hostname within a NAT related to the name configured for a compromised system

`source.local_ip`

Type: IPAddress

Some sources report a internal (NATed) IP address related a compromised system. N.B. RFC1918 IPs are OK here.

`source.network`

Type: IPNetwork

CIDR for an autonomous system. Also known as BGP prefix. If multiple values are possible, select the most specific.

`source.port`

Type: Integer

The port from which the connection originated.

`source.registry`

Type: Registry

The IP registry a given ip address is allocated by.

`source.reverse_dns`

Type: FQDN

Reverse DNS name acquired through a reverse DNS query on an IP address. N.B. Record types other than PTR records may also appear in the reverse DNS tree. Furthermore, unfortunately, there is no rule prohibiting people from writing anything in a PTR record. Even JavaScript will work. A final point is stripped, string is converted to lower case characters.

`source.tor_node`

Type: Boolean

If the source IP was a known tor node.

`source.url`

Type: URL

A URL denotes an IOC, which refers to a malicious resource, whose interpretation is defined by the abuse type. A URL with the abuse type phishing refers to a phishing resource.

`source.urlpath`

Type: String

The path portion of an HTTP or related network request.

`status`

Type: String

Status of the malicious resource (phishing, dropzone, etc), e.g. online, offline.

`time.observation`

Type: DateTime

The time the collector of the local instance processed (observed) the event.

`time.source`

Type: DateTime

The time of occurrence of the event as reported the feed (source).

`tlp`

Type: TLP

Traffic Light Protocol level of the event.