Skip to content

Adding Feeds

Adding a feed doesn't necessarily require any programming experience. There are several collector and parser bots intended for general use. Depending on the data source you are trying to add as a feed, it might be only a matter of creating a working combination of collector bot (such as URL Fetcher) configuration and a parser bot (such as CSV parser) configuration. When you are satisfied with the configurations, add it to the intelmq/etc/feeds.yaml file using the following template and open a pull request!

<NAME OF THE FEED PROVIDER>:
    <NAME OF THE FEED>:
      description: <DESCRIPTION OF WHAT KIND OF DATA THE FEED PROVIDES>
      additional_information: <ANY ADDITIONAL INFORMATION>
      documentation: <FEED HOMEPAGE/DOCUMENTATION URL>
      revision: <DATE WHEN YOU ADDED THIS FEED>
      public: <TRUE/FALSE IF THE DATA SOURCE IS PUBLICLY AVAILABLE>
      bots:
        collector:
          module: <MODULE USED FOR THE COLLECTOR BOT>
          parameters:
            name: __FEED__ # KEEP AS IT IS
            provider: __PROVIDER__  # KEEP AS IT IS
            <ADDITIONAL COLLECTOR BOT PARAMETERS>
        parser:
          module: <MODULE USED FOR THE PARSER BOT>
          parameters:
            <ADDITIONAL PARSER BOT PARAMETERS>

If the data source utilizes some unusual way of distribution or uses a custom format for the data it might be necessary to develop specialized bot(s) for this particular data source. Always try to use existing bots before you start developing your own. Please also consider extending an existing bot if your use-case is close enough to it's features. If you are unsure which way to take, start an issue and you will receive guidance.

Feeds Wishlist

This is a list with potentially interesting data sources, which are either currently not supported or the usage is not clearly documented in IntelMQ. If you want to contribute new feeds to IntelMQ, this is a great place to start!

Note

Some of the following data sources might better serve as an expert bot for enriching processed events.