Think of alerts as digital bear traps that help you catch a problem. You set the trap and when it fires this will be logged on your timeline. You can also configure alerts to be sent to your email address or call a webhook.
You can monitor for certain events to happen. Basically anything you can formulate as a
query can become an Alert. Just go to
alerts in the left hadn menu, click "New alert", and enter the query you want. Next you can specify the trigger condition. The alert will trigger if the number of results of the query is greater than, less then, equal to or not equal to (you can set this) a threshold value you can specify.
Next, you can specify the trigger action. In any case, and event indicating your alert fired will be added to your timeline. Additional actions are sending the results of the query as Excel, csv, json or file file by email, or calling a webhook with the results as json data, or as a message for Microsoft Teams, Slack, Mattermost, Discord, Google Chat or PagerDuty (see
Webhooks).
Click
Run in ShadowTrackr under any query to see what it finds today. If it looks right, create an alert from it: paste the query in a new alert and set the trigger to
is greater than 0.
Always limit an event query to recent events with
earliest=-1d. Without it the query also counts old events, so it keeps matching and an alert that is throttled to once a day would fire again every day.
A port opens on any hostRemote Desktop (port 3389) opensA critical or high severity vulnerability is found on your assetsA certificate starts offering an outdated TLS versionA DMARC policy is changedA website stops sending a security headerA scan that changes several things at once still produces one event on your timeline: "3 ports opened on host X" is one event, not three. To alert on one particular value inside such an event, use
change=:
change=<field>:<value> one specific value, for example change=tcp_port_opened:3389
change=<field> any value, for example change=san_added
change!=<field>:<value> events that do not contain that value
You can use
* as a wildcard in the value, and combine
change= with other terms such as
earliest=-1d or
asset=. The fields are:
Ports tcp_port_opened, tcp_port_closed, tcp_port_changed
udp_port_opened, udp_port_closed, udp_port_changed (value is the port number)
Certificates cn, san, issuer, caa_email, caa_issuer, servername, ciphersuite, protocol
DNS records the record attribute, such as p, rua, mname or serial. Add eid= to pick the record type:
198 DMARC, 196 DKIM, 197 SPF, 142 SOA
Websites header, software, script, css
Certificates, DNS records and the website fields above end in _added, _removed or _changed.
For example protocol_added:TLS 1.0, san_removed:old.example.com or cn_changed.
Websites also have: security_header_removed, keyword_missing, keyword_reappeared,
keyword_changed and status_changed.
A value that contains a space, such as the protocol
TLS 1.0, needs quotes around the whole term:
change="protocol_added:TLS 1.0".
Only events that happened after this feature was introduced can be matched; older events cannot.
Instead of an email, an alert can call a webhook. Choose
Call webhook as the action, enter the webhook URL, and pick what ShadowTrackr should send in the
Webhook payload setting:
No data an empty request, just a ping that the alert fired
Results as JSON a JSON array with one object per result row
Microsoft Teams a Teams message (Adaptive Card)
Slack a Slack message
Mattermost a Mattermost message
Discord a Discord message (embed)
Google Chat a Google Chat message
PagerDuty a PagerDuty event, which creates an incident
Splunk HEC event one Splunk HTTP Event Collector event per result row
Custom headers: under
Custom headers you can add up to 10 HTTP headers, one per line as
Name: value, that are sent with every call. Use this to authenticate to the receiving tool, for example
Authorization: Splunk [HEC token] to post straight to Splunk's HTTP Event Collector. Header values are credentials: they are stored encrypted and shown masked (
Authorization: ********) after you save. Leave a line as it is to keep its value, or type a new value to replace it. Content-Type is always application/json and cannot be changed, and headers like Host and Content-Length are set by ShadowTrackr.
Sending alerts to Splunk: add the header
Authorization: Splunk [HEC token], choose
Splunk HEC event as the payload and use the event endpoint of your HTTP Event Collector as the webhook url,
https://your-splunk:8088/services/collector/event (use the HEC URL of your own Splunk stack. For a paid Splunk Cloud stack it looks like
https://http-inputs-[your stack].splunkcloud.com:443/services/collector/event; other stacks and trials use another host form or port, so copy it from your Splunk settings). If the token has indexer acknowledgement enabled, Splunk also wants a channel, and without one it answers
Data channel is missing (code 10): add a second custom header
X-Splunk-Request-Channel: [any GUID], or add
?channel=[any GUID] to the URL. Each result row is sent as one event with the alert name and id (
alert,
alert_id), the row itself under
result, and the customer it belongs to:
organization {id, name} for the alert of an organization,
group {id, name} for the alert of a group. Splunk discards the HTTP headers of a request, so what you need to recognise a retried delivery is inside every event as well:
delivery_id (the same for every attempt of one call),
attempt (1, 2 or 3) and
result_index (the position of the row). The
time of the event is the start of the call, the same on every attempt. So a result that arrives twice because of a retry has the same alert_id, delivery_id, result_index and time, and you can drop the copy. The host is
shadowtrackr, the sourcetype
shadowtrackr:alert, and the index is the default index of your token. Results over about 4 MB are cut, and a last event with
truncated says how many of the results were sent.
TLS: ShadowTrackr verifies the certificate of your endpoint. A self-signed, expired or mismatching certificate is refused and the alert records
Error: the TLS certificate of [host] is not trusted; there is no option to switch verification off. Put a certificate from a trusted authority on the endpoint. This matters for Splunk: the HTTP Event Collector of a self-managed Splunk ships with a self-signed certificate (and some trial stacks have been reported to do the same), and such an endpoint cannot receive webhooks until it has a trusted one.
The call succeeds if your endpoint answers with a 2xx status code. The status that came back is logged with every alert that fired, so if a webhook is refused you can see why.
The request: ShadowTrackr calls your URL with
POST and the header
Content-Type: application/json, plus your custom headers. With
Results as JSON the body is a JSON array with one object per result row of the alert query. The keys of each object are the columns of that query, so they depend on what you search for. For example:
POST /your/endpoint HTTP/1.1
Content-Type: application/json
X-ShadowTrackr-Delivery: 05f4ffdc6332be9af1c3650c6c85f0aa
X-ShadowTrackr-Attempt: 1
[
{"url": "app.example.com", "grade": "F", "...": "..."},
{"url": "shop.example.com", "grade": "D", "...": "..."}
]
No data sends the same request with an empty body. The chat and incident formats send the JSON their tool expects, see below.
Retries: ShadowTrackr makes up to 3 attempts. It tries again when your endpoint does not answer within 30 seconds (10 seconds to connect), when the connection fails, or when it answers
429 or a
5xx status. It waits 1 second before the second attempt and 3 seconds before the third, or the number of seconds in your
Retry-After header if that is longer (at most 10). Any other answer is final: a 2xx counts as delivered, and a 4xx such as 401 or 404 is not retried because it would be the same next time. The status of the last attempt is logged with the alert. Every attempt of one call carries the same
X-ShadowTrackr-Delivery id, and
X-ShadowTrackr-Attempt says which attempt it is, so if your endpoint may have processed an earlier attempt that did not get an answer through, you can ignore a delivery id you have already seen. A TLS certificate error or an invalid URL is final at once: it would be the same next time. A redirect (3xx) is not followed and counts as a failure with that status. A call that fails all attempts is not sent again later; the results stay available under the alert.
The chat formats (Teams, Slack, Mattermost, Discord and Google Chat) show the name of the alert, the number of results, the first 10 results and a link to the alert in ShadowTrackr. If there are more than 10 results the message says how many more there are. Chat tools refuse or cut off long messages, so the complete results are always available under the alert in ShadowTrackr. Characters in your results that a chat tool would act on, such as @channel or
*, are neutralised so an asset name cannot ping your whole channel. For the full results in the call itself, use
Results as JSON.
Many tools want a token in an HTTP header before they accept a call. Enter these under
Custom headers on the Action tab of the alert, one per line, as
Name: value:
Authorization: Splunk 0a1b2c3d-0000-0000-0000-000000000000
X-Api-Key: 9f8e7d6c
These headers are sent with every call of the alert, with any payload format. This makes it possible to call, for example, the Splunk HTTP Event Collector (with
Results as JSON), or the API of your ticketing system.
A few things to know:
- You can set at most 10 headers per alert.
- Content-Type is always application/json. Host, Content-Length, Transfer-Encoding,
Connection and Expect are set by the connection itself. Lines for these headers,
and lines that are not a valid header, are not saved. The page tells you which.
- Header values are stored with the alert and shown to everyone who can edit it.
Use a token that is only allowed to do what the alert needs.
- The audit log records that the headers changed, never their values.
Teams no longer supports the old Office 365 connectors. Incoming webhooks now run on Power Automate (Workflows), and the trigger there only accepts a Teams message with an Adaptive Card. That is the format ShadowTrackr sends when you choose
Microsoft Teams as the payload. If you send
Results as JSON to that trigger, it is refused with a
400 Bad Request.
To set it up:
1. In Teams, open
Workflows and create a flow from the template
Post to a channel when a webhook request is received (or, in Power Automate, create a flow that starts with the trigger
When a Teams webhook request is received followed by the action
Post card in a chat or channel).
2. Pick the team and channel the alerts should go to, and save the flow.
3. Copy the
HTTP POST URL of the trigger.
4. In ShadowTrackr, edit your alert, set the action to
Call webhook, paste the URL as
Webhook url, and set
Webhook payload to
Microsoft Teams.
5. Save the alert. The card is posted the next time the alert fires.
If you want to decide yourself what the message looks like, or send alerts to something other than Teams, start your flow with the trigger
When an HTTP request is received instead and leave the request body schema empty. Set the payload to
Results as JSON. The body is an array of objects, one per result row, with the columns of your query as keys, so you can use it in any following step.
1. In Slack, create an app for your workspace (
api.slack.com/apps), switch on
Incoming Webhooks and choose
Add New Webhook to Workspace. Pick the channel.
2. Copy the webhook URL. It looks like
https://hooks.slack.com/services/T000/B000/XXXX and is a secret: anyone with the URL can post to your channel.
3. In ShadowTrackr, set the action to
Call webhook, paste the URL and choose
Slack as the payload.
1. In Mattermost, open
Integrations →
Incoming Webhooks and add a webhook for the channel. Your administrator may have to enable incoming webhooks first.
2. Copy the URL, which looks like
https://mattermost.example.com/hooks/xxxxxxxx.
3. In ShadowTrackr, set the action to
Call webhook, paste the URL and choose
Mattermost as the payload.
Mattermost needs to be able to receive the call, so a server that is only reachable from your internal network will not work: ShadowTrackr calls the webhook from the internet.
1. In Discord, open the settings of the channel, choose
Integrations →
Webhooks and create a webhook.
2. Copy the webhook URL, which looks like
https://discord.com/api/webhooks/123/XXXX.
3. In ShadowTrackr, set the action to
Call webhook, paste the URL and choose
Discord as the payload.
The message is posted as an embed with the alert name as its title, linked to the alert in ShadowTrackr. ShadowTrackr never lets a message mention users or roles.
1. In the Google Chat space, open the space name →
Apps & integrations →
Webhooks and add a webhook.
2. Copy the URL, which looks like
https://chat.googleapis.com/v1/spaces/XXXX/messages?key=...&token=....
3. In ShadowTrackr, set the action to
Call webhook, paste the URL and choose
Google Chat as the payload.
Webhooks in Google Chat are only available for Google Workspace accounts, and your administrator may have switched them off.
The other formats put a message in a channel. The PagerDuty payload triggers an incident, so the person on call is paged.
1. In PagerDuty, open your service, go to
Integrations and add the integration
Events API v2.
2. Copy the
Integration Key. This is the routing key.
3. In ShadowTrackr, set the action to
Call webhook, set the webhook URL to
https://events.pagerduty.com/v2/enqueue (
https://events.eu.pagerduty.com/v2/enqueue if your account is in the EU service region), choose
PagerDuty as the payload and paste the key in
PagerDuty routing key.
Every time the alert fires an incident is created with the summary
Alert fired: <name> (N results), a link to the alert, and the first 25 results as custom details. The severity is
warning. An alert without a routing key is saved as Disabled, because PagerDuty would drop the event. Combine this with the
Throttle setting on the alert, or a busy query pages someone every time it fires.
The alert library has ready-made alerts for the things most people want to know about. Each one is a query plus a trigger. You find them in the app under
Alerts →
Alert library, where you can adapt a template and create it as your own alert. Below,
Run in ShadowTrackr shows what a query finds in your own data today.
Every template fires when its query returns more than 0 results, and is throttled to fire at most once a day. The queries that look for events include
earliest=-1d (or
created>-1d) so they only count what happened in the last day.
Critical vulnerability found
This alert will fire if a new vulnerability with a CVSS score of 9 or higher is found in software on your assets. Critical vulnerabilities are quickly picked up by attackers, so patch them first.
Note that there has to be an exact match on the version for the alert to fire. If ShadowTrackr has detected the software but not the version, this alert will not work.
Actively exploited vulnerability matches your software
This alert will fire if the US CISA adds a vulnerability to its Known Exploited Vulnerabilities (KEV) catalog that matches software on your assets. These vulnerabilities are being exploited right now, so every match needs immediate action, and the due date CISA sets is the latest you should patch.
NCSC advisory matches your software
This alert will fire if the Dutch NCSC publishes a security advisory that matches software on your assets. Advisories are issued for vulnerabilities that deserve prompt attention. The alert fires once per advisory, not once per CVE.
Hosts and ports
Remote access port opened
This alert will fire if Remote Desktop (3389), SSH (22), Telnet (23) or VNC (5900) opens on any of your hosts. Remote access services are one of the most common ways in for attackers, so you want to know the moment one appears. Check that it is intended, restrict it to a VPN or an allow-list, and use multi-factor authentication. Add or remove ports in the query to fit your environment.
index=events (change=tcp_port_opened:3389 OR change=tcp_port_opened:22 OR change=tcp_port_opened:23 OR change=tcp_port_opened:5900) earliest=-1dRun in ShadowTrackr →Database port opened
This alert will fire if a database or cache port opens on one of your hosts: MySQL, PostgreSQL, SQL Server, Oracle, MongoDB, Redis, Elasticsearch or Memcached. These services should almost never be reachable from the internet, and exposed databases are a leading cause of data breaches. Close the port or put it behind a firewall or VPN.
index=events (change=tcp_port_opened:3306 OR change=tcp_port_opened:5432 OR change=tcp_port_opened:1433 OR change=tcp_port_opened:1521 OR change=tcp_port_opened:27017 OR change=tcp_port_opened:6379 OR change=tcp_port_opened:9200 OR change=tcp_port_opened:11211) earliest=-1dRun in ShadowTrackr →Port changes with problems
This alert will fire if a port change on a host involves a service with a known problem, such as unencrypted logins or an insecure protocol. These are the port changes that need attention first.
Any new open port
This alert will fire every time a TCP port opens on any of your hosts. It is the strictest port alert and can be noisy on busy infrastructure, but it is the best way to keep your external attack surface exactly as intended. If it is too much, start with the remote access and database port alerts.
Hosts with problems
You should strive for zero hosts with problems. If you have that, you can use this alert to let you know that a new host with a problem is found, or an existing host has developed a problem.
Certificate expires soon
This alert will fire if a certificate expires within 14 days and has not been renewed. An expired certificate makes browsers show a security warning instead of your website. Use it to catch renewals that silently failed. Certificates that have already expired are included.
Certificates with bad grades
If you want your visitors to trust you, all your certificates should have an A grade. This alert will fire daily if you have any certificate with a lower grade.
Outdated TLS version enabled
This alert will fire if a server starts offering SSL 3.0, TLS 1.0 or TLS 1.1. These protocol versions are outdated and insecure, and modern guidelines require them to be switched off. A new entry usually means a configuration change or a rebuilt server that lost its hardening.
index=events (change="protocol_added:SSL 3.0" OR change="protocol_added:TLS 1.0" OR change="protocol_added:TLS 1.1") earliest=-1dRun in ShadowTrackr →Certificate issuer changed
This alert will fire if the certificate on one of your assets is now issued by a different certificate authority. That can be a planned move, but it can also mean someone else obtained a certificate for your domain. Check who requested it.
New names added to a certificate
This alert will fire if names are added to the Subject Alternative Names (SAN) of a certificate on one of your assets. It helps you spot new subdomains and services that appear without anyone telling security, and certificates that cover more than they should.
Domains and DNS
Domain expires soon
This alert will fire when one of your domains is within three weeks of expiring, with further reminders at two weeks and one week. Renew in time: once a domain expires, someone else can register it and use it for phishing or to take over your mail. Dutch .nl domains do not publish expiry dates, so they are not covered.
Domain nameservers changed
This alert will fire if the nameservers of one of your domains change at the registry. Changing nameservers is how domains are hijacked, because whoever runs the nameservers controls where your website and email go. If you did not do it, contact your registrar immediately.
Domain registrar changed
This alert will fire if one of your domains moves to a different registrar. That is normal after a planned transfer, but an unexpected transfer is a sign of domain hijacking. If you did not request it, contact your registrar immediately.
DNSSEC switched off
This alert will fire if DNSSEC is switched off for one of your domains. DNSSEC protects your DNS answers against tampering. Turning it off by accident, for example during a migration, silently removes that protection.
Email security records changed
This alert will fire if the DKIM, SPF or DMARC record of one of your domains changes. These records protect your domain against spoofing and decide whether your mail is delivered. An unexpected change can mean a mistake, or someone preparing to send mail in your name.
New phishy domains
This alert will fire if any new phishy domains are found. It should run daily.
Website defaced
This alert will fire if the content of one of your websites changes in a way that looks like defacement. Defacement is visible to your visitors and damages trust, so you want to be the first to know.
Website security header removed
This alert will fire if a website stops sending a security header it used to send, such as Content-Security-Policy or Strict-Transport-Security. That usually follows a deployment or a webserver change that lost its hardening.
Website security grade changed
This alert will fire if the security grade of one of your websites changes. Use it to notice when a deployment improves or weakens your website's protection.
Assets and discovery
New assets to review
This alert will fire if ShadowTrackr finds new domains, subdomains or hosts that may belong to you. Review them under Suggestions and add them as assets, so they are monitored too. Unknown assets are exactly where unpatched software tends to hide.
New subdomain found for specific domain
This alert will fire every time a new subdomain is found. It uses a list of specific event ids for all new URLs, and then filters out the suggestions and a specific domain (shadowtrackr.com in the example).
Don't forget to change "shadowtrackr.com" to your domain.
index=events eid IN (101, 114, 127, 130 , 133, 146 , 147, 150, 151, 152, 153, 191 , 600, 650, 1351, 1390, 1391, 1396, 1410, 1664) message="Discovered and added url*shadowtrackr.com" AND created > -1dRun in ShadowTrackr →Asset appeared on blacklist
This alert will fire if an asset is found to be on a blacklist. A level above 80 means a warning or a problem, so you'll want to take action and get it removed, otherwise your visitors and clients may get security warnings when they visit your website or send you email.
Only new listings are reported. Informational lists, such as VPN ranges, are left out. Tor exit nodes and Bitcoin nodes are flagged as warnings, so they are included.
Keyword found in data dump or news
This alert will fire if one of your keywords appears on a data dump or paste site, or in the news. Use keywords such as your domain, a project name or an internal hostname to find leaked documents, source code and credentials early.
Email address found in a data breach
This alert will fire if an email address on one of your assets turns up in a known data breach. Ask the affected users to change their passwords, and consider enforcing multi-factor authentication to protect against reused passwords.
There are two basic types of OSINT sources you can monitor for keyword: datadumps and news websites. You monitor datadumps for dataleaks: internal documents, source code, hacked accounts, passwords, api key and so on. Since journalists report about security problems as well, it can be useful to monitor the news too.
If, for example, you want to be alerted when your email addresses appear on a copy-paste site on the internet, you can set a trap to monitor for the domain part of your email address (
@shadowtrackr.com) in datadumps. We monitor lots of sources and regularly revise them, but these days more than 90% of alerts fire on pastebin.com and github.com.
When your company appears in the news due to a databreach or hack you'll want to be the first to know of course. For us, this means we monitor the following keyword combinations in the news:
shadowtrackr dataleak
shadowtrackr hack
shadowtrackr confidential
Note that if you use multiple keywords the trap will fire only if they all occur in the text (but there can be other words in between). If you want a literal match (no other words in between), like "strictly confidential", you can put double quotes around the words. You can combine this with other, non-literal words, just like in a Google search.
By default only whole words are matched. If you want to match partial words (like
sub in
subdomain) you need to tick the optional checkbox for that.
Ever heard of the canaries that miners took with them in coal mines? These little birds are more sensitive to dangerous gasses than humans. If a canary suddenly died the miners knew there was a dangerous gas leak, and still had time to get out. Canary tokens are the digital version of this. You hide them on your server, laptop, website or even in your mailbox. When they are used, you know something is wrong.
If you want to know more about canary tokens, read this
introduction to Canary tokens on the Thinkst blog. You can create canary tokens for free on
canarytokens.org. The process is simple and fast.
You can add your canary tokens to ShadowTrackr. This means they will be logged and you have the option to convert them into email alerts or push messages to your smartphone (if you install the app).
