We strive to make searching your data as easy as possible. You should be to just drop in and IP address or url and get results. This is possible and you'll find more examples under Easy Search.
ShadowTrackr also supports more complex queries. Instead of requiring you to learn another query language, we'll just stick to things you're likely already familiar with. Both Elastic Search and Splunk are widely used in cybersecurity, and the basic of both query languages are supported. You'll find examples below.
If you want to now what type of data you can search in ShadowTracker, have a look at the
Data Model.
If you type URL in the search bar, ShadowTrackr assumes you want to check if it is in your assets and will do that. The following are all considered valid URLs:
test.shadowtrackr.com
http://test.shadowtrackr.com
https://test.shadowtrackr.com
Similarly, if you put in an IP address ShadowTrackr will search your hosts, an email address will be searched in the Emails index, and "ASN1234" will show you all hosts in the Hosts index for that ASN. Lastly, anything else will be considered a keyword and ShadowTrackr will search all Events (those on your Timeline) for that keyword. You can easily
export all search results.
If you type in a URL and want to search the Events index instead of the Urls index, or you want to search the Events index for an IP address instead of the Hosts index, you will need to write a bit more advanced query. You can do this with both Elastic Search and Splunk Query syntax.
Elastic Search (or Lucene) query language generally looks like this:
certificates.grade: A
websites.url: shadowtrackr.com
dns.url:*.shadowtrackr.com AND NOT dns.rrtype:TXT
hosts.ip:139.162.214.30
The searchbar will show you suggestions while typing so you don't have to lookup which indexes and fields are available:
The ShadowTrackr query parser is quite forgiving. If you by accident use "=" instead of the ":" which is common in Elastic Search syntax, it will still work. An extra space after ":" or "=" is also accepted. And you can even leave the index part out for search terms as long as it's mentioned at least once. Indexes can be referred to in plural or singular. So these are all considered the same query:
hosts.ip:139.162.214.30 AND hosts.asn:63949
hosts.ip: 139.162.214.30 AND hosts.asn: 63949
hosts.ip=139.162.214.30 AND hosts.asn= 63949
host.ip: 139.162.214.30 AND host.asn: 63949
host.ip: 139.162.214.30 AND asn: 63949
host.ip: 139.162.214.30 AND asn=63949
In Splunk you specify the index you want to search first, and then the fields. Some typical Splunk style queries look like this:
index=certificates grade= A
index=websites.url: shadowtrackr.com
index=dns url=*.shadowtrackr.com AND NOT rrtype=TXT
index=hosts ip=139.162.214.30
The searchbar will provide you with suggestions. ShadowTrackr does not (yet) detect your preferred search style, so the suggestions are presented in Elastic Search style. You can still use this to quickly spot searchable indexes and fields though. If you mix up Elastic Searcha and Splunk search styles, the ShadowTrackr Query parsed will generally just be ok with that.
Comparing values in queries also works similar to Elastic Search and Splunk syntax:
| equal | a = b |
| not equal: | a != b |
| greater than | a > b |
| less than | a < b |
| greater or equal | a => b |
| less or equal | a <= b |
| IN | a IN (b, c, d) |
The keyword
IN allows you to compare multiple values at once. For example if you want a report of all websites that do not return a 201, 403 or 404 HTTP code:
index=websites https_status NOT IN (201, 403, 404)
Just like in Elastic Search and Splunk you can use the Logical operators AND, OR and NOT. And to properly group things, you can use ( and ). Here are some examples:
index=websites url=shadowtrackr.com AND last_seen>2022-01-01
Find all websites with url shadowtrackr.com that are last seen after 1 January 2022index=websites url=shadowtrackr.com AND (http_status=200 OR https_status=200)
Find all websites with url shadowtrackr.com where the http_status is 200 or the https_status=200. Not that ( and ) around the or is needed. If you don't add these then any website with https_status=200 would be included in the search results too.whois.domain:shadowtrackr.com AND NOT whois.dnssec:no
Find all whois records for domain shadowtrackr.com where DNSSEC is not equal to "no"Literal search is useful for when your search term contains spaces.
host.asn_name:"LINODE-AP Linode, LLC, US"
index=whois registrar = "GoDaddy.com, LLC"
The wildcard character is * and you can use it everywhere. Even in literals.
index=hosts asn=63*
index=assets ip=139.*
whois.registrar = "GoDaddy*"
index=certificates ip=139.*
You can group query results by any field in the index with the keyword "by". It works very much like Splunk "by" or SQL "group by". If you group results by a field you will automatically get an extra field named "count" with the number of results per group. This is useful for generating lists for pie charts.
index=certificates by issuer earliest=-10d
index=hosts country="United States" by asn
index=whois by registrar
There are several search modifiers available. They are generally used at the end of your query, but it will work anywhere. The order is not important and you can combine all modifiers.
earliestThis modifier can be used to include only those search results that are last seen or created
after a certain date or time. The date or time can be absolute or relative. Note that if there is a space in the datetime you need to enclose it in double quotes. If you don't include latest, all search results found since the start of your account will be returned.
index=events earliest=-5d
index=hosts remote_login_services=true earliest=-2m
index=websites earliest=-2months
index=certificates grade=F earliest=-6week
index=whois registrar="*godaddy*" earliest=-1y
index=certifiates earliest=2022-02-01
index=dns rrtype=A earliest="2022-02-01 14:00:00"
latestThis modifier can be used to include only those search results that are last seen or created
before a certain date or time. The date or time can be absolute or relative. Note that if there is a space in the datetime you need to enclose it in double quotes. If you don't include latest, all search results up until now will be returned.
index=events latest=-5d
index=hosts latest=-2m earliest=-3m
index=hosts asn=1103 latest=-2months
index=suggestions latest=-6week
index=hosts problem=yes latest=-1y
index=dns latest=2022-02-01
index=assets latest="2022-02-01 14:00:00"
| tableJust like in Splunk, you can "pipe" queries in to other commands. For now only the table command is supported. With table you can explicitly select the columns you want in the search results. It works on all queries, including queries with the "by" keyword. All queries where results are grouped with "by" will always contain the count per group in the last column (named "count"). Examples:
index=websites latest=-7d | table url ip https_status http_status
index=hosts by asn | table asn country
include_ignored=trueBy default this value is false and ignored assets are excluded from your searches. If you want to include them, you just add this to you search query:
index=websites http_server=nginx* include_ignored=true
