We’re glad to have you with us as we continue our Unleash the Power of Censys Search blog series, which talks about ways to get the most out of Censys Search. We recently discussed how threat hunters and teams can work smarter with matched services, as well as collaborate with tags and comments.
In this post, we’re diving into the advantages of custom field selection.
See What You Want to See
The host data available in Censys Search is massive. In quantifiable terms, we’re talking about a dataset that includes 242M+ IPv4 hosts, 175M+ IPv6 hosts, and 1.2B+ virtual hosts, all of which can be explored with thousands of host fields. This volume of data increases every day as our comprehensive scanning captures new hosts that come online.
Access to such extensive data can help teams accelerate their security efforts on a number of fronts. Threat hunters can conduct comprehensive threat investigations, incident responders can more swiftly assess if their organization is at risk, and researchers can observe trends in internet activity on a global scale. However, this treasure trove of data becomes even more useful with the ability to tailor your view of it.
You can do this with an API feature called custom field selection. Custom field selection allows you to curate what is returned in the API response from a host query. Using custom field selection, host previews aren’t limited to default fields; instead, only the custom fields you request will display. This makes for hyper-targeted search results that save teams valuable time and reduce the need for additional lookups.
Custom field selection is available to Censys Search API users leveraging a paid package. These packages include our new self-service options: Censys Search Solo and Censys Search Teams.
Move Beyond Default Fields. Choose Your Own.
Let’s first talk about using the Censys API. API access gives Censys Search users the ability to integrate Censys data into their own tools and work streams. For example, a lot of teams use our API to enrich their SIEM solutions. As with using any API, however, you typically want to execute as efficiently as possible.
Our custom field selection feature helps you do this. Here’s how it works.
When you run a host search, your API response will display previews of hosts matching your specified query. In these previews, there are a small number of default fields that are displayed. These default fields capture common data points of interest like host IP, name, and location.
However, we know that sometimes users want data beyond these default fields. In fact, sometimes depending on the use case, you might not have a need for any of the default fields that display on host previews.
Instead, you may want to look at one of the other 20+ fields that are captured within individual host pages. For example, you may want to pull the CPE, so that you can match up CPE to other data sources, such as the NVD database for CVEs. Without custom field selection, which allows you to add the field “services.software.uniform_resource_identifier” to your query, you’d need to manually sift through your search results and parse the contents of the full host.
Custom field selection, however, lets you request one of these other 20+ fields from the start. When you add a custom field (or multiple), your host previews will display what you’ve specifically called for.
In this way, you can deftly move beyond default fields and achieve a customized view.
Save Time…and Queries
Using custom field selection makes it possible to work faster, and use fewer queries along the way.
Time Savings
Instead of manually parsing through returns, your fields of interest will now display in the host preview. This can save an exceptional amount of time for teams that frequently need to look beyond default fields. Less time spent manually parsing through data means more time to focus on other security matters at hand.
Fewer Queries
Using custom fields also means you’re served up exactly what you’re looking for the first time around. Without custom field selection, you may need to execute a lookup for every single host for which you want additional information. This, in turn, cuts into monthly query quotas. With custom field selection, you can use your query quota to perform more searches, instead of lookups.
How to Use Custom Field Selection
Custom fields can be added directly to your API search request. If you’re a current Censys Search API user, you know that all API GET queries start with the address for Censys API endpoints (https://search.censys.io/api). From there, you can add on syntax for custom fields.
Basic Example
Let’s say your security team is hunting for spoofed domains. You’ve had threat actors redirect visitors to your various web domains in the past, and you want to keep an eye on this potential vulnerability. One way to do this in Censys Search is by looking for malicious favicon use, via favicon hashes.
To view favicon hashes without looking up each individual host, you could provide “services.http.response.favicons.md5_hash” in your search request after “fields” and it would return the favicon hash for each.
Your search request would display as:
curl \
-X ‘GET’ \
-u “${CENSYS_API_ID}:${CENSYS_API_SECRET}” \
‘https://search.censys.io/api/v2/hosts/search?q=services.http.response.headers.server%3A%20nginx%3F%2A&fields=services.http.response.favicons.md5_hash&per_page=1’
Advanced Example
As mentioned above, it’s possible to add multiple custom fields to your search. This is where the value of custom field selection can become exponential; multiple searches can consolidate into just one as your search becomes more specific.
In this example, we want to:
“Search for hosts with an HTTP service reporting nginx in the server header, with at least some additional characters following the exact word, and for each hit, return the server header and any identified software packages in CPE format.”
To do that, our search request would display as:
curl \
-X ‘GET’ \
-u “${CENSYS_API_ID}:${CENSYS_API_SECRET}” \
‘https://search.censys.io/api/v2/hosts/search?=services.http.response.headers.server%3A%20nginx%3F%2A&virtual_hosts=EXCLUDE&fields=services.port%2Cservices.service_name
%2Cservices.software.uniform_resource_identifier&per_page=1’
To use custom fields, all you need to know is the field you want returned. We have a complete guide to those fields here. You can also find examples of queries to run on hosts here.
Additionally, you can use our AI-powered CensysGPT tool to turn natural language queries into Censys Search queries. The CensysGPT tool expedites searches and provides a low barrier-to-entry for Censys Search newcomers.
A Customized API Experience
Time and queries are valuable resources, and custom field selection is designed to help you optimize both! Save time with curated host previews and save queries with searches that return what you’re looking for on first request. With custom field selection, you can avoid parsing through individual hosts and instead enjoy API search responses that are customized to your specific needs.
Need access to custom field selection? Learn more about our Censys Search packages.
A Note for Community Users About the Censys API
We recently shared that we will be making some changes to our Censys Search Community version. Namely, we will be discontinuing API access beyond 60 days. This will apply to both new Community Users and current Community Users. This means that Community Users who created their Censys Search Community accounts on or before December 6, 2023 – the date our self-service packages were launched – will no longer have API access after February 5, 2024.
Any Community User who created a Community account after December 6, 2023 will have API access for 60 days after their specific date of enrollment.
You can read more about this update in our recent blog.
As always, we appreciate your understanding and cooperation as we strive to maintain a high standard of service!
To learn more about how to upgrade your account, please visit our pricing page.
Use Custom Field Selection
