Censeye: Gadgets!

A few weeks ago, we launched Censeye, an open-source utility designed to help researchers and threat hunters explore Censys scan data and automatically pivot into new findings. Check out our original announcement here if you haven’t seen it yet. It’s been incredibly exciting to see the screenshots and discoveries people share—some genuinely fascinating results have already come out of it. Seeing the tool in action and uncovering cool insights? That’s just awesome.

One of the first things we wanted to tackle after the first release was building a framework to make the system easier to extend in two key areas:

1. Query generation: We needed a simple way to hook into the host data processing pipeline and dynamically generate queries for reporting and pivoting. For example, we could parse file names from HTTP open directories and search for those files in other open directories. These generated queries needed to be treated just like the field definitions already in place, pivot weight configurations, and all.
2. Data labeling: We wanted a straightforward way to add custom labels to the final reports, which would give us more control and flexibility over how the output is displayed. For example, we could look up the host in an external database and return a string to label the host.

To make this possible, we built a simple plugin system called Censeye Gadgets and started implementing some of the features we wanted to see. These are now found in the default installation of v1.0.0.

Note You can view all available gadgets by running:

~$ censeye –list-gadgets

Query Generator Gadgets

Query generator gadgets read Censys host scan data and generate one or more search queries, which are treated like any other field in the Censeye configuration. The following section details the gadgets we have implemented currently.

Open-Directory Gadget

This gadget processes a Censys host result, identifies services with an HTTP open directory, extracts file names from the response body, and generates search queries to locate those filenames in other HTTP response bodies.

In the screenshot below, we ran Censeye against 209.105.248.135 using the “-g odir” flag, which enables the open directory gadget. The report now includes several keys labeled “open-directory.gadget.censeye“, with values containing Censys search queries to find specific filenames on other hosts.

If we take a closer look, this gadget generated three queries for the files “sostener.vbs”, “svchost.vbs”, and “sostener1.vbs”. The first file was found on six other hosts with open directories, while the last two appear to be unique to the host we’re viewing.

And just like any other configured fields, this data can be used for auto-pivoting! For example, by running the following command with a minimum pivot weight of 0.9 (to skip NTLM fields for pivoting) and setting the depth to 1:

~$ censeye 209.105.248.135 –gadget odir –min-pivot-weight 0.9 –depth 1

Here, we can see that the open directory file data (“sostener.vbs”) is being used to discover other hosts, as shown in the Pivot Tree below:

Configuring the open-directory gadget

The default open directory gadget configuration looks like this (notice that it is disabled by default):

This gadget has just two configurable variables:

• max_files: Limits query generation to the first N files in the HTTP response body. The default is 32, which prevents massive open directories from consuming excessive time and resources.
• min_chars: Ensures that only filenames with at least this number of characters are processed.

The gadget must be added to the field configuration to ensure these fields are treated like any other. (Yes, that’s a lot of “field” in one sentence!) Fortunately, the default built-in configuration already handles this, and yes, you can also add ignored values just like other fields. Here’s what it looks like:

The Nobbler Gadget

The “Nobbler” gadget targets unknown services (“services.service_name=UNKNOWN”). It extracts the raw banner and generates wildcard searches at different offsets within the response.

If you’ve worked with protocol data before, you’ve likely seen that many binary protocols or framing systems include a structured header or magic number, followed by dynamic payloads. The Nobbler gadget is built to help automatically find other services with that concept in mind.

To show this off, let’s look at a real-world example of how we used this gadget to identify Metasploit payloads!

We encountered a random host (45.144.137.45) with an unusual set of services that Censys couldn’t identify. Running Censeye with the “nobbler” gadget enabled using the following command:

~$ censeye –gadget nobbler 45.144.137.45

The output (screenshot below) revealed four nobbler-generated rows, each with some intriguing metrics.

Nobbler generated the following queries and reported on them:

• services.banner_hex=5748* – 92 hosts
• services.banner_hex=574831ff* – 32 hosts
• services.banner_hex=574831ff48c7c6c4* – 14 hosts

As each query becomes more specific, fewer and fewer hosts match. This strongly suggests that these services present data framed or compiled in a machine-interpretable way. So we decided to check some of this data manually by using Netcat and piping the output to the file “data.bin”:

~$ netcat 45.144.137.45 56241 > data.bin

The file command wasn’t of much use in determining what this was:

~$ file data.bin

data.bin: data

However, binwalk was able to identify that we’re dealing with some sort of ELF binary, at least starting at offset 126 in the file.

Finally, we turned to the super-elite hacker tool, strings, to see if anything meaningful stood out—and that’s when we had an aha! moment.

With a simple web search and almost no effort, we quickly identified this service as a Metasploit (Meterpreter) payload configured to run mettle and connect back to this host.

While different generated payloads may produce varying data in Censys, the “nobbler” gadget simplifies the process by examining the first few bytes at different offsets to uncover commonalities in the raw data. To top it all off, we now know that anything matching this Censys search query is most likely related to Metasploit!

Configuring the nobbler gadget

The default configuration for nobler is disabled by default but can be enabled with the “–gadget nobbler” flag; below is the built-in default configuration:

The “nobbler” gadget has just one configuration directive: the iterations array. This defines the offsets at which wildcard searches are generated for unknown services. By default, it uses offsets 4, 8, 16, and 32.

Since this operates on the services.banner_hex value in Censys data, keep in mind that 2 bytes of this data correspond to 1 byte of actual data. For example, 8 bytes of banner data translate to 2 bytes of actual data, and so on. With the default settings, this means we’re generating queries for 2 bytes, 4 bytes, 8 bytes, and 16 bytes of real data.

For this gadget to be used in pivoting, we must also add it to the fields configuration (which is already there in the default):

Labeler Gadgets

These gadgets are straightforward, with a single goal: to augment host data by adding relevant labels. Currently, we’ve implemented two labeler gadgets:

• ThreatFox (–gadget threatfox): Checks if the host has been seen or reported on ThreatFox.
• VirusTotal (–gadget virustotal): Checks if the host has been seen or reported on VirusTotal.

The following is an example (using the host we looked at previously) of these labeler gadgets:

Developing your own Gadgets

If you would like to implement your own query generator or host labeler, check out the Censeye GitHub repository to see how the existing gadgets work – they’re extremely (and maybe embarrassingly) simple.

Final Thoughts

Censeye has been a fun evolution of simple building blocks stacking on top of each other to create a powerful and helpful utility for a very niche audience, and if you want to learn more, please check out the original announcement post about Censeye.

This open-source utility is hosted on GitHub; comments, issues, and patches are welcome!

One last thing we wanted to mention was that Censeye is now on PyPI and can be installed using Python pip:

~$ pip install censeye –upgrade