Friday, February 21, 2020

Updating a Plugin in Zeek 3.1

By Tim Wojtulewicz

With the release of Zeek 3.1 coming soon, we are now fully deprecating all of the old Bro naming, including for the plugin skeleton. This means that plugins may fail to build once Zeek 3.1 has been installed. This blog post describes a set of instructions for migrating a plugin to the new naming scheme. The changes made here will also work with Zeek 3.0, but 3.1 makes them mandatory.

The plugin skeleton has been updated to take into account all of the new naming changes, and lives in the zeek-aux repo. The easiest first steps are to clone this repo, and then run init-plugin to build a fresh skeleton for your plugin in a separate directory. This will allow you to compare your existing setup with the new naming scheme. The most notable changes will be in the configure and CMakeLists.txt files, the renaming from bro-pkg to zkg, and the renaming of script files from .bro to .zeek.

For most projects, the configure script can be copied over directly from the new skeleton into the existing plugin. The main changes in the new configure script are support for passing in a path to cmake and some minor renaming of a few of the variables used. CMakeLists.txt may contain more changes. The primary change here is the move from CMake 2.x to CMake 3 and the renaming of the CMake project attribute from BroPlugin to ZeekPlugin. If you have any custom changes to your CMakeLists.txt, you’ll likely find it easier to copy over the new skeleton version and merge your changes into it.

Handling the move from bro-pkg to zkg is fairly simply. The bro-pkg.meta file is renamed to zkg.meta. If you have a “depends” block in your zkg.meta file, you’ll need to rename the bro-pkg and bro attributes to zkg and zeek, and bump their version numbers to 2.0 and 3.0 respectively.

There are a few other minor changes that need to be made for code and test. The only real change that needs to be made to the code of a plugin is that bro-config.h is renamed to zeek-config.h and any uses of it need to be fixed. For packet plugins, there are a couple of minor include changes that are necessary for compatibility with the major IO Loop changes in Zeek 3.1. These will be detailed in a longer blog post about the IO Loop work in the near future. For tests, the only real changes are that BRO_SEED_FILE in tests/btest.cfg is now ZEEK_SEED_FILE, and that tests need to call zeek instead of bro.

I recently opened a pull request to make the necessary changes to the bro-af_packet-plugin repository, which can be used as a bit of a roadmap for updating a plugin. The PR can be found at

Tuesday, February 11, 2020

Zeek 3.1 Release Candidate Available

We are very happy to make a release candidate of Zeek 3.1 available today. After last year’s 3.0, this is the first feature release following our new release schedule, bringing new functionality & improvements to users interested in upgrading more frequently than our traditional annual cycle allowed for.

Some highlights in Zeek 3.1 include:

  • We are introducing the first, experimental version of a new supervisor framework, which provides an entirely new deployment mode for Zeek. Currently, ZeekControl is the primary tool to operate clusters of Zeek processes. The supervisor framework shifts that functionality into Zeek itself, enabling Zeek to spawn & monitor cluster nodes as child processes—workers, loggers, and managers. We plan to further evolve this new model over time, with the goal to eventually come to a more standard Unix-style service model that can fully replace ZeekControl. We encourage you to give the supervisor framework a try. You will need to write a bit of custom script code to configure your cluster, but we have documentation to get you started.

  • We have replaced Zeek’s old select-based I/O loop with a new architecture that does no longer spin waiting for active I/O sources. The new architecture now waits for sources to actively notify it when activity occurs. If you’re running Zeek on a low-volume traffic stream, you should see a substantially reduced CPU load. These changes do come with a couple of breaking changes, though: Zeek now supports only a single packet source at a time (i.e., no more multiple -r or -i flags); and for C++ plugins, the IOSource API has changed fairly substantially—except for existing packet sources, which should remain functional with little to no changes. Let us know if you encounter any problems with your plugin, and we’ll be happy to help. We also have more documentation on the new architecture upcoming.

  • There is now a new &on_change table/set attribute that allows script writers to specify a callback function to execute when container elements are inserted/updated/deleted. This functionality prepares for some upcoming updates to Zeek’s cluster state synchronization, but it can already be very useful on it own as well. See the on_change documentation for information.

  • A new optional script policy/protocols/conn/speculative-service.zeek adds a speculative_service field to conn.log that reports educated guesses on a connection’s protocol/service in cases where Zeek’s normal protocol detection has exhausted the buffers required for a more reliable decision.

Starting with Zeek 3.1, compiling Zeek from source now requires a C++17 compiler, as well as CMake >= 3.0. The installation section of the manual includes updated instructions for several platforms.

Please note that in Zeek 3.1, the backwards-compatibility wrappers & work-arounds that we had introduced with 3.0 to ease the Bro-to-Zeek renaming pain, have either changed their operation or in some cases been removed. Generally, anything that was reporting a naming-related warning in 3.0 now aborts with a corresponding error message. In cases where 3.0 silently continued to accept old names, 3.1 now reports warnings. Most importantly, the latter includes loading of scripts with .bro endings, which are now flagged and should be renamed to .zeek.

As usual, see NEWS for full 3.1 release notes, and CHANGES for the exhaustive list of changes. We don’t recommend release candidates for production usage, but we encourage you to give them a try and let us know if you spot any rough edges. We’ll do our best to address any issues for the final release, which we expect to come out in about two weeks.