Monday, April 22, 2019

Google Season of Docs

As part of the submission and ongoing docs refresh for below is the list of projects we are submitting for Google Season of Docs consideration.

  • Introduction to Zeek (rewrite)
  • How to install Zeek (rewrite)
  • How to write a Script for Zeek Guide (rewrite and new)
  • How to write a Plugin for Zeek (rewrite and here)
  • Updating and Deepening Framework Documents (rewrite)
  • Update Try documents (rewrite)
  • Using Elastic to Analyze Zeek data (new)

Zeek documentation can be found on our Read The Docs site.

More information about Zeek can be on the Zeek website.

We are going to be refreshing all the documentation as part of the name change from Bro to Zeek.

We’d like to hear from you, the Zeek community, on what you think is missing from our documentation (if not listed above).

Also, what sections of the documentation do you rely on most and what improvements to those sections would you like to see? Please send suggestions to

Helpful Links and information:

Getting Involved: If you would like to be part of the Open Source Zeek Community and contribute to the success of the project please sign up for our mailing lists, join our IRC Channel, come to our events, follow the blog and or Twitter feed. If you’re writing scripts or plugins for Zeek we would love to hear from you! Can’t figure out what your next step should be just reach out. Together we can find a place for you to actively contribute and be a part of this growing community.

About Zeek (formerly Bro): Zeek is a powerful network analysis framework that is much different from the typical IDS you may know.

Thursday, April 18, 2019

Save the Date - ZeekWeek 2019

Save the Date 

October 8th - 10th

ZeekWeek 2019 
(formerly BroCon)

King Street Ballroom & Perch, Hilton Embassy Suites

255 South King Street, Seattle WA 98104

This year ZeekWeek (formerly BroCon) will be held 8-10 October 2019 in the King Street Ballroom & Perch at the newly renovated Hilton Embassy Suites in Seattle Washington.

Attendees will be able to “Zeek-out” on workshops, training, community presentations and visit with each of the vendors, sponsors and more.

Haven’t been to a Zeek event? Check out the lineup from last year.

Registration and Call for Participation will open soon, so check back often.

Helpful Links and information:

Getting Involved: If you would like to be part of the Open Source Zeek Community and contribute to the success of the project please sign up for our mailing lists, join our IRC Channel, come to our events, follow the blog and or Twitter feed. If you’re writing scripts or plugins for Zeek we would love to hear from you! Can’t figure out what your next step should be just reach out. Together we can find a place for you to actively contribute and be a part of this growing community.

About Zeek (formerly Bro): Zeek is a powerful network analysis framework that is much different from the typical IDS you may know.

Wednesday, April 17, 2019

People of Zeek - Interview Series - Robin Sommer

Today we kick off our weekly interview series with the Open Source Zeek Leadership Team and community contributors with Robin Sommer.

Before we get started, I’d like to thank Robin for taking the time to answer my interview questions and kick off this series.

Amber Graner (AG): As one of the core developers of Zeek (formerly Bro) I am sure you are no stranger to the community. However, can you introduce yourself to readers who may not know you?

Robin Sommer (RS): I became involved with Zeek in the early 2000s when I was still a graduate student in Germany looking for a research subject in network security. After starting to contribute small patches, Zeek quickly became my primary research platform; that work led to a number of new capabilities that then made back into the open source distribution. During that time I spent a couple of summers in Berkeley working with Vern at the International Computer Science Institute, and I later joined his group as staff researcher. When the National Science Foundation decided to invest into building out Zeek as an operational tool for the US research and education community, I began leading the team behind that effort. Today, I am the CTO at Corelight, a startup that Vern, Seth, and myself co-founded to bring Zeek’s capabilities to enterprise environments.

AG: When you got interested in then Bro, now Zeek, what was the driving factor that made you want to join forces with Vern to get involved and drive this project?

RS: Zeek has always offered both a powerful platform for experimentation and a strong deployment base in real-world networks—a combination that’s gold for academic research. On the platform side, Zeek’s clean architecture separates low-level packet processing from high-level analysis and detection. As result, Zeek can not only accommodate very different environments, but also support new technical approaches that would be challenging to incorporate into other systems. On the deployment side, some of the most demanding sites started using Zeek pretty much from the beginning—which provided invaluable operational feedback for evaluation & refinement.

AG: At the recent Open Source Zeek European Workshop 2019 held at CERN in Geneva, Switzerland, you gave a talk, Looking Forward: On Supervisors, Packages, and Sandboxes. In this talk you discussed 2 topics, 1) Replacing BroControl and 2) Rethinking Zeek Packages. For those who weren’t able to be at this event and hear your talk can you give a summary here?

RS: This talk was a bit of an outlook into new things that we have on our roadmap. The more concrete of the two efforts is developing a replacement for BroControl, Zeek’s current management framework. BroControl was originally developed for a setting that’s quite different to what most Zeek users are using today: a physically distributed cluster of individual PCs all running Zeek. BroControl came out of our own desire to avoid some of the maintenance tasks coming with such a setup: installing Zeek across many identical systems, pushing out new configurations, health monitoring, etc. Today, however, even large environments are often running Zeek just on a single system, leveraging all the cores available these days instead of scaling across systems. On a single system, BroControl can feel pretty quirky unfortunately. To address that, we are planing to turn Zeek itself into a persistently running service that—while still using the traditional cluster architecture internally—will have a dedicated supervisor process that spawns and manages the necessary processes as its children behind the scenes. Doing so will fit much better into standard Unix service models.

The second part of the talk was really only some early brainstorming exploring better support for running externally provided, untrusted scripts in your Zeek installation. Zeek’s package manager has been a huge success, with more than 80 packages now out there contributed by the community. However, when installing one of these packages, there’s currently no isolation between what your local scripts and that external code. That can quickly turn out problematic: It’s pretty common for Zeek scripts to exhibit widely different performance characteristics depending on the network they are running in. If an external package ends up using up all your CPU or memory, that will shut down all your monitoring. To address that, I’d like to explore isolating scripts by running them inside separate processes. This idea is quite speculative at the moment; it’s unclear if it’s even technical feasible. But it’s an example of how we have often approached new Zeek functionality in the past: Starting with a proposal for discussion and feedback, we’d then try implementing a first prototype to gauge if it makes sense to go in further.

AG: In addition to Replacing BroControl and Rethinking the Zeek Packages, what else would you like the Zeek community to get involved with. Scripts? Plugins?

RS: There are really many ways to contribute to Zeek these days, you don’t need to write C++ code to bring value to the community (although you can!). The most direct way is publishing Zeek packages: If you have written a custom Zeek script, I encourage you to make that available; it's not hard. Beyond scripts, documentation is probably (still) our main bottleneck. While we want to improve, there's only so much that we can do on the project's side. The real value of Zeek comes from the many different use cases that it supports, and only the community as a whole can collect & disseminate that expertise. So I would like to encourage people to share what they have learned: Write blog postings, give presentations, participate on the mailing lists, join meet-ups.

AG: Besides Zeek, are there any other projects that you’re involved with and would like to share?

RS: Most of my open source work focuses on the broader Zeek ecosystem—there’s always plenty to do there. One of the related projects that I’m most excited about is Spicy: A new protocol parser generator to replace Zeek’s existing BinPAC system. As many in the community may have heard, there’s already a Spicy prototype available that demonstrates the new capabilities, but it isn’t ready for production usage yet. At Corelight, we are working on a new version of Spicy that will make Spicy real. There’s no release date yet, but the code will become open source under BSD license.

AG: Is there anything that you’d like to share with readers about Zeek that I haven’t asked you about?

RS: I would just like to thank the Zeek community for all the amazing things everybody is doing. There’s a reason that many of us on the core team have been involved with the project for such a long time: It’s extremely rewarding to work with such an excited community that always finds new ways to push the system’s boundaries into directions nobody had ever anticipated.

Thanks again, Robin!

Helpful Links and information:

Events: Stay tuned we’ll be announcing ZeekWeek dates and location this week.

Getting Involved: If you would like to be part of the Open Source Zeek Community and contribute to the success of the project please sign up for our mailing lists, join our IRC Channel, come to our events, follow the blog and or Twitter feed. If you’re writing scripts or plugins for Zeek we would love to hear from you! Can’t figure out what your next step should be just reach out. Together we can find a place for you to actively contribute and be a part of this growing community.

About Zeek (formerly Bro): Zeek is a powerful network analysis framework that is much different from the typical IDS you may know.

About Corelight:
Corelight makes powerful network security monitoring solutions that transform network traffic into rich logs, extracted files, and security insights for incident responders and threat hunters. Corelight Sensors run on open-source Zeek (formerly called “Bro”) and simplify Zeek deployment and management while expanding its performance and capabilities.

Tuesday, April 9, 2019

“Mission First, People Always.”

I’d like to take a moment and introduce myself.  I'm Amber Granerand I’m excited to join Corelight, Inc as the Director of Community for the open source Zeek project.  
When I volunteered to join the U.S. Army in 1989, the saying “Mission first, people always” was something that was often demonstrated by my senior leadership.  We worked hard. We played hard. I served, protected and belonged to something bigger than myself. Being in the Army let me be a part of history as I worked together with those in my unit to protect against enemy threats. The espirit de corps I felt when I was part of the Army, was something I always looked for in every job after the military.
You may be asking yourself, “Ok, Amber, what does joining the Army have to do with becoming the Director of Community at Corelight for the Zeek (formerly Bro) community?”  
I am so glad you asked!
When I was interviewed by the team at Corelight, I heard the words “Mission first,” and I saw “people always,” demonstrated by all of the employees that I met. I leaned in. I listened. I wanted to know more.  
Corelight is a cybersecurity company founded on open source software whose employees are demonstrating the principles and philosophies that I believe in, live by and are passionate about.  I was hooked.
Open Source –  check!
Mission First, People Always – check!
Freedoms – check!
Protect against threats – check!
Belonging to something bigger than myself – check!
Opportunity to serve and collaborate with a community – check!
There was something challenging, yet comfortably familiar about this opportunity. I wanted to be part of this organization. I wanted to get to know the people. I wanted to serve the community.
When Corelight offered me the position, I answered, “yes!”
It seemed like a perfect fit.
Before coming to Corelight I was the community manager and operations director for the Open Compute Project (OCP) Foundation, a 501(c) (6) organization which was founded in 2011 by Facebook, Intel, and Rackspace with a mission to apply the benefits of open source to hardware and rapidly increase the pace of innovation in, near and around the data center and beyond.
Prior to OCP I was the first community manager for Linaro, a collaborative engineering organization consolidating and optimizing open source software and tools for the Arm architecture.
Before diving headfirst into applying open source principles and philosophies to hardware development and innovation, I got involved in the Ubuntu community. I began my personal journey into open source in 2009, when I was given an Ubuntu 8.10 live CD, a laptop and instructions to “use the community” if I ran into any problems.
Once involved in the Ubuntu community, I was elected to the Ubuntu Community Council and became the editor-in-chief for the Ubuntu Weekly Newsletter (UWN) as well as an Ubuntu Women Project leader. Later I was asked to be a technical reviewer of edition 5 of the Official Ubuntu Book and later co-author for editions 6 and 7 of the same book.  I was also a technical reviewer for Jono Bacon’s Art of Community, Editions 1 and 2 and have written for the Linux New Media publication which featured my “You in Ubuntu” blog. My articles have also appeared in print in the Ubuntu User magazine.
However, my love for open source began way before 2009, when in 1994 I sat on the dropzone at Fort Bragg, North Carolina, watching members of the 82nd Airborne Division establish an uplink to an aircraft in flight using Red Hat Linux (Halloween Edition) and the early Acer laptops. This feat proved intelligence could be provided en route to a battlefield allowing our troops to have the best possible information before they had to exit the aircraft, and ensured the safety and protection of the troops and their area of operations.
Mission first, people always”
The opportunity to work for Corelight and collaborate with the open source Zeek  community completes the circle and brings me back to my roots.
I’m looking forward to getting to know the community. If you are passionate about open source, protecting your network and want to get involved in the open source Zeek community, ask me how.  
Look for me on the mailing lists, blog, twitter and in person at various Corelight and events. Without “U” there is no commUnity, so let’s work together to protect your network and beyond!

[Reposted with permission from the blog.]

Monday, April 1, 2019

New Zeek Release Schedule

Over the years we have released new Zeek (Bro) versions on a somewhat regular annual basis, often around the time of BroCon. We also often did smaller bug fix releases in between, typically without adding any new functionality. However, while this annual cycle gave Zeek users a stable version to run operationally, it also meant that sites seeking to leverage new features that hadn’t made it into a release yet, either had to wait a substantial amount of time before getting access, or were forced to switch to running an unstable development version from Zeek’s master git branch. To address this challenge, we are switching to a more frequent release cycle that combines the best of both worlds by (1) offering regular feature releases while also (2) providing dedicated longer-term stable versions for users who prioritize stability over new functionality. 


The next (and first :-) Zeek version will be 3.0.0, cut from git’s master branch at the time of release. It’ll be a stable version that we will support with critical bug fixes for one year, released as 3.0.x versions. During that year we will also be providing smaller feature releases 3.x.0 about every four months, again cutting them from master at that time. We’ll always support the most recent feature release with critical bug fixes (3.x.y).

About a year later, the process will repeat with stable version 4.0.0, followed by feature releases 4.x.0and so on. For stable releases, we’ll select what we’ll deem the most solid base at that point. That could mean current master, a patched version of the most recent feature release, or even just that feature release itself. For all releases with substantial changesfeature and stablewe’ll be making beta versions available early for users to validate.

In short:
  • x.0.0 are stables releases with one year of support
  • x.y.0 with y > 0 are feature releases with about 4 months of support
  • x.y.z with z > 0 are maintenance releases 

Backwards Compatibility

As a key part of the new release schedule, we will strive to maintain backwards compatibility between two subsequent stable releases, usually by first deprecating legacy functionality in stable release X, and then removing it in stable release X+1. That means that users of stable versions will have a year to adapt their installations before functionality becomes unavailable. The in-between feature releases will go ahead and take out functionality that’s cleared for removal with the next stable version. For example, if a feature becomes deprecated with 3.0.0, we can remove it in 3.1.0; and then also in 4.0.0.

Please note that we cannot promise 100% backwards compatibility between stable versions, as sometimes there’s just no good deprecation path without blocking development for an extended period. However, we will seek community input in cases where we need to choose.

Next Steps 

Zeek 3.0.0 will be the first version to follow the new schedule. We don’t have a release date for that yet. The main remaining work items for that version are finishing the renaming, and removing any functionality that’s currently deprecated in 2.6.

Monday, March 18, 2019

Beyond BroControl - A New Process Supervision Model for Zeek

Current State of Affairs

A near-term item on the Zeek Roadmap is to provide an alternative, and eventual successor, to BroControl.  For context on why that's the case, there's the following pain points:
  • Process supervision in an external tool/process like BroControl is flaky.
Other modern examples of process supervision tools recognize the benefit/control gained from being the direct parent of supervised-processes.  By moving all the process supervision logic into Zeek itself, we can have more confidence and control of the ongoing health/state of a deployment.
  • It's awkward to develop and test new scripts that are destined for production environments.
The common use-case is load-balanced network traffic across a cluster of worker processes.  We need to make it easy to test, from the command-line, using just PCAP files, a complete cluster deployment (scaled down) as it would work in production.  Having to use a separate/intermediate tool, like BroControl, during development is not conducive to a fluid programming workflow or feedback loop.
  • Atypical system/service/container management and administration.
A goal is to cater more towards modern sysadmin expectations and we've gotten feedback that the current approach of using BroControl over Zeek/Bro directly is not a typical way of operating.  We want to improve that by no longer requiring an install/run of an entirely separate software.  The main Zeek process will be all that's needed to both supervise a deployment and serve as the central point of integration in existing service/system management schemes.

BroControl evolved from a prior tool that was originally built to satisfy a particular research use-case, not necessarily modern deployments.  That's expected, coming from such an early point in time, however, with a large user-base now depending on Zeek for production use, it's wise to design a new tool that, from the start, takes into account the wider community needs.

The Plan

There's been a brief round of internal discussion already with the following design and implementation notes produced from that:

Zeek Supervisor Design Doc

To summarize the goal: we want to make the main Zeek/Bro process the point of entry for deployments and allow just running the Zeek/Bro process to create a cluster deployment comparable to what BroControl would currently configure.

We haven't started implementing any of this yet in order to capture and respond to community feedback, so please get in touch with any you may have.  The mailing list ( is a good place to discuss.