Tuesday, February 11, 2020

Zeek 3.1 Release Candidate Available

We are very happy to make a release candidate of Zeek 3.1 available today. After last year’s 3.0, this is the first feature release following our new release schedule, bringing new functionality & improvements to users interested in upgrading more frequently than our traditional annual cycle allowed for.

Some highlights in Zeek 3.1 include:

  • We are introducing the first, experimental version of a new supervisor framework, which provides an entirely new deployment mode for Zeek. Currently, ZeekControl is the primary tool to operate clusters of Zeek processes. The supervisor framework shifts that functionality into Zeek itself, enabling Zeek to spawn & monitor cluster nodes as child processes—workers, loggers, and managers. We plan to further evolve this new model over time, with the goal to eventually come to a more standard Unix-style service model that can fully replace ZeekControl. We encourage you to give the supervisor framework a try. You will need to write a bit of custom script code to configure your cluster, but we have documentation to get you started.

  • We have replaced Zeek’s old select-based I/O loop with a new architecture that does no longer spin waiting for active I/O sources. The new architecture now waits for sources to actively notify it when activity occurs. If you’re running Zeek on a low-volume traffic stream, you should see a substantially reduced CPU load. These changes do come with a couple of breaking changes, though: Zeek now supports only a single packet source at a time (i.e., no more multiple -r or -i flags); and for C++ plugins, the IOSource API has changed fairly substantially—except for existing packet sources, which should remain functional with little to no changes. Let us know if you encounter any problems with your plugin, and we’ll be happy to help. We also have more documentation on the new architecture upcoming.

  • There is now a new &on_change table/set attribute that allows script writers to specify a callback function to execute when container elements are inserted/updated/deleted. This functionality prepares for some upcoming updates to Zeek’s cluster state synchronization, but it can already be very useful on it own as well. See the on_change documentation for information.

  • A new optional script policy/protocols/conn/speculative-service.zeek adds a speculative_service field to conn.log that reports educated guesses on a connection’s protocol/service in cases where Zeek’s normal protocol detection has exhausted the buffers required for a more reliable decision.

Starting with Zeek 3.1, compiling Zeek from source now requires a C++17 compiler, as well as CMake >= 3.0. The installation section of the manual includes updated instructions for several platforms.

Please note that in Zeek 3.1, the backwards-compatibility wrappers & work-arounds that we had introduced with 3.0 to ease the Bro-to-Zeek renaming pain, have either changed their operation or in some cases been removed. Generally, anything that was reporting a naming-related warning in 3.0 now aborts with a corresponding error message. In cases where 3.0 silently continued to accept old names, 3.1 now reports warnings. Most importantly, the latter includes loading of scripts with .bro endings, which are now flagged and should be renamed to .zeek.

As usual, see NEWS for full 3.1 release notes, and CHANGES for the exhaustive list of changes. We don’t recommend release candidates for production usage, but we encourage you to give them a try and let us know if you spot any rough edges. We’ll do our best to address any issues for the final release, which we expect to come out in about two weeks.

Friday, January 31, 2020

Keep Austin weird.logs - Save The Date - ZeekWeek2020

ZeekWeek 2020 (formerly BroCon) will be held on 7-9 October at the AT&T Executive Education and Conference Center in Austin, Texas. 
Attendees will be able to “Zeek-out” on workshops, training, community presentations and visit with sponsors, core maintainers, other “Zeekers” and more.
Thanks for all your feedback after ZeekWeek last year. We listened to you and have added more training sessions and an additional threat hunting track to the lineup this year.  
Below is some of what you can expect: 
Day 1: Zeek Related Training Sessions - There will be a total of six, four-hour training sessions, four of which will be available as part of the sponsorship prospectus.
(Watch for when sponsorship opportunities open up as there will only be 4 slots available.)
Day 2 and 3: Keynote and two (2) tracks: Track 1 is for threat hunting and incident responders; Track 2 is designed for developers. 
(Watch for the call for papers for more details on submissions for each track).
Haven’t been to a Zeek event? Check out the lineup from last year.
Registration, call for participation, and sponsorship opportunities will open soon, so check back often.
Helpful Links and information:
Getting Involved: If you would like to be part of the open source Zeek community and contribute to the success of the project please sign up for our mailing lists, join our IRC Channel, come to our events, follow the blog and or Twitter feed. If you’re writing scripts or plugins for Zeek we would love to hear from you! Can’t figure out what your next step should be? Just reach out to me. Together we can find a place for you to actively contribute and be a part of this growing community.
About Zeek (formerly Bro): Zeek is the world’s leading platform for network security monitoring. Flexible, open source, and powered by defenders.  

Thursday, January 16, 2020

Detecting CVE-2020-0601 with Zeek

CVE-2020-0601 is a major security issue affecting recent versions of Microsoft Windows. In a nutshell, NSA found a vulnerability in core Windows libraries that perform certificate validation. This vulnerability can be used to craft certificates that are accepted as valid by Windows - even though they do not have a valid signature of a trusted certificate authority. The vulnerability can, for example, be used to impersonate TLS servers, to fake signature, or to fake email and file signatures.
I will not go into exactly how the vulnerability works; however if you are interested in the mathematical background, check out  this post.
Hours after the announcement of the vulnerability, we published the first package that can detect TLS exploit attempts. This was followed by a second package, that gives a slightly higher quality signal, 24 hours  day later.
In this blog post, I will provide a high-level overview of some of the basics of the exploit,and how Zeek can be used to detect it.

Elliptic Curve Certificates

Elliptic Curve Cryptography (ECC) is a public-key cryptographic scheme. From a user perspective, it works similarly to RSA - there is a public key as well as a private key. In addition, for ECC the participants need to agree on the domain parameters making up the elliptic curve - typically these domain parameters are just referred to as “the curve.” These curves are generally not made up by the participants - instead they are chosen from lists of published curves. For example, a typical curve that is used is secp384r1, which was standardized by NIST in FIPS 186-4 (note: to make things slightly confusing NIST curves have two names - secp384r1 is referred to as P-384 in the NIST publication).
When creating a certificate that uses Elliptic Curve DSA (ECDSA) as the signature scheme, the curve is typically just referred to by name - meaning that in the ASN.1 representation of the X.509 certificate just the name of the curve is given. It is expected that all participants know the parameters of the well-known curves. An example certificate using ECDSA from the Zeek test suite looks like this (unnecessary details omitted):


Note the line saying “ASN1 OID: sec384r1” which defines the curve.
There is a second way to share the curve parameters in certificates: by defining them explicitly. Instead of just giving a curve name, in this case all parameters are included in the certificate. This feature nowadays is rarely used for a variety of reasons - and it is also not supported by most software. Indeed RFC 5840, which specifies the use of ECC in X.509 certificates in the Internet, specifically states that “specifiedCurve, which is of type SpecifiedECDomain type (defined in [X9.62]), allows all of the elliptic curve domain parameters to be explicitly specified. This choice MUST NOT be used.”
The following is the OpenSSL output for a certificate that contains an explicitly defined curve.

Note that the generator, order, etc., of the curve is given directly in the certificate. Without going into specifics, the exploit requires a certificate containing such an explicitly defined curve. In addition, the curves that are used by the exploit are manipulated, meaning that they will not be from the list of standard curves.

Detecting CVE-2020-0601 using a Zeek script

To detect if someone is trying to exploit CVE-2020-0601, we need to determine if someone is trying to use a EC-certificate using a non-standard curve. It turns out that this is rather easy to do in a Zeek script. We can just use the x509_certificate event. The X509::Certificate parameter of this event contains both the key algorithm that is used by the certificate - which tells us that the certificate has an elliptic curve key - as well as the curve that is used in a certificate.
So, we can just write a simple test to check if a certificate is an EC-certificate but does not use a named curve and output a notice:

This is the exact code that is used by our detection script.
This just leaves us with one question: what happens when a certificate contains a standard curve, but encodes the parameters explicitly?
It turns out that the answer to this question is a bit complex with different OpenSSL installations behaving in various ways. Some OpenSSL versions will automatically convert well-known explicit curves back to their named equivalent. In this case, the Zeek code above will work as expected - and certificates with well-known curves that are explicitly specified will not generate a warning. Other versions will not do this; in this case you will get a false positive with this code when encountering such a certificate. Please note, however, that explicitly defined parameters are highly unusual, so the danger of false positives is very low.
To address this potential of false positives, we added a test to our package which determines if your OpenSSL installation behaves as expected. If not, you will get a test failure. You can still use the package, you just have to be aware that there is a small risk of false positives.
Furthermore, we have a second version of our package, which completely mitigates this issue. In this version, we created a C plugin that uses OpenSSL functionality to test if the curve contained in a certificate matches one of the well-known curves. The source is split across two files, a bif-file which creates a function that can test a certificate from script-land, and a second file that performs the test. The code in the second file is very long (more than 3,000 lines) - because it contains all curve definitions. This code is mostly a copy from the OpenSSL source code - which contains the functionality that we need - but does sadly not expose it directly in their API.
The drawback of this package is the higher complexity. Furthermore the package requires OpenSSL 1.1.1 - which might not be available for your Zeek installation if you are on an older release of your Linux distribution.

Getting Involved

If you have any questions about this script - or if you use it and find exploit attempts - we would love to hear from you. The best way to contact us is to share feedback on our mailing list, or you can email me directly.
The Zeek community is always interested in package contributions! If you write an interesting package that can detect an attack, adds a feature, etc., add it to the package manager and let the mailing list know.

Tuesday, January 14, 2020

Zeek Monthly Newsletter, Issue 1 - January 2020

Welcome to the Zeek Monthly Newsletter, Issue 1 covers December 2019 as well as upcoming events.

In this Issue:
  • ●  General Community News/Updates
  • ●  Development Updates
  • ●  Zeek In the News
  • ●  Interviews
  • ●  Threat of the Month
  • ●  Upcoming Events
  • ●  Contribution/Contributor of the Month
  • ●  Zeek Related Issues
  • ●  Publication Schedule
  • ●  Get Involved 

General Community News/Updates

We’re very excited to publish this first Zeek newsletter. This newsletter will be published monthly on the first full week of each month. The publication schedule can be found below.

In this issue you’ll notice some sections will only have a description as we are actively looking for content and contributors. Please consider becoming an editor. If you are interested please email news@zeek.org

Development Updates

Zeek 3.0.1 release available- This is a bug-fix release that most notably addresses a JSON logging performance regression in 3.0.0, but also fixes other minor bugs. 

Zeek in the News

This section will be for articles published outside of the Zeek Blog. If you come across articles referencing Zeek in your news feed and you’d like us to share it in the newsletter, please send the link to news@zeek.orgor add it to the Issue 2working document.

Interviews with LT/Core Contributors/Other

What community member or contributor would you like to know more about? If you have suggestions please email us at news@zeek.org.Since there wasn’t any from December and this is our first newsletter below are the links to a couple from earlier in 2019.

* Robin Sommer - https://blog.zeek.org/2019/04/people-of-zeek-interview-series-robin.html 
* Zeke Medley - https://blog.zeek.org/2019/06/people-of-zeek-interview-series.html
* Fatema Bannat Wala - https://blog.zeek.org/2019/07/people-of-zeek-interview-series.html

Threat of the Month

Do you have a threat you’d like to share with the community and how using Zeek in your security stack helped you identify that threat? Please email news@zeek.organd we’ll work with you to get it written up and shared in the next newsletter.

Upcoming Events

If you or your organization would like to host a Zeek event or if you know of any public Zeek related workshops, please send us the links so that we can include those in the newsletter and on the website.

Ask the Zeeksperts

Ask the Zeeksperts is a one hour bi-weekly call that is hosted by various “Zeeksperts” in the community. This is where you can drop by and ask your Zeek Related questions. The webinars are free to attend, but registration is required.

* 16 January 2020 - 12:30pm PT - Hosted by Seth Hall
* 30 January 2020 - 12:30pm PT - Host Robin Sommer

How to use a Raspberry Pi as a Network Sensor Webinar

* 16 January 2020 - 2:00PM EST

This webinar is being hosted by Black Hills Information Security and presented by Bill Sterns,
from Active Countermeasures.

This webcast will cover running a network sensor using a Raspberry Pi, a miniature single-board computer that runs most anything you can run under Linux.

Bill will show you how to install and use the Zeek IDS and cover the performance aspects you'll need to know. Setting up IDSs that cost about the same as a bike means you can monitor far more network segments simultaneously, and hide them behind a power brick if you have to.

Register for this webinar at: https://attendee.gotowebinar.com/register/2540509980495221261

Zeek Days Workshops

18 February 2020 - Portland OR

This is a one day Zeek workshop hosted by Portland State University and sponsored by Corelight, Inc.

This workshop is free to attend by registration is required.

In this workshop we'll introduce you to Zeek, best practice around deploying and running Zeek then we'll take a deeper look at the Zeek logs and how to use the data derived from the Zeek logs.

And for those who want to know more and become an active contributor and participant in the Zeek community we'll get you started.

Registration and more information -

Contribution/Contributor of the Month

Check out packages.zeek.org Every month we’ll pick a package to highlight. Consider contributing a package during January 2020, so we can highlight your contribution on the Zeek Blog and in the February 2020 Zeek Newsletter.

If you’d like to review the packages and interview the contributors, please email news@zeek.org.

Zeek-related Jobs

If you or someone you know have any Zeek related job postings you’d like us to share in the monthly newsletter please send links to news@zeek.org.

Publication Schedule

* Issue 1 - January 2020 (Covers December 2019) - 14 January 2020 
* Issue 2 - February 2020 (Covers January 2020) - 3 February 2020 
* Issue 3 - March 2020 (Covers February 2020) - 2 March 2020
* Issue 4 - April 2020 (Covers March 2020) - 6 April 2020

* Issue 5 - May 2020 (Covers April 2020) - 4 May 2020
* Issue 6 - June 2020 (Covers May 2020) - 1 June 2020
* Issue 7 - July 2020 (Covers June 2020) - 6 July 2020
* Issue 8 - August 2020 (Covers July 2020) - 3 August 2020
* Issue 9 - September 2020 (Covers August 2020) - 7 September 2020
* Issue 10 - Special Issue 1 - September 2020 (Covers ZeekWeek 2020) - 21 September 2020 

* Issue 11 - October 2020 (Covers September 2020) - 5 October 2020
* Issue 12 - November 2020 (Covers October 2020) - 2 November 2020 
* Issue 13 - December 2020 (Covers November 2020) - 7 December 2020 
* Issue 14 - Special Issue 2 - (Year End Review) - 21 December 2020

Get Involved

If you are interested in getting involved with the Zeek Newsletter, please email news@zeek.org

More information about the newsletter can be found here.

Stay up to date by subscribing to the Zeek Mailing List.

Follow us on Twitter

Thursday, January 9, 2020

Getting started with Zeek (Docker-style): Part 1

by David Lapsley Ph.D. 


First of all, welcome to the community! If you are reading this, then you’ve heard all about how great Zeek is and you are interested in finding out for yourself by getting it up and running and playing with it. Good move!

The goal of this article is to make the process of getting Zeek up and running as easy as possible for you, so you can spend more time exploring Zeek, and less time running over speed bumps. To do this, we are going to leverage the extremely convenient Docker toolchain in combination with some cool Docker work that has already been done in the open source community.

Good luck and have fun!

Dockerizing your computer

Some good news: most of the work to get Zeek up and running has already been automated! However, in order to leverage that automation, we need to get your computer to the point where the foundation has been laid for the automation to kick off. Fortunately, that is quite easy: all you need is to have Docker installed and running. In the following, I am going to assume you are running a Mac OSX computer like me (Windows and Linux users, we love you too, but this is the only platform I have validated this on so far -- the automation we will review will work on any platform that supports Docker -- I just haven’t verified that yet!).

So, let’s get started. On your Mac, you’ll need to install the following:

You will also need to make sure you have git installed on your Mac. If it’s not, install it with the following command:

$ brew install git

We’re almost ready to get started. One thing we need to do first is to make sure we give Docker enough RAM to be able to compile Zeek. Docker Desktop for Mac uses a VM behinds the scenes to host the Docker runtime environment. By default it allocated 2 GB of RAM to the VM. That’s not enough to compile Zeek! If you try compiling with the default RAM settings, you will hit a compile error that looks something like this:

c++: internal compiler error: Killed (program cc1plus)
Please submit a full bug report,
with preprocessed source if appropriate.
See <file:///usr/share/doc/gcc-7/README.Bugs> for instructions.

This is due to the VM hitting an Out Of Memory condition. To avoid this you will need to allocate more RAM to the VM. Click on the Docker Icon in your menubar and select "Preferences". Click on the "Advanced" tab and then use the slider to select 8 GB of RAM (6 also works, but use 8 just in case). Docker Desktop will restart and then you will be ready to go!

Checkout the Code

The zeek-docker repo (https://github.com/zeek/zeek-docker) hosts a number of Dockerfiles that can be used to build your favorite version of Zeek. Dockerfiles like the one at https://github.com/zeek/zeek-docker/blob/master/3.0.0.Dockerfile have all the instructions you need to build your Zeek container in a completely repeatable, reliable, and convenient manner. It’s definitely worth taking a read through the Dockerfile to see the steps involved in building and installing Zeek.

If you are new to Docker, the docker documentation is worth checking out. The main documentation page is https://docs.docker.com/. If you are a first timer, the getting started section is very useful: https://docs.docker.com/get-started/. If you’ve had some experience, the Dockerfile reference is always handy to have nearby:  https://docs.docker.com/engine/reference/builder/.

Ok, time to download the code so we can start doing cool stuff! Use the following command to clone the code onto your laptop:

$ git clone https://github.com/zeek/zeek-docker.git

Building your first Zeek Container

It’s time! Type in the commands below to build your first Zeek Container (in this case we are building a version 3.0.0 container):

$ cd zeek-docker
$ make build-stamp_3.0.0

That’s all you need to do! Now watch as the wonders of automation unfold, and your Zeek container is built. You should see something like this on your terminal console:

Step 24/24 : CMD /bin/bash -l
 ---> Running in c1263b7d2ea3
Removing intermediate container c1263b7d2ea3
 ---> 5bc774250a9a
Successfully built 5bc774250a9a
Successfully tagged broplatform/bro:3.0.0
touch build-stamp_3.0.0

Once the container has been built, you will see the container image is available in your local docker registry:

$ docker images  | grep -e broplatform -e REPO
broplatform/bro  3.0.0 5bc774250a9a 8 minutes ago  215MB

Hello Zeek!

We are ready to run our first Zeek container!

The command below executes the container and sets up an interactive bash shell within the container.It also mounts the current host directory (currently inside the zeek-docker repo) in the /pcap directory of the container so you can copy files into/out of the container.

$ docker run -it -v `pwd`:/pcap broplatform/bro:3.0.0 /bin/bash

At this point we are inside the container. Type the command below and you can see there is a freshly built zeek executable ready to use!

root@3535953ccd99:/# which zeek

The /pcap directory inside the container corresponds to the host working directory (this can be anywhere on your computer, for me it happens to be the zeek-docker repo directory). You’ll probably want to put pcap files in here as well so you can process the data inside the container using zeek while storing the logs in the mounted directory for later access (e.g. after the container is gone!).

root@3535953ccd99:/zeek/bin# ls -1r /pcap

Congratulations! You are now the proud owner of a brand new Zeek container, which contains an installed Zeek deployment in its own execution environment. (And you built it all by yourself !)

Putting Zeek to Work

There are two ways we can use Zeek:

  • Run it from the command line against a pre-captured PCAP file
  • Run it against a live network interface

For this post, we’ll be going through the first use case. In a follow up post, we’ll walk through how to run Zeek against a live network interface.

Now that our execution environment is all setup and we have a Zeek container, let’s process some PCAP files! (My colleague keith.jones@corelight.com has some interesting PCAPs you can take a look at in his article: https://blog.zeek.org/2019/12/how-to-add-jpeg-file-analyzer-to-zeek.html?m=1). Copy your favorite pcap file (let’s assume it’s “test.pcap”) into your current working directory and execute the following:

$ docker run -it -v `pwd`:/pcap broplatform/bro:3.0.0 /bin/bash
root@5ea58f4bb9be:/# cd /pcap
root@5ea58f4bb9be:/pcap# ls *.pcap

Cool! Now we’re running inside the container, and we can see our test.pcap file residing on our host computer via the mounted filesystem on our container’s /pcap.

Time to run Zeek! Execute the following command.

root@5ea58f4bb9be:/pcap# zeek -r test.pcap

Done! Zeek has quietly processed the pcap file, and output a bunch of data to log files in the current directory. Let’s take a look.

root@5ea58f4bb9be:/pcap# ls *.log
conn.log  dnp3.log  packet_filter.log

So now we have the log files generated by Zeek using its default set of scripts, cool! If these logs are new to you, then you might want to head over to the official Zeek documentation which contains a deep dive on the various log files and fields: https://docs.zeek.org/en/stable/script-reference/log-files.html.

As an example, we’ll take a look at the contents of the conn.log file (https://docs.zeek.org/en/stable/scripts/base/protocols/conn/main.zeek.html#type-Conn::Info).

root@5ea58f4bb9be:/pcap# cat conn.log
#separator \x09
#set_separator        ,
#empty_field        (empty)
#unset_field        -
#path        conn
#open        2019-12-04-04-38-26
#fields        ts        uid        id.orig_h        id.orig_p        id.resp_h        id.resp_p        proto        service        duration        orig_bytes        resp_bytes        conn_state        local_orig        local_resp        missed_bytes        history        orig_pkts        orig_ip_bytes        resp_pkts        resp_ip_bytes        tunnel_parents
#types        time        string        addr        port        addr        port        enum        string        interval        count        count        string        bool        bool        count        string        count        count        count        count        set[string]
1252963725.444796        Cn0OCi3r0R0aO5nAda        1413        20000        tcp        dnp3_tcp        1.049805        15        17        SF        --        0        ShADadFtf        8        343        7        322        -
1252963725.788546        CbSrUpY4Adua4a6Ad        1400        5450        tcp        -        9.006836        60        140        OTH        -        -0        DdA        10        460        5        340        -
#close        2019-12-04-04-38-26

There you have it! Success!


Here we are at the end of today’s journey, and the beginning of your Zeek journey! We have been able to Dockerize our computer, build a Zeek container, execute Zeek within the container, and process a PCAP file using the Zeek container we built.

I hope this has been useful to you, saved you a lot of time and helped to accelerate you on your Zeek journey. I also hope you had as much fun going through this as I had writing it!

I would love to hear from you, so please feel free to drop me a line (david.lapsley@corelight.com). I really appreciate feedback (especially if it helps to make this tutorial easier for folks to read/follow).

Thank you for following along, good luck, and Happy Zeeking!

About David Lapsley, Ph. D. 

Dr. Lapsley has over 20 years of industry experience. Roughly a third of
that has been spent doing applied research for various government agencies,
a third working for large telecom vendors, and a third working at startup
companies. Dr. Lapsley's expertise includes software development,
data analysis, distributed infrastructure, open source, cloud computing,
and forming and leading high performing teams.

Dr. Lapsley holds a Ph. D. in Electrical and Electronics Engineering from
the University of Melbourne, Australia. He also holds a B. S. (Computer
Science) and B. E. with Honours (Electrical and Computer Systems
Engineering) from Monash University, Australia.