Day 3 – Falco – Container runtime security tool
In the simplest definition, Falco is the tool to detect events which is suspicious in your environment.
The complex definition is that Falco is a behavioral activity monitor that detects any suspicious activity defined by a set of rules using Sysdig powerful and flexible filtering expression. It supports container and orchestrator and has a flexible notification method using standard output, Syslog, and programs like Slack. On top of that, it’s open-source, so any can contribute.
How Falco works
Once installed, it taps into the stream of system call events. To detect which event is considered an anomaly, Falco uses rules. Falco already has several rules implemented by default, and these rules are defined inside the file(/etc/falco/falco_rules.yaml). Once it detects the event, it sends out the appropriate message.
Falco provides a powerful ruleset that triggers alerts when a certain condition is met. Some of the examples are:
- If someone is trying to shell inside the container(as shown in the below example)
- Someone is trying to open the secure file like /etc/shadow
As Falco already provides this powerful ruleset but new rules and macros are easy to create.
Installing Falco in Ubuntu
- Configure the repository, and update the packages
curl -s https://falco.org/repo/falcosecurity-3672BA8F.asc | apt-key add - echo "deb https://download.falco.org/packages/deb stable main" | tee -a /etc/apt/sources.list.d/falcosecurity.list apt-get update -y
- Install kernel headers package
apt-get -y install linux-headers-$(uname -r)
- Install falco now
apt-get install -y falco
- Start the falco service and enable it on boot
systemctl daemon-reload systemctl start falco --now
- You can also run Falco as a docker container. For more info, please check https://falco.org/docs/getting-started/running/#docker
- To see how Falco works, let’s create a new pod
$ kubectl run nginx --image=nginx pod/nginx created
- Verify in which node it’s running
$ kubectl get pod -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES nginx 1/1 Running 0 7s 192.168.126.1 worker2 <none> <none>
- Login to that node and run. This will allow us to inspect the events generated by the Falco service(Make sure you have falco installed on all worker nodes by following the above method).
journalctl -fu falco
- Now try to login to the pod you have created
kubectl exec -it nginx -- bash root@nginx:/#
- In the journalctl output, you will immediately see the alert that a shell is spawned inside the container.
journalctl -fu falco
Oct 26 03:55:58 worker2 falco: 03:55:58.929949778: Notice A shell was spawned in a container with an attached terminal (user=root user_loginuid=-1 nginx (id=20e01b3e2c42) shell=bash parent=runc cmdline=bash terminal=34816 container_id=20e01b3e2c42 image=docker.io/library/nginx) Oct 26 03:55:58 worker2 falco: 03:55:58.929949778: Notice A shell was spawned in a container with an attached terminal (user=root user_loginuid=-1 nginx (id=20e01b3e2c42) shell=bash parent=runc cmdline=bash terminal=34816 container_id=20e01b3e2c42 image=docker.io/library/nginx)
- As per the above output, Falco knows that someone has spawned the shell inside the container. Now the question is, how do Falco know about it? Using rules file as discussed above(How Falco works?). Rules for “ A shell spawned inside the container” look like below.
- rule: Terminal shell in container desc: A shell was used as the entrypoint/exec point into a container with an attached terminal. condition: > spawned_process and container and shell_procs and proc.tty != 0 and container_entrypoint and not user_expected_terminal_shell_in_container_conditions output: > A shell was spawned in a container with an attached terminal (user=%user.name user_loginuid=%user.loginuid %container.info shell=%proc.name parent=%proc.pname cmdline=%proc.cmdline terminal=%proc.tty container_id=%container.id image=%container.image.repository) priority: NOTICE
- As you can the entire file is divided into 5 main section
* rule: Here you need to define the name of the rule * desc: Description of the rule * condition: This is the most important field and it matches the filter based on the events matching the rule. To match these condition you need to use sysdig filters. The falco policy engine make use of these filters to extract information about an event such as current container ID or process name etc. For more information check this link https://falco.org/docs/rules/supported-fields/ * output: Output to be generated for the event. Output also relying on filters to log the events. * priority: what is the severity of the event(other supported fields is DEBUG, ERROR, EMERGENCY, CRITICAL etc)
- Let’s look at some more use cases of Falco by performing a few more activities inside the Nginx pod. Let open /etc/shadow file
- Verify the Falco logs. As you can see, Falco saw this as suspicious activity and immediately notified us.
Oct 28 14:26:56 controlplane falco: 14:26:56.529909999: Warning Sensitive file opened for reading by non-trusted program (user=root user_loginuid=-1 program=cat command=cat /etc/shadow file=/etc/shadow parent=bash gparent=<NA> ggparent=<NA> gggparent=<NA> container_id=e353fc6c92db image=nginx)
- Let’s explore one more use case. As this is a Debian-based host, let’s upgrade the package using the below command.
# apt update
- Verify the logs to see, again Falco see this as a suspicious activity
Oct 28 14:31:55 controlplane falco: 14:31:55.549740450: Error Package management process launched in container (user=root user_loginuid=-1 command=apt update container_id=e353fc6c92db container_name=k8s_nginx_nginx_default_8e9bd36b-e4ee-4805-b1bf-bfd6d3a02c51_0 image=nginx:latest)
- For both cases, you can check the /etc/falco/falco_rules.yaml and check the corresponding rule which triggers the alert.
- Opening Sensitive Files
- rule: Read sensitive file trusted after startup desc: > an attempt to read any sensitive file (e.g. files containing user/password/authentication information) by a trusted program after startup. Trusted programs might read these files at startup to load initial state, but not afterwards. condition: sensitive_files and open_read and server_procs and not proc_is_new and proc.name!="sshd" and not user_known_read_sensitive_files_activities output: > Sensitive file opened for reading by trusted program after startup (user=%user.name user_loginuid=%user.loginuid command=%proc.cmdline parent=%proc.pname file=%fd.name parent=%proc.pname gparent=%proc.aname container_id=%container.id image=%container.image.repository) priority: WARNING tags: [filesystem, mitre_credential_access]
- Updating Packages
- rule: Launch Package Management Process in Container desc: Package management process ran inside container condition: > spawned_process and container and user.name != "_apt" and package_mgmt_procs and not package_mgmt_ancestor_procs and not user_known_package_manager_in_container output: > Package management process launched in container (user=%user.name user_loginuid=%user.loginuid command=%proc.cmdline container_id=%container.id container_name=%container.name image=%container.image.repository:%container.image.tag) priority: ERROR tags: [process, mitre_persistence]
As you can see, using Falco, you can gain complete visibility of your container and application behavior. It’s easy to install and open-source.