Security Bulletins

Security Bulletins

From time to time, we might release security bulletins related to ProjectCalico. All security bulletins for ProjectCalico are described here.

 

Description Severity Notes

Tigera Technical Advisory TTA-2019-003

Title: CVE-2019-11253 aka Billion Laughs Vulnerability Mitigations
Date published: 2019-Dec-2

HIGH N/A

Summary

Calico Enterprise API Server, and Calicoctl, if accessed by an authenticated user with network-admin role, are vulnerable to denial of service attacks due to weaknesses in included version of goyaml, which is used by the affected components. This could negatively affect other workloads on the same host, or cause the components to be evicted or restarted due to the excess consumption of resources. A successful denial of service attack against Calico Enterprise API Server or Calicoctl could potentially cause denial of service for an entire cluster.

Note that even though CVE-2019-11253 has been patched/resolved by Kubernetes API Server and Kubectl, the vulnerable Calico Enterprise API Server is still reachable.

Severity

HIGH

Affected Releases

Calico (only Calicoctl)

  • 3.10 and older

Calico Enterprise (previously known as Tigera Secure)

  • 2.6.0 and older

Indicators of Impact/Compromise

Calico Enterprise components allocate a large amount of memory and/or crash unexpectedly

Workaround / Remediation

Calicoctl
Review all YAML files to be processed by Calicoctl. If Calicoctl is run within a Kubernetes cluster as a pod, set usage limits to prevent the pod from consuming excessive resources. If Calicoctl is run as a binary on a host, terminate the process if it takes longer than 10 seconds.

API Server
The Calico Enterprise API Server requires the authenticated user to be of network-admin or above in order for the API Server to process the input YAML. You can mitigate this by:

  • Ensuring only intended user has access the Calico Enterprise Management UI
  • Review all existing ClusterRole/Role to ensure only intended user can access Tigera Secure EE API endpoints

Fixed Software

The following patched versions of software will fix the underlying issue. We expect these to be available shortly.

Calico

  • 3.11

Calico Enterprise

  • 2.6.1

 

Description Severity Notes

Tigera Technical Advisory TTA-2019-002

Title: Fixes available for vulnerability in VXLAN and IPIP overlay modes
Date published: 2019-July-1

HIGH N/A

Summary

Clusters using VXLAN or IPIP overlays are vulnerable to malicious cluster users being able to potentially violate network policies or spoof their IP addresses. Upgrade to latest Calico or Tigera Secure releases to close this vulnerability.

Severity

HIGH

Affects clusters that use VXLAN or IPIP overlays, including Calico, Tigera Secure, and Canal (Calico policy on Flannel). Clusters which use neither of these, or use Calico policy in conjunction with a cloud-provider CNI (e.g. aws-vpc-cni) are not affected.

Allows a malicious cluster user with permission to create pods to potentially bypass Egress policies assigned to their pod (workload endpoint), or potentially spoof IP addresses and make it look like sent packets are from some other pod (workload endpoint). This could potentially allow a malicious user to bypass Ingress policy and send unauthorized packets to other pods (workload endpoints) in the cluster.

Furthermore in IPIP mode, the vulnerability might allow an attacker to prevent other workloads in the cluster from communicating, or redirect traffic intended to other workloads to itself.

Affected Releases

Calico

            • 3.7.3 and older
            • 3.6.3 and older
            • 3.5.6 and older
            • 3.4.x
            • 3.3.x
            • 3.2.x
            • 3.1.x
            • 3.0.x
            • 2.x
            • 1.x

Calico for Windows

            • 3.7.0

Tigera Secure Enterprise Edition

            • 2.4.1 and older
            • 2.3.2 and older
            • 2.2.x
            • 2.1.x
            • 2.0.x
            • 1.x

Tigera Secure Cloud Edition: not affected

Indicators of Impact/Compromise

In VXLAN mode:

Conntrack entries or flow logs from a pod IP to a node IP on UDP port 4789.

In IPIP mode:

Contrack entries or flow logs from a pod IP to a node IP using protocol 4 (IPIP).

Workaround / Remediation

In order to exploit the vulnerability, a malicious pod must transmit packets to node IP addresses. You can prevent this by using an Egress policy. Do this if you cannot upgrade immediately to a fixed version.

If you are using VXLAN, deny egress to destination UDP port 4789.

If you are using IPIP, deny egress on protocol 4, which is IPIP encapsulation.

Note that if you apply an egress policy to pods, this sets the default action to deny, so be sure to whitelist any legitimate traffic from your applications. In Tigera Secure Enterprise Edition, you can use a high-order tier to deny problematic traffic without affecting the default action in lower-order tiers (e.g. the default tier).

Fixed Software

Calico

            • 3.8.0 (Available on July 2, 2019)
            • 3.7.4
            • 3.6.4
            • 3.5.7

Tigera Secure Enterprise Edition

            • 2.4.2
            • 2.3.3

 

Description Severity Notes

Tigera Technical Advisory TTA-2019-001

Title: Fixes available for CVE-2019-9946
Date published: 2019-March-28

Medium N/A

Summary

A vulnerability has been discovered in the “portmap” CNI plugin affecting versions prior to 0.7.5 of the portmap plugin. Portmap is a kubernetes component but is bundled in Tigera Calico & Tigera Secure releases. We also recommend you update to a patched version of Kubernetes to ensure the vulnerable plugin is not in use.

Prior to the fix, HostPort rules would take precedence over more specific services like NodePorts, potentially delivering traffic to the wrong location.

Affected Releases

            • All Tigera Secure Enterprise Edition releases up to and including the following patch releases in their respective minor release streams: v2.3.0, v2.2.3, v2.1.1.
            • All Calico releases up to and including the following patch releases in their respective minor release streams: v3.2.6, v3.3.5, v3.4.3, v3.5.3, v3.6.0.
            • All Calico releases in release streams v3.1.x and earlier

Tigera Secure Cloud Edition is not directly affected, but operators are strongly cautioned to consider whether their Kubernetes platform is affected and update to a fixed version of Kubernetes.

Indicators of Impact/Compromise

Audit your Kubernetes cluster for any pods with hostPorts that conflict numerically with any nodePorts. Conflicts indicate that some traffic may be being misdirected.

Workaround / Remediation

We recommend that all users upgrade to a fixed version of Tigera Calico/Tigera Secure and a fixed version of Kubernetes.

If you cannot update to a fixed version, you can manually install a fixed portmap binary on each Kubernetes node. You will also need to configure Tigera Calico/Tigera Secure not to install the portmap binary by setting the environment variable SKIP_CNI_BINARIES=”portmap” on the install-cni container.

If neither upgrading nor manually installing a fixed portmap binary is possible, you can proactively audit your Kubernetes cluster to ensure that pods are not being created with hostPorts that numerically conflict with nodePorts, and immediately remediate any pod hostPorts that conflict.

Fixed Software

This issue has been fixed in the following patches:

Tigera Software

            • Calico v3.2.7
            • Calico v3.3.6
            • Calico v3.4.4
            • Calico v3.5.4
            • Calico v3.6.1
            • Tigera Secure Enterprise Edition 2.3.1

Kubernetes Software

            • 1.11.9
            • 1.12.7
            • 1.13.5
            • 1.14.0

External Reference

https://discuss.kubernetes.io/t/announce-security-release-of-kubernetes-affecting-certain-network-configurations-with-cni-releases-1-11-9-1-12-7-1-13-5-and-1-14-0-cve-2019-9946/5713

 

Description Severity Notes

Tigera Technical Advisory TTA-2018-001

Title: Calico CNI Logging can expose Kubernetes service account tokens
Date published: 2018-Nov-13

Medium N/A

Summary

In certain scenarios, Calico will write configuration data in log files including service account tokens included in the configuration. This will expose client service account tokens in the log files which could lead to unauthorized cluster access or privilege escalation if users are able to access these log files. This advisory is referenced under Google Security Bulletins on November 13, 2018.

Affected Releases

  • All Calico releases
    • Use of Calico on orchestrators other than Kubernetes (e.g. DC/OS) are not affected
  • Tigera Secure Cloud Edition 1.0
  • Tigera Secure Enterprise Edition 2.2, 2.1

Indicators of Impact/Compromise

Check your CNI configuration file; if the field “k8s_auth_token” has been set (which it is not by default in standard Calico manifests), then that token could have been written as an info level log. Otherwise, the exposure is limited to debug level logs. You can check if any service account tokens have been exposed by searching for the following log lines in the kubelet logs:

  • “Updated stdin data”
  • “Kubernetes config”
  • “Configured environment”
  • “Building client for config”
  • “Config overrides”

An example using journalctl:

journalctl -u kubelet --no-pager | grep "Updated stdin data"

You can check if any service account tokens have been exposed in the calico/node logs by searching for the following log lines in the calico/node logs, for example:

kubectl logs -n kube-system -l k8s-app=calico-node -c calico-node | grep "Building client for config"
kubectl logs -n kube-system -l k8s-app=calico-node -c calico-node | grep "Config overrides"

If any of the above queries contain a Kubernetes service account token, then the cluster is affected by this vulnerability.

Audit all token usage and confirm that there is no unauthorized use of client tokens in your cluster.

Service Account Tokens generated by Kubernetes are valid so long as the token secret exists in Kubernetes. Regenerate tokens to ensure any leaked tokens no longer pose a threat.

Workaround / Remediation

The following workaround steps should be taken if you are running a version of Calico that includes this vulnerability.

            1. Turn off debug logging in Calico. In the kube-system namespace:
              1. Modify the calico-config ConfigMap to set log_level to “warning” or higher.
                kubectl edit configmap -n kube-system calico-config
              2. Modify the calico-node container in the calico-node daemonset to have environment variable FELIX_LOGSEVERITYSCREEN set to “info” or higher.
                kubectl patch ds -n kube-system calico-node --patch \ '{"spec": {"template": {"spec": {"containers": [{"name": "calico-node", "env": [{"name": "FELIX_LOGSEVERITYSCREEN", "value": "warning"}]}]}}}}’
            2. Delete the calico-node service account secret. In the kube-system namespace:
              1. Find the correct secret:
                kubectl get secrets -n kube-system | grep calico-node
              2. Delete the secret:
                kubectl delete secret -n kube-system <SECRET NAME>
              3. A new secret will be automatically generated by the token controller.
            3. Restart all calico-node pods in the kube-system namespace to pick up the new configuration.

After upgrading to a fixed version of the software, you can reset logging to whichever level you prefer.

Fixed Software

This issue has been fixed in the following patches:

            • Calico v3.3.1
            • Calico v3.2.4
            • Calico v3.1.4
            • Calico v3.0.9
            • Calico v2.6.12
            • Tigera Secure CE v1.0.1
            • Tigera Secure EE v2.2.1

If you upgrade from an earlier version to a fixed version, if you have not already done so, you should follow steps (2) and (3) under “Workaround / Remediation” above to ensure any leaked secret is no longer valid.