Network Policy and Istio: Deep Dive

Network Policy and Istio: Deep Dive

 

Today, we announced our collaboration with the Kubernetes networking community on an exciting new project, Istio. In that blog post, we included a video showing how the Istio service mesh could be deployed along with Kubernetes network policies (implemented by Project Calico) to deliver a maximally secure application infrastructure. In this blog, we will look in a bit more depth at how you can start to build such policies yourself, to get the best of both worlds: authenticated, encrypted service-to-service communication running on a fully secured underlying network layer.

Istio

Istio is an open platform to connect, manage, and secure microservices. Istio provides an easy way to create a network of deployed services with load balancing, service-to-service authentication, monitoring, and more, without requiring any changes in service code. The project includes a demonstration application called bookinfo https://istio.io/docs/samples/bookinfo.html to show how a microservices based application may be deployed in a Kubernetes cluster using Istio – and we will use that application as the basis of our example.

Calico

Calico enables networking and network policy in Kubernetes clusters across the cloud. Calico uses a pure IP networking fabric to provide high performance networking, and its battle-tested policy engine enforces high-level, intent-focused network policy. Together, Calico and Kubernetes provide a secure, cloud-native platform that can scale your infrastructure to hundreds of thousands of workloads.

Calico integrates with Kubernetes using CNI and can be used to enforce security policies that are defined in Kubernetes via the Network Policy API.

Securing a Microservices Application

Istio can be used to define and build a mesh of micro services that together compose an application. It can define complex routing (L5-L7) policies between the micro services. For now there is no way to also define the permissible set of intercommunications policies at the layer-3/4 that can be used to secure access to the micro-services application – so to enable Istio to work, you need to turn off network policy enforcement on all nodes. This leaves an undesirably large attack surface to disrupt the application. For example, without network policies in place, it is possible to flood some of the backend micro services-ratings in the case of bookinfo-with spurious requests that can disrupt genuine requests from other micro services.

The rest of this section will show how we can define Kubernetes Network Policies to secure the bookinfo application.

Assuming you have a cluster with the bookinfo application https://istio.io/docs/samples/bookinfo.html  already running, you can observe that the productpage, ratings, reviews and details micro services are running.

# kubectl get svc -o wide

NAME            CLUSTER-IP       EXTERNAL-IP   PORT(S)                       AGE       SELECTOR

details         10.108.133.76    <none>        9080/TCP                      3d        app=details

istio-egress    10.99.78.230     <none>        80/TCP                        3d        istio=egress

istio-ingress   10.105.247.10    <pending>     80:32012/TCP,443:30951/TCP    3d        istio=ingress

istio-manager   10.102.182.17    <none>        8080/TCP,8081/TCP             3d        istio=manager

istio-mixer     10.109.218.37    <none>        9091/TCP,9094/TCP,42422/TCP   3d        istio=mixer

kubernetes      10.96.0.1        <none>        443/TCP                       5d        <none>

productpage     10.110.3.182     <none>        9080/TCP                      3d        app=productpage

ratings         10.110.48.206    <none>        9080/TCP                      3d        app=ratings

reviews         10.111.201.116   <none>        9080/TCP                      3d        app=reviews

sleep           10.99.116.199    <none>        80/TCP                        3d        app=sleep

Setup a default deny policy

The first step in securing any application is defining a default-deny policy for the namespace where the application is running. In this case it will be the namespace ‘default’

# kubectl annotate ns default "net.beta.kubernetes.io/network-policy={\"ingress\":{\"isolation\":\"DefaultDeny\"}}"

Network Policies For Inter-MicroService Communication

Now that we have a default deny policy in place we need to enable communication between the required microservices. The intercommunication path between the services are:

  1. Istio-ingress to Productpage.
  2. ProductPage to all versions (v1, v2 and v3) of Reviews service.
  3. ProductPage to Details service
  4. And all versions of Review to Ratings service.

Let’s go ahead and specify each individual network policy

Istio-Ingress to Productpage:

#kubectl create -f - << EOF

apiVersion: extensions/v1beta1

kind: NetworkPolicy

metadata:

  name: calico-from-ingress-to-ppage

  namespace: default

spec:

  ingress:

  - from:

    - podSelector:

        matchLabels:

          istio: ingress

  podSelector:

    matchLabels:

      app: productpage

EOF

 

Productpage to All Versions of Reviews:

#kubectl create -f - << EOF

apiVersion: extensions/v1beta1

kind: NetworkPolicy

metadata:

  name: calico-from-productpage-to-reviews

  namespace: default

spec:

  ingress:

  - from:

    - podSelector:

        matchLabels:

          app: productpage

          version: v1

  podSelector:

    matchLabels:

      app: reviews

    matchExpressions:

      - {key: version, operator: In, values: [v1, v2, v3]}

EOF

A point to notice here is that we have setup network policy to permit product page to access all versions of the reviews service. Based off how the Istio’s route-rules are setup we can also restrict which specific versions of a reviews service the network policy should be permitted to access.

Productpage to Details service:

#kubectl create -f - << EOF

apiVersion: extensions/v1beta1

kind: NetworkPolicy

metadata:

  name: calico-from-productpagev1-to-detailsv1

  namespace: default

spec:

  ingress:

  - from:

    - podSelector:

        matchLabels:

          app: productpage

          version: v1

  podSelector:

    matchLabels:

      app: details

      version: v1

EOF

Reviews to Ratings service:

#kubectl create -f - << EOF

apiVersion: extensions/v1beta1

kind: NetworkPolicy

metadata:

  name: calico-from-reviews-to-ratings

  namespace: default

spec:

  ingress:

  - from:

    - podSelector:

        matchLabels:

          app: reviews

  podSelector:

    matchLabels:

      app: ratings

EOF

Again, here we have defined a network policy that is quite liberal as it permits all versions of the reviews service to access the ratings service.

With all these network policies in place the attack surface to the bookinfo application is significantly reduced.

Further Work

As you can appreciate, the network policies required to secure a micro services application can be quite complex and it is easy to mis-configure the network policies and that may lead to security holes. But the network policies are highly correlated with the route-rules that are defined for intercommunication between the micro services. We at Calico are working closely with the Istio project to automate the generation and insertion of network policies that are derived from the route-rules that define the micro services application.

Also, the policies shown here have the following limitations:

  1. Only control the ingress path to a service. Using calicoctl commands it is possible to configure network policies that can control the egress paths.
  2. Do not limit ports that may intercommunicate. Using calicoctl commands it is possible to configure network policies that can be used to restrict inter-communication to specific ports.
  3. No interactions with external services. In the real world, there are often legacy services outside the cluster that must be included in the application mesh. With Calico it is possible to configure rules to allow access to/from those services, and even deploy an agent on VMs and standalone servers to enforce bookended policies on the legacy application node.

Examples of using calicoctl commands to configure such policies can be found at http://docs.projectcalico.org/v2.2/getting-started/kubernetes/tutorials/advanced-policy

Avatar
Saurabh is a senior software engineer working at Tigera. He is an expert in networking of both the software-defined and traditional varieties, having held senior engineering roles at CPlane Networks, Vyatta/Brocade, and Cisco.