I have write three posts about the CNI and you can found the other posts below
[CNI] Bridge Network In Docker
[CNI] Container Network Interface Introduction

In this post, I will show how to write your own CNI program.

Information

• Time: 2018/02/10
• Meetup: Cloud Native Taiwan User Group #2

Abstraction

This slide first introduces the limitation of the current Linux network stack, including how the kernel receives packets from the network interface card.

And then it introduces two high perforamce network mechanism, the DPDK and RDMA. the concept of those mechanisms and also how do they work.

In the end, introduce a paper about how to use the RDMA to accelerate the training time in the Tensorflow framework.

Information

• Time: 2018/05/26
• Meetup: Cloud Native Taiwan User Group #5

Abstraction

This slides introduce a basic concept of how docker pulls images from docker hub and private registry first and then assumes a simple scenario which do the CI/CD for applications in the kubernetes cluster.

In that scenario, you will use a kubernetes pod to build your application into a docker image and push that to the private registry and then pull that image to run as another kubernetes Pod in kubernetes cluster.

The slides shows every problems you will meet, including how do you deploy your private registry, the network connectivity between the k8s Node/k8s Pod and private registry and a trusted SSL certificate.

In the last section, the author provides a simple solution for that scenario and use a simple graph to explain how to works.

I have write three posts about the CNI and you can found the other posts below
[CNI] Bridge Network In Docker
[CNI] Write A CNI Plugin By Golang

In this post, I will try to introduce the concept of Container Network Interface (CNI), including why we need this, how it works and what does it do.

If you have not familiar with what is linux network namespace and how docker handles the network for its containers.
You should read the [CNI] Bridge Network In Docker to learn those concepts and that will be helpful for this tutorial.

I have write three posts about the CNI and you can found the other posts below
[CNI] Container Network Interface Introduction
[CNI] Write A CNI Plugin By Golang

If you have any experience about setuping a kubernetes cluster before, you must notice that you need to choose one CNI in your kubernetes cluster, and there’re many candidate that you can choose, including the flannel, weave, calico and so on.

Most of the kubernetes users and operators don’t know what is the different between those CNI plgins and the only thing they care is that the CNI can make the network works well.

So, I will introduce the Container Network Interface (CNI) in the following articles.

• First, I will explain what is the bridge network in docekr and hot it works. Besides, I also introduce the Linux Network Namespace (ns) and use the Linux Network Namespace to create a simple environment.
• Second, We have the basic knowhow about network namespace and we can start to learn what is CNI, why we need the CNI and how CNI works. we also use the simple CNI to demostrate how CNI works with network namespace.
• Third, We have learned what is the CNI before, and we will start to implement our own CNI which is a simple CNI just like the bridge network (the default network of docker). That article will be a tutorial about how to write a CNI in golang

Introduction

2017 SICCOM 上面出現了一篇令人感到有趣的論文，內容是 Towards Zero Copy Dataflows using RDMA，可以到 這邊 閱讀該篇論文內容。

再看一篇論文前，最重要的就是其摘要，根據其摘要我們可以快速的對該篇論文有一個基本的認識，所以先來看一下該篇論文的摘要

ABSTRACT

Remote Direct Memory Access (RDMA) offers ultra-low latency and
CPU bypass networking to application programmers. Existing applications
are often designed around socket based software stack that
manages application buffers separately from networking buffers
and do memory copies between them when sending/receiving data.

先大概介紹了一下 RDMA 的特性與價值，想要瞭解更多關於 RDMA 的可以參考這篇文章。

在這邊文章前，必須要先知道什麼是 Jupyter Notebook，這方面網路上已經有太多的文章在介紹了，所以這邊就簡單介紹就好。

Jupyter Notebook 是一個介於 IDE 與編輯器之間的工具，可以讓使用者一行一行的寫程式並且直接運行觀察其結果，除了大家都熟知的 Python/R 等直譯式語言外，現在連 C++ 這種編譯式語言都可以執行了，非常的令人驚艷，想要看更多關於 C++ 支援的可以參考這篇文章

這篇文章的主軸在於如何透過程式化的方式將已知的 Jupyter Notebook 給轉換成一般常用的 Python 腳本。

CORD-Trellis Example

Trellis 是 ONF 於 2017 年推出的 Network Architecture Solution，整個架構就於 ONOS SDN Controller 以及 Openflow-Enable switch.

本篇文章主要是在於如何透過 ONF 提供個環境快速搭建一個 Trellis 的測試環境。

本次的論文發表於 SIGCOMM 2017, 可以在這邊找到網路上的文章。於 SIGCOMM 2017中，其所屬於的 section 是 Programmable Devices，所以可以料想到本文內容會跟 Programmable Devices 有些關係。果不其然的內容中有提到 P4, NetFPGA，處此之外，本篇內容也提到了 DPDK, Clos Network, MPTCP, DCTCP, incast 等，必須說本篇文章包含了內容實在廣泛，所以要花不少時間去釐清每個元件，才可以了解到本篇論文的內容。

由於本篇論文長達 14 頁,本來也有想過作了簡單的總結就好，大概講述一下想要做到什麼事情，怎麼做到之類的就好。
但是後來想一想，今天會有新東西誕生，勢必就是要解決舊東西的限制。
如果在學習之於，不但能夠學習新知識，更可以強化自己對於舊知識的概念，何樂而不為?

因此本篇一開始就先針對本文做一個簡單的介紹，直接瞭解到本論文希望做什麼，達到什麼目的，為什麼要這樣做。
有興趣的人可以在慢慢往下翻，畢竟內容實在有夠長..XD

簡介

本論文所發生的場景是在 Data Center內，不適用於一般的網際網路，主要是因為網際網路充滿太多未知性與無法掌控的裝置，譬如每條 Link 的頻寬, Switch 的設定等。有這些未知的情況下，無法設計一個好的運作模式來滿足 Low Latency 與 High Throughut 的需求。因此環境都只考慮 Data Center。

本論文提出了 NDP (New Datacenter Protocol Architecture) 這個概念，其具體目標，評比對象以及實作方法如下。

本文希望能夠達到的需求有

1. 對於 short-flow 能夠有盡量低的 latency
2. 對於 long-flow 能夠有盡量高的 throughput
3. 對於一個高度負載的網路中，能夠充分利用整個網路拓墣上的 Link 來傳輸封包 (high network capacity)
4. 能夠將 incast 情況對網路造成的影響降到最小

本文評比對象有

1. DCTCP (Data Center TCP)
2. MPTCP (Multi Path TCP)
3. DCQCN (Data Center QCN)

   DCQCN is a congestion control protocol for large scale RDMA networks, developed jointly by Microsoft and Mellanox.

本文實作方法

1. 透過 DPDK 打造 NDP Application (拋棄 TCP, 重新打造 L4 Protocol)
2. 於 NetFPGA SUME上面實作 NDP 相關功能，作為一個 Hardware Switch
3. 於 P4上面實作 NDP 相關功能，作為一個 Software Switch

看到這邊如果對於上述的內容感到興趣的話，就可以開始往下看了!

此文章主要接續前篇文章 OVS + DPDK + Docker 共同玩耍 進行後續探討。

根據 Jalen Lin 提出的一篇文章 Linux kernel bug delivers corrupt TCP/IP data to Mesos, Kubernetes, Docker containers，其中的原因似乎可以用來解釋此問題。

因此本篇文章會分成兩個部分，第一部份是先針對上述文章進行探討，第二部分則是將第一部分的結果與先前經驗去結合，來追出更深層的問題所在。