Go performance optimize May 06, 2016

**Go性能优化技巧(By 雨痕)

  1. 字符串(string)作为一种不可变类型,在与字节数组(slice, [ ]byte)转换时需付出 “沉重” 代价,根本原因是对底层字节数组的复制。
package main

import (
    "fmt"
    "unsafe"
)

func str2bytes(s string) []byte {
    ptr := (*[2]uintptr)(unsafe.Pointer(&s))
    btr := [3]uintptr{ptr[0], ptr[1], ptr[1]}
    return *(*[]byte)(unsafe.Pointer(&btr))
}

func bytes2str(b []byte) string {
    return *(*string)(unsafe.Pointer(&b))
}

func main() {
    s := "abcdefghi"
    b := str2bytes(s)
    s2 := bytes2str(b)

    fmt.Println(s, b, s2)
}
  1. Go Proverbs: A little copying is better than a little dependency. 对于一些短小的对象,复制成本远小于在堆上分配和回收操作。

    ...
The Rise of Cloud Computing Systems - Jeff Dean May 05, 2016

{% pdf http://feiskyer.github.io/assets/ccs.pdf %}

Reading notes of week 17 Apr 29, 2016

SIG-Networking: Kubernetes Network Policy APIs Coming in 1.3

One problem many users have is that the open access network policy of Kubernetes is not suitable for applications that need more precise control over the traffic that accesses a pod or service. Today, this could be a multi-tier application where traffic is only allowed from a tier’s neighbor. But as new Cloud Native applications are built by composing microservices, the ability to control traffic as it flows among these services becomes even more critical.

...
runc and runV Apr 28, 2016

runc is a CLI tool for spawning and running containers according to the OCI specification, while runV is a hypervisor-based runtime for OCI. Both of them are recommanded (implementations](https://github.com/opencontainers/runtime-spec/blob/master/implementations.md) of OCI.

Playing with runc

Install runc:

yum install -y libseccomp-devel
mkdir -p $GOPATH/src/github.com/opencontainers
cd $GOPATH/src/github.com/opencontainers
git clone https://github.com/opencontainers/runc
cd runc
make
sudo make install

Run busybox:

$ docker pull busybox
$ mkdir rootfs
$ docker export $(docker create busybox) | tar -C rootfs -xvf -
$ runc spec .
$ runc start test
/ # ps
PID   USER     COMMAND
1 root     sh
9 root     ps

Playing with docker-containerd

docker-containerd is installed togather with docker 1.11.

...
Container runtime in Docker v1.11 Apr 28, 2016

Docker v1.11正式集成了runc(终于支持OCI了),并将原来的一个二进制文件拆分为多个,同时还保持docker CLI和API不变:

  • docker
  • docker-containerd
  • docker-containerd-shim
  • docker-runc
  • docker-containerd-ctr

...
DPDK Introduction Apr 24, 2016

DPDK Introduction

Intel DPDK全称Intel Data Plane Development Kit,是intel提供的数据平面开发工具集,为Intel architecture(IA)处理器架构下用户空间高效的数据包处理提供库函数和驱动的支持,它不同于Linux系统以通用性设计为目的,而是专注于网络应用中数据包的高性能处理。DPDK应用程序是运行在用户空间上利用自身提供的数据平面库来收发数据包,绕过了Linux内核协议栈对数据包处理过程。Linux内核将DPDK应用程序看作是一个普通的用户态进程,包括它的编译、连接和加载方式和普通程序没有什么两样。DPDK程序启动后只能有一个主线程,然后创建一些子线程并绑定到指定CPU核心上运行。

...
Tips for cgo Apr 24, 2016

cgo的一些tips

基本类型

The standard C numeric types are available under the names C.char, C.schar (signed char), C.uchar (unsigned char), C.short, C.ushort (unsigned short), C.int, C.uint (unsigned int), C.long, C.ulong (unsigned long), C.longlong (long long), C.ulonglong (unsigned long long), C.float, C.double, C.complexfloat (complex float), and C.complexdouble (complex double). The C type void* is represented by Go’s unsafe.Pointer. The C types __int128_t and __uint128_t are represented by [16]byte.

...
cgo in go 1.6 Apr 19, 2016

The major change is the definition of rules for sharing Go pointers with C code, to ensure that such C code can coexist with Go’s garbage collector. Briefly, Go and C may share memory allocated by Go when a pointer to that memory is passed to C as part of a cgo call, provided that the memory itself contains no pointers to Go-allocated memory, and provided that C does not retain the pointer after the call returns. These rules are checked by the runtime during program execution: if the runtime detects a violation, it prints a diagnosis and crashes the program. The checks can be disabled by setting the environment variable GODEBUG=cgocheck=0, but note that the vast majority of code identified by the checks is subtly incompatible with garbage collection in one way or another. Disabling the checks will typically only lead to more mysterious failure modes. Fixing the code in question should be strongly preferred over turning off the checks.

...
Upgrade CentOS kernel Mar 30, 2016

终于耐不住要升级下kernel了,目前epel提供两个版本: kernel-lt (4.4)和kernel-ml (4.5):

  • The kernel-ml packages are built from the sources available from the “mainline stable” branch of The Linux Kernel Archives (external link). The kernel configuration is based upon the default RHEL-7 configuration with added functionality enabled as appropriate. The packages are intentionally named kernel-ml so as not to conflict with the RHEL-7 kernels and, as such, they may be installed and updated alongside the regular kernel.
  • The kernel-lt packages are built from the sources available from The Linux Kernel Archives (external link), just like the kernel-ml packages. The difference is that kernel-lt is based on a “long term support” branch and kernel-ml is based on the “mainline stable” branch.

升级到lt的步骤很简单:

...
Borg, Omega, and Kubernetes (ACM Queue) Mar 04, 2016

Brendan Burns, Brian Grant等在Borg, Omega, and Kubernetes - Lessons learned from three container-management systems over a decade分享了Google在容器管理的经验教训。

在谷歌的历史上,开发了三种容器管理调度系统:

...