Sign in Subscribe

parsec spire K8s

BryanZhang

• 8 min read

enable nested virtualbox

https://ostechnix.com/how-to-enable-nested-virtualization-in-virtualbox/

minikube start --memory=1800 --cpus=2 --disk-size=2000M
--v=8
--vm-driver=virtualbox
--docker-env http_proxy=http://192.168.1.102:8118 --docker-env https_proxy=http://192.168.1.102:8118
--docker-env NO_PROXY=localhost,127.0.0.1,192.168.99.115
--extra-config=apiserver.service-account-signing-key-file=/var/lib/minikube/certs/sa.key
--extra-config=apiserver.service-account-key-file=/var/lib/minikube/certs/sa.pub
--extra-config=apiserver.service-account-issuer=api
--extra-config=apiserver.service-account-api-audiences=api,spire-server
--extra-config=apiserver.authorization-mode=Node,RBAC
--extra-config=kubelet.authentication-token-webhook=true

      imagePullPolicy: IfNotPresent
        periodSeconds: 600
        timeoutSeconds: 30

 minikube start \
     --docker-env HTTP_PROXY=http://10.38.32.9:10808/ \
     --docker-env HTTPS_PROXY=http://10.38.32.9:10808/ \
     --docker-env NO_PROXY=index.docker.io,\
 registry.hub.docker.com,\
 registry-1.docker.io,\
 registry.docker-cn.com,\
 registry-mirror-cache-cn.oss-cn-shanghai.aliyuncs.com,\
 192.168.99.100\
     --registry-mirror https://registry.docker-cn.com

export NO_PROXY=localhost,127.0.0.1,10.96.0.0/12,192.168.99.0/24,192.168.39.0/24
Proxies and VPNs
How to use minikube with a VPN or HTTP/HTTPS Proxy
minikube

--register-mirror=registry.cn-hangzhou.aliyuncs.com/google_containers does not work for me. And --image-mirror-country=cn too.

But --docker-env HTTP_PROXY=http://10.0.2.2:35033 --docker-env HTTPS_PROXY=http://10.0.2.2:35033 works for me.

It is really strange that I have proxy configured. i can ping through google.com. But can not pull images directly.

minikube start --vm-driver=virtualbox --registry-mirror=https://registry.docker-cn.com --image-mirror-country=cn --image-repository=registry.cn-hangzhou.aliyuncs.com/google_containers

  • NO_PROXY - A comma-separated list of hosts which should not go through the proxy.

The NO_PROXY variable here is important: Without setting it, minikube may not be able to access resources within the VM. minikube uses two IP ranges, which should not go through the proxy:

  • 192.168.99.0/24: Used by the minikube VM. Configurable for some hypervisors via --host-only-cidr
  • 192.168.39.0/24: Used by the minikube kvm2 driver.
  • 192.168.49.0/24: Used by the minikube docker driver’s first cluster.
  • 10.96.0.0/12: Used by service cluster IP’s. Configurable via --service-cluster-ip-range
--docker-env NO_PROXY=192.168.99.0/24

install  k8s

使用minikube在Ubuntu上搭建一个单机版的k8s
https://www.huaweicloud.com/articles/05059d30eb656040ad6d27c44afb693f.html

https://www.cnblogs.com/z-books/p/14191232.html

install docker

https://lequ7.com/ubuntu2004lts-an-zhuang-dockerqbit.html

更新索引

sudo apt update

安装依赖包

sudo apt install apt-transport-https   ca-certificates    curl     gnupg-agent     software-properties-common

curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -

if error happens

apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys D8576A8BA88D21E9

sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 76F1A20FF987672F

gpg: 找不到可写的钥匙环：eof

https://www.cnblogs.com/jack123/p/12956016.html

添加仓库

sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 7EA0A9C3F273FCD8

确认 docker 版本

sudo apt update
apt-cache policy docker-ce

install docker

sudo apt install docker-ce docker-ce-cli containerd.io

免 sudo 运行
添加并切换用户组

添加用户组

$USER 可以换为特定用户名

sudo usermod -aG docker $USER

切换用户组

newgrp docker
测试
docker run hello-world

2.下载minikube的修改版的二进制：

axel -n 10 http://kubernetes.oss-cn-hangzhou.aliyuncs.com/minikube/releases/v1.2.0/minikube-linux-amd64
3.让minikube的二进制可执行：

chmod +x minikube
4.将minikube的可执行文件放到/usr/local/bin/，类似于之前的kubectl

sudo mv minikube /usr/local/bin/

#############
###################
国内Ubuntu20.04下安装kubectl  #和helm
https://my.oschina.net/ykbj/blog/4315903

更新系统

sudo apt-get update && sudo apt-get install -y apt-transport-https

加入国内kubernetes-xenial源

echo "deb http://mirrors.ustc.edu.cn/kubernetes/apt kubernetes-xenial main" | sudo tee -a /etc/apt/sources.list.d/kubernetes.list

更新

sudo apt-get update

W: GPG 错误：http://mirrors.ustc.edu.cn/kubernetes/apt kubernetes-xenial InRelease: 由于没有公钥，无法验证下列签名： NO_PUBKEY 6A030B21BA07F4FB NO_PUBKEY 8B57C5C2836F4BEB

sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys +上面报错的密钥

sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 6A030B21BA07F4FB
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 8B57C5C2836F4BEB

安装kubectl

sudo apt-get install -y kubectl

build spire

make V=1 (0,1=verbose,2)

run spire server

https://spiffe.io/docs/latest/spire/installing/getting-started-linux-macos-x/
bin/spire-server run -config conf/server/server.conf &
bin/spire-server healthcheck
bin/spire-server token generate -spiffeID spiffe://example.org/myagent

bin/spire-agent run -config conf/agent/agent.conf -joinToken <token_string> &

bin/spire-agent run -config conf/agent/agent.conf -joinToken 6e8c4d05-e19b-429b-b8e4-eb24be3c6973 &
bin/spire-agent healthcheck

create registration polocy

on the current user’s UID ($(id -u))

bin/spire-server entry create -parentID spiffe://example.org/myagent \

-spiffeID spiffe://example.org/myservice -selector unix:uid:$(id -u)

bin/spire-server entry create -parentID spiffe://example.org/myagent
-spiffeID spiffe://example.org/myservice -selector unix:uid:1000

Retrieve and view a x509-SVID

bin/spire-agent api fetch x509 -write /tmp/

You can use the openssl command to view the contents of the SVID:

openssl x509 -in /tmp/svid.0.pem -text -noout

spire + k8s

https://spiffe.io/docs/latest/spire/installing/getting-started-k8s/

enable nested virtualbox

https://ostechnix.com/how-to-enable-nested-virtualization-in-virtualbox/

install  k8s

使用minikube在Ubuntu上搭建一个单机版的k8s
https://www.huaweicloud.com/articles/05059d30eb656040ad6d27c44afb693f.html

https://www.cnblogs.com/z-books/p/14191232.html

install docker

https://lequ7.com/ubuntu2004lts-an-zhuang-dockerqbit.html

更新索引

sudo apt update

安装依赖包

sudo apt install apt-transport-https   ca-certificates    curl     gnupg-agent     software-properties-common

curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -

if error happens

apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys D8576A8BA88D21E9

sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 76F1A20FF987672F

gpg: 找不到可写的钥匙环：eof

https://www.cnblogs.com/jack123/p/12956016.html

添加仓库

sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 7EA0A9C3F273FCD8

确认 docker 版本

sudo apt update
apt-cache policy docker-ce

install docker

sudo apt install docker-ce docker-ce-cli containerd.io

免 sudo 运行
添加并切换用户组

添加用户组

$USER 可以换为特定用户名

sudo usermod -aG docker $USER

切换用户组

newgrp docker
测试
docker run hello-world

2.下载minikube的修改版的二进制：

axel -n 10 http://kubernetes.oss-cn-hangzhou.aliyuncs.com/minikube/releases/v1.2.0/minikube-linux-amd64
3.让minikube的二进制可执行：

chmod +x minikube
4.将minikube的可执行文件放到/usr/local/bin/，类似于之前的kubectl

sudo mv minikube /usr/local/bin/

#############
###################
国内Ubuntu20.04下安装kubectl  #和helm
https://my.oschina.net/ykbj/blog/4315903

更新系统

sudo apt-get update && sudo apt-get install -y apt-transport-https

加入国内kubernetes-xenial源

echo "deb http://mirrors.ustc.edu.cn/kubernetes/apt kubernetes-xenial main" | sudo tee -a /etc/apt/sources.list.d/kubernetes.list

更新

sudo apt-get update

W: GPG 错误：http://mirrors.ustc.edu.cn/kubernetes/apt kubernetes-xenial InRelease: 由于没有公钥，无法验证下列签名： NO_PUBKEY 6A030B21BA07F4FB NO_PUBKEY 8B57C5C2836F4BEB

sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys +上面报错的密钥

sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 6A030B21BA07F4FB
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 8B57C5C2836F4BEB

安装kubectl

sudo apt-get install -y kubectl

build spire

make V=1 (0,1=verbose,2)

run spire server

https://spiffe.io/docs/latest/spire/installing/getting-started-linux-macos-x/
bin/spire-server run -config conf/server/server.conf &
bin/spire-server healthcheck
bin/spire-server token generate -spiffeID spiffe://example.org/myagent

bin/spire-agent run -config conf/agent/agent.conf -joinToken <token_string> &

bin/spire-agent run -config conf/agent/agent.conf -joinToken 6e8c4d05-e19b-429b-b8e4-eb24be3c6973 &
bin/spire-agent healthcheck

create registration polocy

on the current user’s UID ($(id -u))

bin/spire-server entry create -parentID spiffe://example.org/myagent \

-spiffeID spiffe://example.org/myservice -selector unix:uid:$(id -u)

bin/spire-server entry create -parentID spiffe://example.org/myagent
-spiffeID spiffe://example.org/myservice -selector unix:uid:1000

Retrieve and view a x509-SVID

bin/spire-agent api fetch x509 -write /tmp/

You can use the openssl command to view the contents of the SVID:

openssl x509 -in /tmp/svid.0.pem -text -noout

spire + k8s

https://spiffe.io/docs/latest/spire/installing/getting-started-k8s/

start minikube

sudo swapoff -a
sudo minikube start --vm-driver=none --memory=2048 --cpus=2 --disk-size=2g --apiserver-ips 127.0.0.1 --apiserver-name localhost --extra-config=kubelet.cgroup-driver=systemd --v=5

start minikube for spire

sudo minikube start --vm-driver=none --memory=2048 --cpus=2 --disk-size=2g --v=5
--extra-config=apiserver.service-account-signing-key-file=/var/lib/minikube/certs/sa.key
--extra-config=apiserver.service-account-key-file=/var/lib/minikube/certs/sa.pub
--extra-config=apiserver.service-account-issuer=api
--extra-config=apiserver.service-account-api-audiences=api,spire-server
--extra-config=apiserver.authorization-mode=Node,RBAC
--extra-config=kubelet.authentication-token-webhook=true

https://stackoverflow.com/questions/53877516/how-do-i-set-the-imagepullpolicy-with-minikube
https://serverfault.com/questions/964307/kubernetes-deployment-failed-to-pull-image-with-local-registry-minikube

sudo kubectl apply -f spire-namespace.yaml
sudo kubectl get namespaces

config spire server

sudo kubectl apply
-f server-account.yaml
-f spire-bundle-configmap.yaml
-f server-cluster-role.yaml

sudo kubectl apply
-f server-configmap.yaml
-f server-statefulset.yaml
-f server-service.yaml

check status

sudo  kubectl get statefulset --namespace spire
sudo kubectl get pods --namespace spire
sudo kubectl get services --namespace spire

config spire agent

sudo kubectl apply
-f agent-account.yaml
-f agent-cluster-role.yaml
sudo kubectl apply
-f agent-configmap.yaml
-f agent-daemonset.yaml

check status

sudo kubectl get daemonset --namespace spire
sudo kubectl get pods --namespace spire

register workload

sudo kubectl exec -n spire spire-server-0 --
/home/bryan/code/parsec/spire/bin/spire-server entry create
-spiffeID spiffe://example.org/ns/spire/sa/spire-agent
-selector k8s_sat:cluster:demo-cluster
-selector k8s_sat:agent_ns:spire
-selector k8s_sat:agent_sa:spire-agent
-node

sudo kubectl exec -n spire spire-server-0 --
/home/bryan/code/parsec/spire/bin/spire-server entry create
-spiffeID spiffe://example.org/ns/default/sa/default
-parentID spiffe://example.org/ns/spire/sa/spire-agent
-selector k8s:ns:default
-selector k8s:sa:default

https://github.com/kubernetes/minikube/issues/4589

Sign up for more like this.

Enter your email
Subscribe