IT 끄적이는 물고기

2023.10.03

드디어 작년부터 준비하던 CKA시험을 응시하고 취득을 완료했다.

22년 사이버 먼데이때 50% 할인을 받아서 신청을했다. 정가 $ 395달러 기준 ($ 197.50)

생각보다 어렵지 않았던거 같다.

•  접수 사이트

https://training.linuxfoundation.org/certification/certified-kubernetes-administrator-cka/

공부 참고 사이트

1. 뭄샤드 강의

• https://www.udemy.com/course/certified-kubernetes-administrator-with-practice-tests/

2. kubernets 공식 문서

• https://kubernetes.io/docs/home/

3. KodeKloud(뭄샤드 실습)

• https://kodekloud.com/courses/labs-certified-kubernetes-administrator-with-practice-tests/

4. Killer.sh (자격증 접수하면 2회 가능, 마지막 날 풀어보기 반복, 실제 시험보다 어려움)

• https://killer.sh/

중요하지않지만 간단한 꿀팁 !

1. 간단하게 배점 먹고들어갈 수 있는 jsonpath 사용법 배우기

2. kubectl cheat sheet 어느정도 배우고 가기

 - https://kubernetes.io/docs/reference/kubectl/cheatsheet/

3. 북마크를 이용해서 쿠버네티스 문서 yaml 파일 빨리 만들기

4. 시험 시작할때 무조건 context 수정하기 꼭! 꼭! 꼭!

5. 시험 원격화면이 굉장히 작으므로 해상도 큰거 사용하기

6. 시험환경은 많이 개선되어서 생각보다 빠르고 쾌적

시험에 나온 문제 (기억나는 부분만...)

1. 현재 사용가능한 Node들의 정보 출력 및 저장 ( Node Name, Taint=NoSchedule, 사용가능 list)

2. 현재 사용중인 Pods중에 특정 Label 가지고있는 Pods 출력 및 저장

3. Stateful Scale size 조절 

4. Deploy Scale Size 조절, Deploy된 Pods들의 image update

5. 현재 사용중인 Controlplane node만 Cluster upgrade 하기 (Kubeadm, Kubelet, Kubectl)

6. 하나의 Pod에서 2개 Container사용 (Sidecar, EmptyDir)

7. Deployment 생성, Pods 생성 간단한 생성 등등..

8. 사용자 Rule, Rulebinding, Serviceaccount 생성 및 binding 작업

9. ClusterRule, ClusterRulebinding 연동 설정

10. Storage 관련 Pod에 PV, PVC 생성 및 연동

11. 배점이 제일 높은 Node Troubleshooting 문제

다음 목표는 11월 말까지 CKAD 취득 !!

레쓰꼬  !!

2023.05.20

★ 클러스터에서 실행 중인 ETCD의 버전은 무엇인가요?

-> etcd-version : 3.5.6

controlplane ~ ➜  kubectl -n kube-system logs etcd-controlplane | grep -i 'etcd-version'
{"level":"info","ts":"2023-05-20T05:14:33.291Z","caller":"embed/etcd.go:306","msg":"starting an etcd server","etcd-version":"3.5.6","git-sha":"cecbe35ce","go-version":"go1.16.15","go-os":"linux","go-arch":"amd64","max-cpu-set":36,"max-cpu-available":36,"member-initialized":false,"name":"controlplane","data-dir":"/var/lib/etcd","wal-dir":"","wal-dir-dedicated":"","member-dir":"/var/lib/etcd/member","force-new-cluster":false,"heartbeat-interval":"100ms","election-timeout":"1s","initial-election-tick-advance":true,"snapshot-count":10000,"max-wals":5,"max-snapshots":5,"snapshot-catchup-entries":5000,"initial-advertise-peer-urls":["https://192.6.237.3:2380"],"listen-peer-urls":["https://192.6.237.3:2380"],"advertise-client-urls":["https://192.6.237.3:2379"],"listen-client-urls":["https://127.0.0.1:2379","https://192.6.237.3:2379"],"listen-metrics-urls":["http://127.0.0.1:2381"],"cors":["*"],"host-whitelist":["*"],"initial-cluster":"controlplane=https://192.6.237.3:2380","initial-cluster-state":"new","initial-cluster-token":"etcd-cluster","quota-backend-bytes":2147483648,"max-request-bytes":1572864,"max-concurrent-streams":4294967295,"pre-vote":true,"initial-corrupt-check":true,"corrupt-check-time-interval":"0s","compact-check-time-enabled":false,"compact-check-time-interval":"1m0s","auto-compaction-mode":"periodic","auto-compaction-retention":"0s","auto-compaction-interval":"0s","discovery-url":"","discovery-proxy":"","downgrade-check-interval":"5s"}

controlplane ~ ➜

★ controlplane node에서 ETCD 클러스터에 연결할 수 있는 주소는 어디인가요?

 ->  https://127.0.0.1:2379

controlplane ~ ➜  kubectl -n kube-system describe pod etcd-controlplane | grep -i 'listen-client-url'
      --listen-client-urls=https://127.0.0.1:2379,https://192.6.237.3:2379

controlplane ~ ➜

★ ETCD 서버 인증서 파일은 어디에 있나요?

 -> --cert-file=/etc/kubernetes/pki/etcd/server.crt

controlplane ~ ➜  kubectl -n kube-system describe pod etcd-controlplane | grep -i 'cert-file'
      --cert-file=/etc/kubernetes/pki/etcd/server.crt
      --peer-cert-file=/etc/kubernetes/pki/etcd/peer.crt

controlplane ~ ➜

★ ETCD CA 인증서 파일은 어디에 있나요?

 -> --peer-trusted-ca-file=/etc/kubernetes/pki/etcd/ca.crt

controlplane ~ ➜  kubectl -n kube-system describe pod etcd-controlplane | grep -i 'ca-file'
      --peer-trusted-ca-file=/etc/kubernetes/pki/etcd/ca.crt
      --trusted-ca-file=/etc/kubernetes/pki/etcd/ca.crt

controlplane ~ ➜

★ 클러스터의 master node는 오늘 밤에 재부팅이 예정되어 있습니다. 문제가 발생할 것으로 예상되지는 않지만 필요한 백업을 수행해야 합니다. 기본 제공 스냅샷 기능을 사용하여 ETCD 데이터베이스의 스냅샷을 만듭니다.

ETCDCTL_API=3 etcdctl --endpoints=https://[127.0.0.1]:2379 \
 --cacert=/etc/kubernetes/pki/etcd/ca.crt \
 --cert=/etc/kubernetes/pki/etcd/server.crt \
 --key=/etc/kubernetes/pki/etcd/server.key \
 snapshot save /opt/snapshot-pre-boot.db

controlplane ~ ➜ ETCDCTL_API=3 etcdctl --endpoints=https://[127.0.0.1]:2379 \
> --cacert=/etc/kubernetes/pki/etcd/ca.crt \
> --cert=/etc/kubernetes/pki/etcd/server.crt \
> --key=/etc/kubernetes/pki/etcd/server.key \
> snapshot save /opt/snapshot-pre-boot.db
Snapshot saved at /opt/snapshot-pre-boot.db

controlplane ~ ➜  ls /opt/
cni  containerd  snapshot-pre-boot.db

controlplane ~ ➜

★ 재부팅 후 마스터 노드가 다시 온라인 상태가 되었지만 애플리케이션에 액세스할 수 없습니다. 클러스터의 애플리케이션 상태를 확인하세요. 무슨 문제인가요?

- 배포가 없습니다.
- 서비스가 존재하지 않음
- 파드가 없음
 위의 모든 것

★ 백업 파일을 사용하여 클러스터의 원래 상태를 복원합니다.

controlplane ~ ➜  ETCDCTL_API=3 etcdctl  --data-dir /var/lib/etcd-from-backup \
> snapshot restore /opt/snapshot-pre-boot.db
2023-05-20 02:06:58.280313 I | mvcc: restore compact to 2461
2023-05-20 02:06:58.287347 I | etcdserver/membership: added member 8e9e05c52164694d [http://localhost:2380] to cluster cdf818194e3a8c32

2023.05.20

★ Cluster 버전 확인

 -> v1.25.0

controlplane ~ ➜  kubectl get nodes 
NAME           STATUS   ROLES           AGE    VERSION
controlplane   Ready    control-plane   116m   v1.25.0
node01         Ready    <none>          116m   v1.25.0

controlplane ~ ➜

★ Worker node인 node는 어떤 노드인가요 ?

 -> Taints 가 없으니 다 Worker node

controlplane ~ ➜  kubectl describe nodes controlplane | grep Taints
Taints:             <none>

controlplane ~ ➜  kubectl describe nodes node01 | grep Taints
Taints:             <none>

★ deploy되는 pods는 어떤 node에 올라가나요 ?

 -> node01, controlplane

controlplane ~ ➜  kubectl get pods -o wide
NAME                    READY   STATUS    RESTARTS   AGE   IP           NODE           NOMINATED NODE   READINESS GATES
blue-5db6db69f7-94bd2   1/1     Running   0          14m   10.244.1.4   node01         <none>           <none>
blue-5db6db69f7-d4ckd   1/1     Running   0          14m   10.244.1.2   node01         <none>           <none>
blue-5db6db69f7-fgrvp   1/1     Running   0          14m   10.244.1.3   node01         <none>           <none>
blue-5db6db69f7-fkvdv   1/1     Running   0          14m   10.244.0.5   controlplane   <none>           <none>
blue-5db6db69f7-kcnkl   1/1     Running   0          14m   10.244.0.4   controlplane   <none>           <none>

★ 클러스터를 업그레이드하는 작업을 수행해야 합니다. 애플리케이션에 접속하는 사용자에게 영향을 미치지 않아야 하며 새 VM을 프로비저닝할 수 없습니다. 클러스터를 업그레이드하기 위해 어떤 방법을 사용하시겠습니까?

 -> Worker node를 다른 노드로 이동하면서 한 번에 한 노드씩 업그레이드

★ 현재 쿠버네티스의 안정적인 최신 버전은 무엇인가요?

 -> v1.27.2

controlplane ~ ➜  kubeadm upgrade plan
[upgrade/config] Making sure the configuration is correct:
[upgrade/config] Reading configuration from the cluster...
[upgrade/config] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'
[preflight] Running pre-flight checks.
[upgrade] Running cluster health checks
[upgrade] Fetching available versions to upgrade to
[upgrade/versions] Cluster version: v1.25.0
[upgrade/versions] kubeadm version: v1.25.0
I0519 23:10:12.713401   17274 version.go:256] remote version is much newer: v1.27.2; falling back to: stable-1.25
[upgrade/versions] Target version: v1.25.10
[upgrade/versions] Latest version in the v1.25 series: v1.25.10

★ 현재 버전의 kubeadm 도구가 설치된 상태에서 업그레이드할 수 있는 최신 버전은 무엇인가요?

 -> v1.25.10

controlplane ~ ➜  kubeadm upgrade plan
[upgrade/config] Making sure the configuration is correct:
[upgrade/config] Reading configuration from the cluster...
[upgrade/config] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'
[preflight] Running pre-flight checks.
[upgrade] Running cluster health checks
[upgrade] Fetching available versions to upgrade to
[upgrade/versions] Cluster version: v1.25.0
[upgrade/versions] kubeadm version: v1.25.0
I0519 23:10:12.713401   17274 version.go:256] remote version is much newer: v1.27.2; falling back to: stable-1.25
[upgrade/versions] Target version: v1.25.10
[upgrade/versions] Latest version in the v1.25 series: v1.25.10

Components that must be upgraded manually after you have upgraded the control plane with 'kubeadm upgrade apply':
COMPONENT   CURRENT       TARGET
kubelet     2 x v1.25.0   v1.25.10

Upgrade to the latest version in the v1.25 series:

COMPONENT                 CURRENT   TARGET
kube-apiserver            v1.25.0   v1.25.10
kube-controller-manager   v1.25.0   v1.25.10
kube-scheduler            v1.25.0   v1.25.10
kube-proxy                v1.25.0   v1.25.10
CoreDNS                   v1.9.3    v1.9.3
etcd                      3.5.4-0   3.5.4-0

You can now apply the upgrade by executing the following command:

        kubeadm upgrade apply v1.25.10

Note: Before you can perform this upgrade, you have to update kubeadm to v1.25.10.

★ controlplane node를 업그레이드할 것입니다. 컨트롤 플레인 노드에서 워크로드를 비우고 예약 불가능으로 설정해주세요.

controlplane ~ ✖ kubectl drain controlplane --ignore-daemonsets 
node/controlplane already cordoned
Warning: ignoring DaemonSet-managed Pods: kube-flannel/kube-flannel-ds-qljgv, kube-system/kube-proxy-xkrtl
evicting pod kube-system/coredns-565d847f94-8ldgw
evicting pod default/blue-5db6db69f7-jhn5p
evicting pod default/blue-5db6db69f7-6pkjn
evicting pod kube-system/coredns-565d847f94-2bbdc
pod/blue-5db6db69f7-jhn5p evicted
pod/blue-5db6db69f7-6pkjn evicted
pod/coredns-565d847f94-2bbdc evicted
pod/coredns-565d847f94-8ldgw evicted
node/controlplane drained

controlplane ~ ➜

★ Controlplane node 업그레이드

controlplane ~ ➜  apt update
Get:1 https://packages.cloud.google.com/apt kubernetes-xenial InRelease [8,993 B]    
Hit:2 https://download.docker.com/linux/ubuntu focal InRelease
...

controlplane ~ ➜  apt-get install kubeadm=1.26.0-00
Reading package lists... Done
Building dependency tree       
Reading state information... Done
...

controlplane ~ ➜  kubeadm upgrade apply v1.26.0
[upgrade/config] Making sure the configuration is correct:
[upgrade/config] Reading configuration from the cluster...
...

controlplane ~ ➜  kubectl get nodes 
NAME           STATUS                     ROLES           AGE   VERSION
controlplane   Ready,SchedulingDisabled   control-plane   85m   v1.26.0
node01         Ready                      <none>          84m   v1.25.0

controlplane ~ ➜

★ Controlplane node에서 예약 가능으로 설정하세요.

controlplane ~ ✖ kubectl uncordon controlplane 
node/controlplane uncordoned

★ node01를 업그레이드할 것입니다. 워크로드를 비우고 예약 불가능으로 설정해주세요.

 -> drain하게되면 node에 있던 pod들은 이동한다.

controlplane ~ ➜  kubectl drain node01 --ignore-daemonsets 
node/node01 cordoned
Warning: ignoring DaemonSet-managed Pods: kube-flannel/kube-flannel-ds-2stnt, kube-system/kube-proxy-n5hmd
evicting pod kube-system/coredns-787d4945fb-lqjjv
evicting pod default/blue-5db6db69f7-225hb
evicting pod default/blue-5db6db69f7-c7ptb
evicting pod default/blue-5db6db69f7-gkbz4
evicting pod default/blue-5db6db69f7-r4rvt
evicting pod default/blue-5db6db69f7-tv2p9
evicting pod kube-system/coredns-787d4945fb-kr6xw
pod/blue-5db6db69f7-gkbz4 evicted
pod/blue-5db6db69f7-tv2p9 evicted
pod/blue-5db6db69f7-r4rvt evicted
pod/blue-5db6db69f7-225hb evicted
pod/blue-5db6db69f7-c7ptb evicted
pod/coredns-787d4945fb-kr6xw evicted
pod/coredns-787d4945fb-lqjjv evicted
node/node01 drained

controlplane ~ ➜  kubectl get pods -o wide
NAME                    READY   STATUS    RESTARTS   AGE   IP            NODE           NOMINATED NODE   READINESS GATES
blue-5db6db69f7-5sld2   1/1     Running   0          17s   10.244.0.7    controlplane   <none>           <none>
blue-5db6db69f7-9c4w2   1/1     Running   0          17s   10.244.0.12   controlplane   <none>           <none>
blue-5db6db69f7-9mhbr   1/1     Running   0          17s   10.244.0.9    controlplane   <none>           <none>
blue-5db6db69f7-j7tgx   1/1     Running   0          17s   10.244.0.11   controlplane   <none>           <none>
blue-5db6db69f7-p98g5   1/1     Running   0          17s   10.244.0.10   controlplane   <none>           <none>

controlplane ~ ➜

★ node01 업그레이드

controlplane ~ ➜  ssh node01

root@node01 ~ ➜  apt-get update
Get:2 http://archive.ubuntu.com/ubuntu focal InRelease [265 kB]                      
Get:3 https://download.docker.com/linux/ubuntu focal InRelease [57.7 kB]             
Get:1 https://packages.cloud.google.com/apt kubernetes-xenial InRelease [8993 B]
...
root@node01 ~ ➜  apt-get install kubeadm=1.26.0-00
Reading package lists... Done
Building dependency tree       
Reading state information... Done
...
oot@node01 ~ ➜  kubeadm upgrade node
[upgrade] Reading configuration from the cluster...
[upgrade] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'
[preflight] Running pre-flight checks
...
root@node01 ~ ➜  apt-get install kubelet=1.26.0-00
Reading package lists... Done
Building dependency tree       
Reading state information... Done
...
root@node01 ~ ➜  systemctl daemon-reload 
root@node01 ~ ➜  systemctl restart kubelet.service 

controlplane ~ ➜  kubectl get node
NAME           STATUS                        ROLES           AGE    VERSION
controlplane   Ready                         control-plane   118m   v1.26.0
node01         NotReady,SchedulingDisabled   <none>          117m   v1.26.0

controlplane ~ ➜

★ node01 노드를 예약 가능으로 설정하세요.

controlplane ~ ➜  kubectl uncordon node01
node/node01 uncordoned