반응형

 

 

2023.05.29

★ kube-apiserver에서 사용하고 있는 인증서 파일 위치 확인

1. kube-apiserver.yaml 파일 확인

-> --tls-cert-file=/etc/kubernetes/pki/apiserver.crt

controlplane ~ ➜ cat /etc/kubernetes/manifests/kube-apiserver.yaml | grep tls
    - --tls-cert-file=/etc/kubernetes/pki/apiserver.crt
    - --tls-private-key-file=/etc/kubernetes/pki/apiserver.key

controlplane ~ ➜

2. kube-apiserver pod 확인

-> --tls-cert-file=/etc/kubernetes/pki/apiserver.crt

controlplane ~ ➜  kubectl -n kube-system describe pods kube-apiserver-controlplane | grep tls
      --tls-cert-file=/etc/kubernetes/pki/apiserver.crt
      --tls-private-key-file=/etc/kubernetes/pki/apiserver.key

controlplane ~ ➜

 

 

★ kube-apiserver에서 사용하고 있는 etcd-client 인증서 파일 위치 확인

-> etcd-certfile=/etc/kubernetes/pki/apiserver-etcd-client.crt

controlplane ~ ➜  cat /etc/kubernetes/manifests/kube-apiserver.yaml | grep etcd    
    - --etcd-cafile=/etc/kubernetes/pki/etcd/ca.crt
    - --etcd-certfile=/etc/kubernetes/pki/apiserver-etcd-client.crt
    - --etcd-keyfile=/etc/kubernetes/pki/apiserver-etcd-client.key
    - --etcd-servers=https://127.0.0.1:2379

controlplane ~ ➜

 

 

★ kube-apiserver에서 사용하고 있는 kubelet 키 파일 위치 확인

-> --kubelet-client-key=/etc/kubernetes/pki/apiserver-kubelet-client.key

controlplane ~ ➜  cat /etc/kubernetes/manifests/kube-apiserver.yaml | grep kubelet
    - --kubelet-client-certificate=/etc/kubernetes/pki/apiserver-kubelet-client.crt
    - --kubelet-client-key=/etc/kubernetes/pki/apiserver-kubelet-client.key
    - --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname

controlplane ~ ➜

 

 

★ etcd server 에서 사용하고 있는 인증서 파일 위치 확인

-> --cert-file=/etc/kubernetes/pki/etcd/server.crt

controlplane ~ ➜  cat /etc/kubernetes/manifests/etcd.yaml | grep cert
    - --cert-file=/etc/kubernetes/pki/etcd/server.crt
    - --client-cert-auth=true
    - --peer-cert-file=/etc/kubernetes/pki/etcd/peer.crt
    - --peer-client-cert-auth=true
      name: etcd-certs
    name: etcd-certs

controlplane ~ ➜

 

 

★ etcd server 에서 사용하고 있는 자체 ca 인증서 파일 위치 확인

-> --trusted-ca-file=/etc/kubernetes/pki/etcd/ca.crt

controlplane ~ ➜  cat /etc/kubernetes/manifests/etcd.yaml | grep ca
    - --peer-trusted-ca-file=/etc/kubernetes/pki/etcd/ca.crt
    - --trusted-ca-file=/etc/kubernetes/pki/etcd/ca.crt
  priorityClassName: system-node-critical

controlplane ~ ➜

 

 

★ Kube-apiserver 인증서에 구성된 CN 이름은 무엇인가요?

-> Subject: CN = kube-apiserver

controlplane ~ ➜  openssl x509 -in /etc/kubernetes/pki/apiserver.crt -text
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number: 8483000120492181273 (0x75b9abaa2b884719)
        Signature Algorithm: sha256WithRSAEncryption
        Issuer: CN = kubernetes
        Validity
            Not Before: May 29 11:43:33 2023 GMT
            Not After : May 28 11:43:33 2024 GMT
        Subject: CN = kube-apiserver
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                RSA Public-Key: (2048 bit)
...

 

 

★ Kube-apiserver 인증서를 발급한 CA의 이름은 무엇인가요?

-> Issuer: CN = kubernetes

controlplane ~ ➜  openssl x509 -in /etc/kubernetes/pki/apiserver.crt -text
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number: 8483000120492181273 (0x75b9abaa2b884719)
        Signature Algorithm: sha256WithRSAEncryption
        Issuer: CN = kubernetes
        Validity
            Not Before: May 29 11:43:33 2023 GMT
            Not After : May 28 11:43:33 2024 GMT
        Subject: CN = kube-apiserver
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                RSA Public-Key: (2048 bit)
...

 

 

★ Kube-apiserver 인증서에 구성된 대체 이름은 무엇인가요?

-> DNS:controlplane, DNS:kubernetes, DNS:kubernetes.default, DNS:kubernetes.default.svc, DNS:kubernetes.default.svc.cluster.local, IP Address:10.96.0.1, IP Address:192.26.249.9

controlplane ~ ➜  openssl x509 -in /etc/kubernetes/pki/apiserver.crt -text
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number: 8483000120492181273 (0x75b9abaa2b884719)
        Signature Algorithm: sha256WithRSAEncryption
        Issuer: CN = kubernetes
        Validity
            Not Before: May 29 11:43:33 2023 GMT
            Not After : May 28 11:43:33 2024 GMT
        Subject: CN = kube-apiserver
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                RSA Public-Key: (2048 bit)
                Modulus:
                    00:a3:2f:ff:75:ed:b2:38:74:01:f9:b1:41:51:aa:
                    f5:bb:9a:39:02:46:c2:5b:05:b1:0e:8f:75:9b:46:
                    18:a5:35:52:2f:2d:22:3b:fe:37:e3:ea:98:32:c5:
                    79:b4:2d:1b:f2:67:cd:f6:7d:4e:fa:e8:a0:69:b4:
                    4b:c8:25:46:20:4b:ad:69:dd:fa:63:56:b4:5c:4f:
                    ce:b7:28:bb:43:de:59:5f:c6:e7:c7:16:08:11:cf:
                    28:b2:4a:7f:20:74:3d:f4:53:6a:b6:33:37:25:98:
                    3e:a7:02:56:da:1b:75:7a:39:bd:0a:31:d5:26:cb:
                    30:8b:3d:bf:a5:58:48:8c:a8:5d:b4:eb:51:0d:72:
                    52:32:85:60:0d:56:2f:46:3c:65:90:4a:9b:a3:01:
                    b3:d9:01:b2:d9:ea:70:68:38:49:d5:1a:29:9f:52:
                    b8:54:72:71:0c:4a:88:4b:73:63:6f:05:a0:b6:23:
                    03:31:12:be:c3:cf:6c:b7:2b:e6:4e:50:a1:1b:7f:
                    ab:2a:ba:5f:92:16:3d:4c:ac:d8:02:11:78:8b:bf:
                    4e:43:3b:e5:0c:57:fb:6f:8a:81:ef:51:7e:a3:92:
                    2a:de:2b:96:ae:95:2e:dc:e3:97:ce:c7:af:8d:42:
                    67:2c:6a:3a:fa:fa:67:79:d2:14:52:47:eb:65:ca:
                    53:af
                Exponent: 65537 (0x10001)
        X509v3 extensions:
            X509v3 Key Usage: critical
                Digital Signature, Key Encipherment
            X509v3 Extended Key Usage: 
                TLS Web Server Authentication
            X509v3 Basic Constraints: critical
                CA:FALSE
            X509v3 Authority Key Identifier: 
                keyid:AB:7D:E2:A1:2C:F0:E0:27:53:52:72:D8:C9:46:76:09:F8:77:0D:63

            X509v3 Subject Alternative Name: 
                DNS:controlplane, DNS:kubernetes, DNS:kubernetes.default, DNS:kubernetes.default.svc, DNS:kubernetes.default.svc.cluster.local, IP Address:10.96.0.1, IP Address:192.26.249.9

 

 

★ etcd server 인증서에 구성된 CN은 무엇인가요?

-> Subject: CN = controlplane

controlplane ~ ➜  openssl x509 -in /etc/kubernetes/pki/etcd/server.crt -text
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number: 253720388574558331 (0x38565596142b87b)
        Signature Algorithm: sha256WithRSAEncryption
        Issuer: CN = etcd-ca
        Validity
            Not Before: May 29 11:43:34 2023 GMT
            Not After : May 28 11:43:34 2024 GMT
        Subject: CN = controlplane
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                RSA Public-Key: (2048 bit)
...

 

★ etcd server 인증서는 발급일로부터 얼마 동안 유효하나요?

-> 1 years 

controlplane ~ ➜  openssl x509 -in /etc/kubernetes/pki/etcd/server.crt -text
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number: 253720388574558331 (0x38565596142b87b)
        Signature Algorithm: sha256WithRSAEncryption
        Issuer: CN = etcd-ca
        Validity
            Not Before: May 29 11:43:34 2023 GMT
            Not After : May 28 11:43:34 2024 GMT
        Subject: CN = controlplane
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                RSA Public-Key: (2048 bit)
...

 

 

★ etcd server ca인증서는 발급일로부터 얼마 동안 유효하나요?

->  10 years

controlplane ~ ➜  openssl x509 -in /etc/kubernetes/pki/ca.crt -text
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number: 0 (0x0)
        Signature Algorithm: sha256WithRSAEncryption
        Issuer: CN = kubernetes
        Validity
            Not Before: May 29 11:43:33 2023 GMT
            Not After : May 26 11:43:33 2033 GMT
        Subject: CN = kubernetes
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                RSA Public-Key: (2048 bit)
...

 

 

★ kubectl 명령어가 동작하지 않습니다. etcd server를 확인해보고 틀린부분을 수정하시오.

-> etcd 서버 yaml 파일을 확인해보면 cert-file 경로에 server-certificate.crt 파일이라고 저장되어있다.

controlplane ~ ➜  cat /etc/kubernetes/manifests/etcd.yaml | grep cert-file
    - --cert-file=/etc/kubernetes/pki/etcd/server-certificate.crt
    - --peer-cert-file=/etc/kubernetes/pki/etcd/peer.crt

controlplane ~ ➜

->  해당 경로에 crt 파일을 확인해보면 파일명이 다른걸 확인할 수 있다.

controlplane ~ ➜  ls /etc/kubernetes/pki/etcd/server* | grep .crt
/etc/kubernetes/pki/etcd/server.crt

controlplane ~ ➜

->  경로명을 수정해주고 기다리면 api-server가 정상적으로 작동한다.

controlplane ~ ➜  vi /etc/kubernetes/manifests/etcd.yaml 

controlplane ~ ➜  cat /etc/kubernetes/manifests/etcd.yaml | grep cert-file
    - --cert-file=/etc/kubernetes/pki/etcd/server.crt
    - --peer-cert-file=/etc/kubernetes/pki/etcd/peer.crt

controlplane ~ ➜

 

 

★ kubectl 명령어가 동작하지 않습니다. apiserver 서버의 logs를 확인해서 문제를 해결하시오.

-> crictl ps -a | grep kube-apiserver 명령어로 pod 상태 확인 

controlplane ~ ➜  crictl ps -a | grep kube-apiserver
9bd19c0102ca6       a31e1d84401e6       36 seconds ago      Exited              kube-apiserver            5                   5413bff6f15be       kube-apiserver-controlplane

controlplane ~ ➜

-> logs를 확인해보니 etcd ca 인증서에 문제가 있는 것을 확인했습니다.

controlplane ~ ➜  crictl logs --tail=5 9bd19c0102ca6
  "BalancerAttributes": null,
  "Type": 0,
  "Metadata": null
}. Err: connection error: desc = "transport: authentication handshake failed: x509: certificate signed by unknown authority"
E0529 13:03:01.408411       1 run.go:74] "command failed" err="context deadline exceeded"

controlplane ~ ➜

-> kube-apiserver.yaml 파일의 etcd cafile 위치가 잘못되어있습니다. 수정해주세요.

controlplane ~ ➜  cat /etc/kubernetes/manifests/kube-apiserver.yaml | grep etcd
    - --etcd-cafile=/etc/kubernetes/pki/ca.crt
    - --etcd-certfile=/etc/kubernetes/pki/apiserver-etcd-client.crt
    - --etcd-keyfile=/etc/kubernetes/pki/apiserver-etcd-client.key
    - --etcd-servers=https://127.0.0.1:2379

controlplane ~ ➜  cd /etc/kubernetes/pki/etcd/

controlplane kubernetes/pki/etcd ➜  ls -al
total 40
drwxr-xr-x 2 root root 4096 May 29 08:44 .
drwxr-xr-x 3 root root 4096 May 29 08:44 ..
-rw-r--r-- 1 root root 1086 May 29 08:44 ca.crt
-rw------- 1 root root 1675 May 29 08:44 ca.key
-rw-r--r-- 1 root root 1159 May 29 08:44 healthcheck-client.crt
-rw------- 1 root root 1679 May 29 08:44 healthcheck-client.key
-rw-r--r-- 1 root root 1208 May 29 08:44 peer.crt
-rw------- 1 root root 1679 May 29 08:44 peer.key
-rw-r--r-- 1 root root 1208 May 29 08:44 server.crt
-rw------- 1 root root 1675 May 29 08:44 server.key

controlplane kubernetes/pki/etcd ➜

-> --etcd-cafile=/etc/kubernetes/pki/etcd/ca.crt

controlplane ~ ➜  cat /etc/kubernetes/manifests/kube-apiserver.yaml | grep etcd
    - --etcd-cafile=/etc/kubernetes/pki/etcd/ca.crt
    - --etcd-certfile=/etc/kubernetes/pki/apiserver-etcd-client.crt
    - --etcd-keyfile=/etc/kubernetes/pki/apiserver-etcd-client.key
    - --etcd-servers=https://127.0.0.1:2379

controlplane ~ ➜

 

 

 

 

 

 

 

 

 

 

반응형
저작자표시

'Kubernetes > Kubernetes Exam' 카테고리의 다른 글

[K8s] CKA 취득 연습문제#7 (kubeconfig view)  (0) 2023.05.30
[K8s] CKA 취득 연습문제#6 (csr approve, reject)  (0) 2023.05.30
[K8s] CKA 취득 연습문제#4 (etcd backup, recovery)  (0) 2023.05.24
[K8s] CKA 취득 연습문제#3 (etcd backup, snapshot)  (0) 2023.05.20
[K8s] CKA 취득 연습문제#2 (node upgrade)  (0) 2023.05.17

+ Recent posts

티스토리툴바