前提条件
- 已开通云容器引擎,至少有一个云容器引擎集群实例。产品入口:云容器引擎
- 开通天翼云服务网格实例
操作步骤
创建测试命名空间
kubectl create ns foo
打开sidecar自动注入
kubectl label ns foo istio-injection=enabled
部署sleep和httpbin应用
apiVersion: v1
kind: ServiceAccount
metadata:
name: sleep
---
apiVersion: v1
kind: Service
metadata:
name: sleep
labels:
app: sleep
service: sleep
spec:
ports:
- port: 80
name: http
selector:
app: sleep
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: sleep
spec:
replicas: 1
selector:
matchLabels:
app: sleep
template:
metadata:
labels:
app: sleep
spec:
terminationGracePeriodSeconds: 0
serviceAccountName: sleep
containers:
- name: sleep
image: registry-vpc-crs-huadong1.ctyun.cn/library/curl
command: ["/bin/sleep", "infinity"]
imagePullPolicy: IfNotPresent
volumeMounts:
- mountPath: /etc/sleep/tls
name: secret-volume
volumes:
- name: secret-volume
secret:
secretName: sleep-secret
optional: true
---
部署外部授权服务
apiVersion: v1
kind: Service
metadata:
name: ext-authz
labels:
app: ext-authz
spec:
ports:
- name: http
port: 8000
targetPort: 8000
- name: grpc
port: 9000
targetPort: 9000
selector:
app: ext-authz
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: ext-authz
spec:
replicas: 1
selector:
matchLabels:
app: ext-authz
template:
metadata:
labels:
app: ext-authz
spec:
containers:
- image: registry-vpc-crs-huadong1.ctyun.cn/library/ext-authz:1.16.2
imagePullPolicy: IfNotPresent
name: ext-authz
ports:
- containerPort: 8000
- containerPort: 9000
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: httpbin
---
apiVersion: v1
kind: Service
metadata:
name: httpbin
labels:
app: httpbin
service: httpbin
spec:
ports:
- name: http
port: 8000
targetPort: 80
selector:
app: httpbin
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: httpbin
spec:
replicas: 1
selector:
matchLabels:
app: httpbin
version: v1
template:
metadata:
labels:
app: httpbin
version: v1
spec:
serviceAccountName: httpbin
containers:
- image: registry-vpc-crs-huadong1.ctyun.cn/library/httpbin
imagePullPolicy: IfNotPresent
name: httpbin
ports:
- containerPort: 80
部署完成后,在foo命名空间下看到3个服务
不使用授权策略的情况下,验证从sleep应用访问httpbin应用没有被拦截(返回状态码200):
kubectl exec "$(kubectl get pod -l
app=sleep -n foo -o jsonpath={.items..metadata.name})" -c sleep -n foo --
curl http://httpbin.foo:8000/ip -s -o /dev/null -w "%{http_code}\n"
200
查看外部授权服务已经启动,HTTP和gRPC授权服务分别监听8000和9000端口:
kubectl logs "$(kubectl get pod -l
app=ext-authz -n foo -o jsonpath={.items..metadata.name})" -n foo -c
ext-authz
2023/08/14 11:26:54 Starting HTTP server at
[::]:8000
2023/08/14 11:26:54 Starting gRPC server at
[::]:9000
添加外部授权服务
将上面的HTTP和gRPC服务添加到服务网格的外部授权服务内
定义授权策略&验证访问
定义如下授权策略,对httpbin应用的/headers路径的请求将被转发到第三方授权服务进行验证:
apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
name: ext-authz
spec:
selector:
matchLabels:
app: httpbin
action: CUSTOM
provider:
# The provider name must match the extension provider defined in the mesh config.
# You can also replace this with sample-ext-authz-http to test the other external authorizer definition.
name: sample-ext-authz-grpc
rules:
# The rules specify when to trigger the external authorizer.
- to:
- operation:
paths: ["/headers"]
从sleep应用请求httpbin的/headers路径,由于请求头带了"x-ext-authz:deny",请求被拦截:
kubectl exec "$(kubectl get pod -l app=sleep -n foo -o jsonpath={.items..metadata.name})" -c sleep -n foo -- curl "http://httpbin.foo:8000/headers" -H "x-ext-authz: deny" -s
denied by ext_authz for not found header `x-ext-authz: allow` in the request
修改请求,带上"x-ext-authz: allow"头部,请求放行:
kubectl exec "$(kubectl get pod -l app=sleep -n foo -o jsonpath={.items..metadata.name})" -c sleep -n foo -- curl "http://httpbin.foo:8000/headers" -H "x-ext-authz: allow" -s
{
"headers": {
"Accept": "*/*",
"Host": "httpbin.foo:8000",
"User-Agent": "curl/8.2.1",
"X-B3-Parentspanid": "eb42a8165099e7db",
"X-B3-Sampled": "1",
"X-B3-Spanid": "cd6540ba1cfd9e8c",
"X-B3-Traceid": "e7e15cc10dc66630eb42a8165099e7db",
"X-Envoy-Attempt-Count": "1",
"X-Ext-Authz": "allow",
"X-Ext-Authz-Additional-Header-Override": "grpc-additional-header-override-value",
"X-Ext-Authz-Check-Received": "source:{address:{socket_address:{address:\"10.1.0.25\" port_value:37654}} principal:\"spiffe://cluster.local/ns/foo/sa/sleep\"} destination:{address:{socket_address:{address:\"10.1.0.24\" port_value:80}} principal:\"spiffe://cluster.local/ns/foo/sa/httpbin\"} request:{time:{seconds:1692015383 nanos:990298000} http:{id:\"7462237371770661564\" method:\"GET\" headers:{key:\":authority\" value:\"httpbin.foo:8000\"} headers:{key:\":method\" value:\"GET\"} headers:{key:\":path\" value:\"/headers\"} headers:{key:\":scheme\" value:\"http\"} headers:{key:\"accept\" value:\"*/*\"} headers:{key:\"user-agent\" value:\"curl/8.2.1\"} headers:{key:\"x-b3-sampled\" value:\"1\"} headers:{key:\"x-b3-spanid\" value:\"eb42a8165099e7db\"} headers:{key:\"x-b3-traceid\" value:\"e7e15cc10dc66630eb42a8165099e7db\"} headers:{key:\"x-envoy-attempt-count\" value:\"1\"} headers:{key:\"x-ext-authz\" value:\"allow\"} headers:{key:\"x-forwarded-client-cert\" value:\"By=spiffe://cluster.local/ns/foo/sa/httpbin;Hash=c32db24acfa670a8bbe46f0897ebb70b9ccc0e630ee32afcd1ec037d6616e6c5;Subject=\\\"\\\";URI=spiffe://cluster.local/ns/foo/sa/sleep\"} headers:{key:\"x-forwarded-proto\" value:\"http\"} headers:{key:\"x-request-id\" value:\"aba7b68c-6bee-9d58-b163-7454075c6ece\"} path:\"/headers\" host:\"httpbin.foo:8000\" scheme:\"http\" protocol:\"HTTP/1.1\"}} metadata_context:{}",
"X-Ext-Authz-Check-Result": "allowed",
"X-Forwarded-Client-Cert": "By=spiffe://cluster.local/ns/foo/sa/httpbin;Hash=c32db24acfa670a8bbe46f0897ebb70b9ccc0e630ee32afcd1ec037d6616e6c5;Subject=\"\";URI=spiffe://cluster.local/ns/foo/sa/sleep"
}
}
使用Opa策略引擎实现访问控制
-
在服务网格控制台->网格管理->OPA功能开关菜单下打开OPA开关,主要是安装OPA控制面组件
-
在sidecar管理->sidecar代理配置菜单下选择命名空间tab,选择我们验证功能要用的命名空间,这里选择default,配置sidecar入流量不拦截8181端口(OPA agent的配置端口)
-
在default命名空间下部署以下yaml,用于OPA agent相关功能的准备;其中OPA-istio-config这个config map定义了OPA agent启动的配置;OpaPolicy定义了默认的OPA策略;ext-authz这个EnvoyFilter定义了当前命名空间下的外部授权策略,请求进入sidecar之后会执行这个外部授权插件,调用OPA agent服务
apiVersion: v1
kind: ConfigMap
metadata:
name: opa-istio-config
data:
config.yaml: |
plugins:
envoy_ext_authz_grpc:
addr: :9191
path: istio/authz/allow
decision_logs:
console: true
---
apiVersion: v1
kind: ConfigMap
metadata:
name: opa-policy
data:
policy.rego: |
package istio.authz
import input.attributes.request.http as http_request
import input.parsed_path
default allow = false
- 给default命名空间打上标签,自动注入istio sidecar和opa sidecar
kubectl label namespace default opa-istio-injection="enabled"
kubectl label namespace default istio-injection="enabled
-
部署bookinfo应用和sleep应用,可以看到pod里除了业务容器之外还有两个容器,分别是istio sidecar和OPA sidecar,OPA sidecar用于实现外部授权服务
-
最后,我们需要定义外部授权服务和授权策略(AuthorizationPolicy),其中外部授权服务就是我们注入的OPA sidecar,授权服务的地址总是127.0.0.1:9191;为了能够对OPA sidecar授权服务寻址,我们还需要定义一个ServiceEntry,如下:
apiVersion: networking.istio.io/v1beta1
kind: ServiceEntry
metadata:
name: ext-authz
namespace: opa-istio
spec:
hosts:
- "ext-authz.opa-istio.internal"
ports:
- number: 9191
name: grpc
protocol: GRPC
location: MESH_INTERNAL
resolution: STATIC
endpoints:
- address: 127.0.0.1
添加外部授权服务,可以根据业务请求body大小调整相关参数配置,如下:
定义AuthorizationPolicy授权策略,注意引用的外部授权服务需要和上面定义的外部授权服务名称一致,可以根据业务的需要执行OPA外部授权策略,如下示例对匹配标签app: productpage的:
apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
name: ext-authz
spec:
selector:
matchLabels:
app: productpage
action: CUSTOM
provider:
# The provider name must match the extension provider defined in the mesh config.
# You can also replace this with sample-ext-authz-http to test the other external authorizer definition.
name: opa-ext-authz-grpc
rules:
# The rules specify when to trigger the external authorizer.
- to:
- operation:
paths: ["/*"]
- 上面的配置中,我们通过workloadSelector指定对productpage应用进行访问授权,我们从sleep应用发起请求,访问productpage,此时采用默认OPA策略,请求总是被拒绝
kubectl exec $(kubectl get pod -l app=sleep -o jsonpath={.items..metadata.name} -n default) -c istio-proxy -n default -- curl http://productpage.default:9080/api/v1/products --user bob:password -o /dev/null -s -w '%{http_code}\n'
403
- 配置OPA策略:以下OPA策略定义了guest和admin角色,guest和admin可以使用GET方法访问/productpage路径,admin另外还可以使用GET方法访问/api/v1/products路径;外部访问时会先解析出用户名,获取用户角色,进一步获取用户访问权限,并与请求进行比对,满足条件则放过,否则拦截
apiVersion: opacontroller.k8s.io/v1
kind: OpaPolicy
metadata:
name: object-policy
namespace: default
spec:
policy: |-
package istio.authz
import input.attributes.request.http as http_request
import input.parsed_path
allow {
roles_for_user[r]
required_roles[r]
}
roles_for_user[r] {
r := user_roles[user_name][_]
}
required_roles[r] {
perm := role_perms[r][_]
perm.method = http_request.method
perm.path = http_request.path
}
user_name = parsed {
[_, encoded] := split(http_request.headers.authorization, " ")
[parsed, _] := split(base64url.decode(encoded), ":")
}
user_roles = {
"alice": ["guest"],
"bob": ["admin"]
}
role_perms = {
"guest": [
{"method": "GET", "path": "/productpage"},
],
"admin": [
{"method": "GET", "path": "/productpage"},
{"method": "GET", "path": "/api/v1/products"},
],
}
workloadSelector:
labels:
version: v1
- 验证:以bob的身份访问/api/v1/products和/productpage,由于bob是admin权限,两个路径都可以访问,返回200
kubectl exec $(kubectl get pod -l app=sleep -o jsonpath={.items..metadata.name} -n default) -c istio-proxy -n default -- curl http://productpage.default:9080/api/v1/products --user bob:password -o /dev/null -s -w '%{http_code}\
n'
200
kubectl exec $(kubectl get pod -l app=sleep -o jsonpath={.items..metadata.name} -n default) -c istio-proxy -n default -- curl http://productpage.default:9080/productpage --user bob:password -o /dev/null -s -w '%{http_code}\n'
200
以alice的身份访问/api/v1/products和/productpage,由于alice是guest权限,所以对/api/v1/products的访问会被拒绝,返回403;对/productpage的访问可以通过,返回200
kubectl exec $(kubectl get pod -l app=sleep -o jsonpath={.items..metadata.name} -n default) -c istio-proxy -n default -- curl http://productpage.default:9080/api/v1/products --user alice:password -o /dev/null -s -w '%{http_code
}\n'
403
kubectl exec $(kubectl get pod -l app=sleep -o jsonpath={.items..metadata.name} -n default) -c istio-proxy -n default -- curl http://productpage.default:9080/productpage --user alice:password -o /dev/null -s -w '%{http_code}\n'
200
- 修改OPA策略,给alice加上admin权限
user_roles = {
"alice": ["guest", "admin"],
"bob": ["admin"]
}
重新验证以alice身份访问/api/v1/products,返回200
kubectl exec $(kubectl get pod -l app=sleep -o jsonpath={.items..metadata.name} -n default) -c istio-proxy -n default -- curl http://productpage.default:9080/api/v1/products --user alice:password -o /dev/null -s -w '%{http_code}\n'
200
使用JWT认证授权
服务网格中使用JWT实现对请求的身份认证,进一步可以配置授权策略,限制对请求的授权。
首先部署测试应用
参考以上示例部署sleep和httpbin应用,pod列表如下:
配置JWT认证策略:
apiVersion: security.istio.io/v1beta1
kind: RequestAuthentication
metadata:
name: "jwt-example"
namespace: bookinfo
spec:
selector:
matchLabels:
app: httpbin
jwtRules:
- issuer: "testing@secure.istio.io"
jwks: '{ "keys":[ {"e":"AQAB","kid":"DHFbpoIUqrY8t2zpA2qXfCmr5VO5ZEr4RzHU_-envvQ","kty":"RSA","n":"xAE7eB6qugXyCAG3yhh7pkDkT65pHymX-P7KfIupjf59vsdo91bSP9C8H07pSAGQO1MV_xFj9VswgsCg4R6otmg5PV2He95lZdHtOcU5DXIg_pbhLdKXbi66GlVeK6ABZOUW3WYtnNHD-91gVuoeJT_DwtGGcp4ignkgXfkiEm4sw-4sfb4qdt5oLbyVpmW6x9cfa7vs2WTfURiCrBoUqgBo_-4WTiULmmHSGZHOjzwa8WtrtOQGsAFjIbno85jp6MnGGGZPYZbDAa_b3y5u-YpW7ypZrvD8BgtKVjgtQgZhLAGezMt0ua3DRrWnKqTZ0BJ_EyxOGuHJrLsn00fnMQ"}]}'
使用非法的JWT访问,可以看到返回了401错误:
kubectl exec "$(kubectl get pod -l app=sleep -n bookinfo -o jsonpath={.items..metadata.name})" -c sleep -n bookinfo -- curl "http://httpbin:8000/headers" -sS -o /dev/null -H "Authorization: Bearer invalidToken" -w "%{http_code}\n"
401
不带JWT头部时会放过请求(返回200):
kubectl exec "$(kubectl get pod -l app=sleep -n bookinfo -o jsonpath={.items..metadata.name})" -c sleep -n bookinfo -- curl "http://httpbin:8000/headers" -sS -o /dev/null -w "%{http_code}\n"
200
创建授权策略,要求请求带有合法的JWT才允许访问:
apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
name: require-jwt
namespace: bookinfo
spec:
selector:
matchLabels:
app: httpbin
action: ALLOW
rules:
- from:
- source:
requestPrincipals: ["testing@secure.istio.io/testing@secure.istio.io"]
再次不带JWT访问返回了403:
kubectl exec "$(kubectl get pod -l app=sleep -n bookinfo -o jsonpath={.items..metadata.name})" -c sleep -n bookinfo -- curl "http://httpbin:8000/headers" -sS -o /dev/null -w "%{http_code}\n"
403
使用已经生成好的token验证访问,可以看到返回了200状态码:
TOKEN= eyJhbGciOiJSUzI1NiIsImtpZCI6IkRIRmJwb0lVcXJZOHQyenBBMnFYZkNtcjVWTzVaRXI0UnpIVV8tZW52dlEiLCJ0eXAiOiJKV1QifQ.eyJleHAiOjQ2ODU5ODk3MDAsImZvbyI6ImJhciIsImlhdCI6MTUzMjM4OTcwMCwiaXNzIjoidGVzdGluZ0BzZWN1cmUuaXN0aW8uaW8iLCJzdWIiOiJ0ZXN0aW5nQHNlY3VyZS5pc3Rpby5pbyJ9.CfNnxWP2tcnR9q0vxyxweaF3ovQYHYZl82hAUsn21bwQd9zP7c-LS9qd_vpdLG4Tn1A15NxfCjp5f7QNBUo-KC9PJqYpgGbaXhaGx7bEdFWjcwv3nZzvc7M__ZpaCERdwU7igUmJqYGBYQ51vr2njU9ZimyKkfDe3axcyiBZde7G6dabliUosJvvKOPcKIWPccCgefSj_GNfwIip3-SsFdlR7BtbVUcqR-yv-XOxJ3Uc1MI0tz3uMiiZcyPV7sNCU4KRnemRIMHVOfuvHsU60_GhGbiSFzgPTAa9WTltbnarTbxudb_YEOx12JiwYToeX0DCPb43W1tzIBxgm8NxUg
kubectl exec "$(kubectl get pod -l app=sleep -n bookinfo -o jsonpath={.items..metadata.name})" -c sleep -n bookinfo -- curl "http://httpbin:8000/headers" -H "Authorization: Bearer $TOKEN" -sS -o /dev/null -w "%{http_code}\n"
200
修改授权策略,只允许JWT信息中groups字段包含group1的时候才允许访问,策略如下:
apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
name: require-jwt
namespace: bookinfo
spec:
selector:
matchLabels:
app: httpbin
action: ALLOW
rules:
- from:
- source:
requestPrincipals: ["testing@secure.istio.io/testing@secure.istio.io"]
when:
- key: request.auth.claims[groups]
values: ["group1"]
使用上述TOKEN访问返回403:
kubectl exec "$(kubectl get pod -l app=sleep -n bookinfo -o jsonpath={.items..metadata.name})" -c sleep -n bookinfo -- curl "http://httpbin:8000/headers" -H "Authorization: Bearer $TOKEN" -sS -o /dev/null -w "%{http_code}\n"
403
更新TOKEN
TOKEN= eyJhbGciOiJSUzI1NiIsImtpZCI6IkRIRmJwb0lVcXJZOHQyenBBMnFYZkNtcjVWTzVaRXI0UnpIVV8tZW52dlEiLCJ0eXAiOiJKV1QifQ.eyJleHAiOjM1MzczOTExMDQsImdyb3VwcyI6WyJncm91cDEiLCJncm91cDIiXSwiaWF0IjoxNTM3MzkxMTA0LCJpc3MiOiJ0ZXN0aW5nQHNlY3VyZS5pc3Rpby5pbyIsInNjb3BlIjpbInNjb3BlMSIsInNjb3BlMiJdLCJzdWIiOiJ0ZXN0aW5nQHNlY3VyZS5pc3Rpby5pbyJ9.EdJnEZSH6X8hcyEii7c8H5lnhgjB5dwo07M5oheC8Xz8mOllyg--AHCFWHybM48reunF--oGaG6IXVngCEpVF0_P5DwsUoBgpPmK1JOaKN6_pe9sh0ZwTtdgK_RP01PuI7kUdbOTlkuUi2AO-qUyOm7Art2POzo36DLQlUXv8Ad7NBOqfQaKjE9ndaPWT7aexUsBHxmgiGbz1SyLH879f7uHYPbPKlpHU6P9S-DaKnGLaEchnoKnov7ajhrEhGXAQRukhDPKUHO9L30oPIr5IJllEQfHYtt6IZvlNUGeLUcif3wpry1R5tBXRicx2sXMQ7LyuDremDbcNy_iE76Upg
再次访问返回200
kubectl exec "$(kubectl get pod -l app=sleep -n bookinfo -o jsonpath={.items..metadata.name})" -c sleep -n bookinfo -- curl "http://httpbin:8000/headers" -H "Authorization: Bearer $TOKEN" -sS -o /dev/null -w "%{http_code}\n"
200