架构简介
在Envoy的架构中,存在两个子系统listener与cluster。前者负责管理下游的事务,否则负责管理上游的事务。将两者连接在一起的,这是filter。其中,http filter是最重要的filter。
Envoy的设计核心理念是基于事件的线程模型。主线程负责管理服务的生命周期,配置处理,数据统计等。一些worker线程负责处理请求。所有的线程都围绕事件循环(event loop)进行操作,任意一个下游的tcp链接都将被一个worker线程处理。每个worker都有自己的连接池来与上游节点进行连接。UDP利用SO_REUSEPORT实现链接的网络四元组的一致性哈希,使UDP链接被哈希到同一worker线程。UDP filter的状态被负责处理UDP的worker共享,而tcp filter及基于tcp的filter,如http filter则不然,tcp的状态由每个连接自身维持。
请求流程
一个请求的全流程如下:
1. 一个来自下游的TCP链接被处于某个worker线程的listener接受;
2. listener filter链被创建,一个listener可以有多个filter链,主要用于SNI、相关等处理。一旦处理完成,listener将匹配一个network filter链,如HTTP connection manager。listener filter链可以与TLS关联,以解密被加密的数据;
3. network filter链被创建,如HTTP connection manager;
4. HTTP/2编解码器将TCP流解帧为独立的stream,每个stream处理一对request/response;
5. 对于每个HTTP stream,一个下游http filter被创建,其中最重要的是route filter,它必须位于HTTP filter链的末端。 route filter根据配置来选定请求要被路由到哪个cluster(route filter将从cluster获取HTTP connection pool);
6. cluster通过负载均衡策略选定最终的上游节点,其中还涉及到了断路器与健康检查等机制;若HTTP connection pool不存在存活的链接,则一个与上游节点的新链接将被建立;
7. 对于每个stream,一个上游HTTP filter链将被创建,默认情况下只有codec filter,它主要负责将请求编码给上游节点,并将上游节点的回包解码;
8. 处理与上游节点相关的TLS,而后将请求发送到上游节点;
9. 收到上游节点的回包后,回包以相反顺序依次经过HTTP filter,包括上游codec filter,route filter等filter,最终被发送到下游。而后,stream被销毁。最后,更新整个过程中产生的统计数据与日志。
配置样例
static_resources:
listeners:
# There is a single listener bound to port 443.
- name: listener_https
address:
socket_address:
protocol: TCP
address: 0.0.0.0
port_value: 443
# A single listener filter exists for TLS inspector.
listener_filters:
- name: "envoy.filters.listener.tls_inspector"
typed_config:
"@type": type.googleapis.com/envoy.extensions.filters.listener.tls_inspector.v3.TlsInspector
# On the listener, there is a single filter chain that matches SNI for acme.com.
filter_chains:
- filter_chain_match:
# This will match the SNI extracted by the TLS Inspector filter.
server_names: ["acme.com"]
# Downstream TLS configuration.
transport_socket:
name: envoy.transport_sockets.tls
typed_config:
"@type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.DownstreamTlsContext
common_tls_context:
tls_certificates:
- certificate_chain: {filename: "certs/servercert.pem"}
private_key: {filename: "certs/serverkey.pem"}
filters:
# The HTTP connection manager is the only network filter.
- name: envoy.filters.network.http_connection_manager
typed_config:
"@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
stat_prefix: ingress_http
use_remote_address: true
http2_protocol_options:
max_concurrent_streams: 100
# File system based access logging.
access_log:
- name: envoy.access_loggers.file
typed_config:
"@type": type.googleapis.com/envoy.extensions.access_loggers.file.v3.FileAccessLog
path: "/var/log/envoy/access.log"
# The route table, mapping /foo to some_service.
route_config:
name: local_route
virtual_hosts:
- name: local_service
domains: ["acme.com"]
routes:
- match:
path: "/foo"
route:
cluster: some_service
# CustomFilter and the HTTP router filter are the HTTP filter chain.
http_filters:
# - name: some.customer.filter
- name: envoy.filters.http.router
typed_config:
"@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router
clusters:
- name: some_service
# Upstream TLS configuration.
transport_socket:
name: envoy.transport_sockets.tls
typed_config:
"@type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.UpstreamTlsContext
load_assignment:
cluster_name: some_service
# Static endpoint assignment.
endpoints:
- lb_endpoints:
- endpoint:
address:
socket_address:
address: 10.1.2.10
port_value: 10002
- endpoint:
address:
socket_address:
address: 10.1.2.11
port_value: 10002
typed_extension_protocol_options:
envoy.extensions.upstreams.http.v3.HttpProtocolOptions:
"@type": type.googleapis.com/envoy.extensions.upstreams.http.v3.HttpProtocolOptions
explicit_http_config:
http2_protocol_options:
max_concurrent_streams: 100
- name: some_statsd_sink
# The rest of the configuration for statsd sink cluster.
# statsd sink.
stats_sinks:
- name: envoy.stat_sinks.statsd
typed_config:
"@type": type.googleapis.com/envoy.config.metrics.v3.StatsdSink
tcp_cluster_name: some_statsd_sink