Assume this artitecture
................................................
. .
. .
. .
. .
┌───────────────────┐ ┌─────────────────┐
┌──────────────────┤ Customer Firewall ├─┐ ┌────────────────────┤Provider Firewall├─┐
│ └──────────┬────────┘ │ │Provider Network └──┬──────────────┘ │
│ Customer Network │ │ │ ┌─────────┴─┐ │
│ │ │ │ ┌─────┤ Envoy - B ├───────┐ │
│ │ │ │ │ └─────┬─────┘ │ │
│ ┌──────┴─────┐ │ │ │ │ │ │
│ ┌─────────┐ │ │ │ │ │ │ │ │
│ │ Client ├────┤ Envoy - A │ │ │ ┌────┴────┐ ┌───┴─────┐ ┌────┴────┐ │
│ └─────────┘ │ │ │ │ │ Service │ │ Service │ │ Service │ │
│ └────────────┘ │ │ └─────────┘ └─────────┘ └─────────┘ │
│ │ │ │
└────────────────────────────────────────┘ └────────────────────────────────────────┘
The client wants to connect to services at the provider network, but the Customer Firewall has a strict policy on HTTP filtering and SNI filtering.
It will drop any unauthorized traffic, So any unallowed Domain/SNI will be dropped.
The SNI/Domain of Envoy B (envoy-b.example.com) is allowed in the firewall, but they don't Allow any other domain.
All services are HTTP/HTTPS
Here is the configuration for the envoy servers:
Envoy - A:
- It listens on port
80, and on every HTTP traffic, it will forward the traffic to Envoy - B using HTTP2 + TLSv1.3 - It listens on port
443, and on every TPC traffic, it will forward the traffic to Envoy - B without any further encryption
static_resources:
listeners:
# HTTP
- name: http_forwarder
address:
socket_address:
protocol: TCP
address: 0.0.0.0
port_value: 80
filter_chains:
- filters:
- name: envoy.filters.network.http_connection_manager
typed_config:
"@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
stat_prefix: ingress_http
route_config:
name: http_route
virtual_hosts:
- name: http_forwarder
domains: [ "*" ]
routes:
- match:
prefix: "/"
route:
cluster: forwarder_to_envoy_b_http
http_filters:
- name: envoy.filters.http.router
typed_config:
"@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router
# HTTPS/TLS
- name: https_forwarder
address:
socket_address:
protocol: TCP
address: 0.0.0.0
port_value: 443
filter_chains:
- filters:
- name: envoy.filters.network.tcp_proxy
typed_config:
"@type": type.googleapis.com/envoy.extensions.filters.network.tcp_proxy.v3.TcpProxy
cluster: forwarder_to_envoy_b_tls
stat_prefix: https_passthrough
access_log:
- name: envoy.access_loggers.stdout
typed_config:
"@type": type.googleapis.com/envoy.extensions.access_loggers.stream.v3.StdoutAccessLog
clusters:
- name: forwarder_to_envoy_b_http
type: STRICT_DNS
lb_policy: ROUND_ROBIN
load_assignment:
cluster_name: forwarder_to_envoy_b_http
endpoints:
- lb_endpoints:
- endpoint:
address:
socket_address:
address: envoy-b.example.com
port_value: 8888
dns_resolution_config:
resolvers:
- socket_address:
address: "1.1.1.1"
port_value: 53
dns_resolver_options:
no_default_search_domain: true
transport_socket:
name: envoy.transport_sockets.tls
typed_config:
"@type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.UpstreamTlsContext
common_tls_context:
tls_params:
tls_minimum_protocol_version: TLSv1_3
tls_maximum_protocol_version: TLSv1_3
- name: forwarder_to_envoy_b_tls
type: STRICT_DNS
lb_policy: ROUND_ROBIN
load_assignment:
cluster_name: forwarder_to_envoy_b_tls
endpoints:
- lb_endpoints:
- endpoint:
address:
socket_address:
address: envoy-b.example.com
port_value: 8889
dns_resolution_config:
resolvers:
- socket_address:
address: "1.1.1.1"
port_value: 53
dns_resolver_options:
no_default_search_domain: true
Envoy - B:
- It listens on port
8888, and on everyHTTP2 + TLSv1.3traffic, it will decrypt the traffic and forward the plain HTTP request to services - It listens on port
8889, and on every TPC traffic, it will forward the traffic to Envoy - B without any further encryption/decryption
static_resources:
listeners:
- name: http_listener_0
address:
socket_address:
protocol: TCP
address: 0.0.0.0
port_value: 8888
filter_chains:
- filters:
- name: envoy.filters.network.http_connection_manager
typed_config:
"@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
stat_prefix: ingress_http
route_config:
name: local_route
virtual_hosts:
- name: local_service
domains: [ "*" ]
routes:
- match:
prefix: "/"
route:
cluster: http_cluster
http_filters:
- name: envoy.filters.http.router
typed_config:
"@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router
transport_socket:
name: envoy.transport_sockets.tls
typed_config:
"@type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.DownstreamTlsContext
common_tls_context:
tls_params:
tls_minimum_protocol_version: TLSv1_3
tls_maximum_protocol_version: TLSv1_3
tls_certificates:
certificate_chain:
filename: /certs/proxy-crt.pem
private_key:
filename: /certs/proxy-key.pem
alpn_protocols: HTTP2
- name: tls_listener_0
address:
socket_address:
protocol: TCP
address: 0.0.0.0
port_value: 8889
filter_chains:
- filters:
- name: envoy.filters.network.tcp_proxy
typed_config:
"@type": type.googleapis.com/envoy.extensions.filters.network.tcp_proxy.v3.TcpProxy
cluster: tls_cluster
stat_prefix: https_passthrough
access_log:
- name: envoy.access_loggers.stdout
typed_config:
"@type": type.googleapis.com/envoy.extensions.access_loggers.stream.v3.StdoutAccessLog
clusters:
- name: http_cluster
type: STRICT_DNS
lb_policy: ROUND_ROBIN
load_assignment:
cluster_name: http_cluster
endpoints:
- lb_endpoints:
- endpoint:
address:
socket_address:
address: private.service.example.com
port_value: 80
dns_resolution_config:
resolvers:
- socket_address:
address: "1.1.1.1"
port_value: 53
dns_resolver_options:
no_default_search_domain: true
- name: tls_cluster
type: STRICT_DNS
lb_policy: ROUND_ROBIN
load_assignment:
cluster_name: tls_cluster
endpoints:
- lb_endpoints:
- endpoint:
address:
socket_address:
address: private.service.example.com
port_value: 443
dns_resolution_config:
resolvers:
- socket_address:
address: "1.1.1.1"
port_value: 53
dns_resolver_options:
no_default_search_domain: true
The problem
When the client sends an HTTP request, Envoy - A will encrypt it, and the SNI will be envoy-b.example.com which is allowed in the client firewall (instead of the HOST header of the original request private.service.example.com)
But when the client sends an HTTPS/TLS traffic, Envoy - A is just forwarding a TCP stream, so the SNI doesn't change and stays private.service.example.com -> As this hostname is not allowed in the firewall it will be dropped!
I want the traffic of the TCP proxy to get encrypted again so the SNI changes to envoy-b.example.com and the firewall doesn't drop the request.
I tried to add transport_socket for TLS traffic on both envoy hosts but it didn't work:
Envoy A > clusters > forwarder_to_envoy_b_tls:
transport_socket:
name: envoy.transport_sockets.tls
typed_config:
"@type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.UpstreamTlsContext
common_tls_context:
tls_params:
tls_minimum_protocol_version: TLSv1_3
tls_maximum_protocol_version: TLSv1_3
Envoy B > listener > tls_listener_0:
transport_socket:
name: envoy.transport_sockets.tls
typed_config:
"@type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.DownstreamTlsContext
common_tls_context:
tls_params:
tls_minimum_protocol_version: TLSv1_3
tls_maximum_protocol_version: TLSv1_3
tls_certificates:
certificate_chain:
filename: /certs/proxy-crt.pem
private_key:
filename: /certs/proxy-key.pem
alpn_protocols: HTTP2
Notes
- The client doesn't accept that envoy B converts HTTPS traffic to HTTP, then transfers it (using HTTPS), then again HTTPS encryption happens at Envoy A
- We don't have access to service private keys, so if we do what is explained in note 1, the certificates changes
- The Envoy B REQUIRES SNI to determine the traffic should forward to which services