Keep Headers._encoding when constructing from another Headers instance

[FatimahAlhumaidhi]

I’m trying to build a custom client that defaults header encoding to UTF-8 (related: #3238).

While looking at httpx.Headers.__init__, it seems that when you pass an existing Headers instance, it copies ._list but ignores ._encoding.

httpx/httpx/_models.py

Lines 151 to 164 in ae1b9f6

	if isinstance(headers, Headers):
	self._list = list(headers._list)
	elif isinstance(headers, Mapping):
	for k, v in headers.items():
	bytes_key = _normalize_header_key(k, encoding)
	bytes_value = _normalize_header_value(v, encoding)
	self._list.append((bytes_key, bytes_key.lower(), bytes_value))
	elif headers is not None:
	for k, v in headers:
	bytes_key = _normalize_header_key(k, encoding)
	bytes_value = _normalize_header_value(v, encoding)
	self._list.append((bytes_key, bytes_key.lower(), bytes_value))
	self._encoding = encoding

If I do Headers(existing_headers) (what BaseClient._merge_headers does), I expected it to behave like a copy, including keeping the same encoding; unless I explicitly pass encoding=.

I believe it should carry over its encoding (unless encoding is provided):

        self._encoding = None

        if isinstance(headers, Headers):
            self._list = list(headers._list)
            self._encoding = encoding if encoding is not None else headers._encoding
        ...
        self._encoding = encoding if encoding is not None else self._encoding

If this is intentional, what’s the recommended way to make a client default to UTF-8 header encoding without losing it when headers get copied around?

[jankesec]

You have identified a subtle behavior in httpx.Headers.__init__: when initializing a Headers instance from another Headers object, _list is shallow-copied but _encoding is not carried over unless explicitly passed via encoding=.

Here is a breakdown of why this happens, how to ensure UTF-8 header encoding today, and how it relates to client design:

---

1. Why _encoding is not copied by default

In HTTP specifications (RFC 7230 §3.2.4 and RFC 9110 §5.5), HTTP header fields are traditionally constrained to ASCII (or Latin-1 / ISO-8859-1 for legacy reasons). HTTPX defaults self._encoding = None (which falls back to ascii / latin-1 during wire serialization in httpcore) to prevent non-standard UTF-8 header values from causing protocol errors with strict HTTP/1.1 upstreams.

However, when a user explicitly instantiates Headers(..., encoding="utf-8"), copying the existing Headers instance (such as inside BaseClient._merge_headers) inadvertently drops the explicit encoding setting and reverts to None.

Preserving self._encoding = encoding if encoding is not None else getattr(headers, "_encoding", None) is a sensible consistency improvement for Headers.__init__.

---

2. How to ensure UTF-8 headers in custom Clients today

Until _encoding propagation is updated upstream, here are the most reliable patterns to guarantee UTF-8 header encoding:

Approach A: Pass Pre-encoded bytes in Headers

HTTPX Headers natively accepts bytes values for header names and values. If you encode the string to UTF-8 before passing it, HTTPX preserves the exact raw bytes over the wire regardless of _encoding:

headers = {
    b"X-Custom-Header": "tëst-vãlüé".encode("utf-8"),
}
client = httpx.Client(headers=headers)

Approach B: Custom Client overriding build_request

If you want strings to be automatically handled with UTF-8 encoding across all requests sent through the client, subclass httpx.Client (or httpx.AsyncClient) and re-apply encoding="utf-8" in build_request:

import httpx

class UTF8Client(httpx.Client):
    def build_request(self, *args, **kwargs) -> httpx.Request:
        request = super().build_request(*args, **kwargs)
        # Re-wrap request headers with explicit utf-8 encoding
        request.headers = httpx.Headers(request.headers, encoding="utf-8")
        return request

Approach C: Custom httpx.Auth or Request Event Hook

You can also attach an event hook to ensure all outgoing requests have their headers configured with encoding="utf-8":

def set_utf8_headers(request: httpx.Request):
    request.headers = httpx.Headers(request.headers, encoding="utf-8")

client = httpx.Client(event_hooks={"request": [set_utf8_headers]})

---

Summary

• Immediate solution: Using bytes values or overriding build_request ensures UTF-8 headers survive _merge_headers.
• Upstream: Preserving headers._encoding in Headers.__init__ when encoding is None is a clean and backwards-compatible proposal worth opening a PR for.

[otrumb]

There are two separate properties here:

1. Raw header bytes in _list.
2. The explicit decoding/encoding policy in _encoding.

Headers(existing_headers) preserves the first but not the second. It copies existing_headers._list, so already-encoded UTF-8 bytes are not re-encoded or lost. With valid non-ASCII UTF-8, autodetection then selects UTF-8 again, which makes the loss mostly invisible:

import httpx

source = httpx.Headers({"x": "é"}, encoding="utf-8")

with httpx.Client(
    headers=source,
    transport=httpx.MockTransport(lambda request: httpx.Response(204)),
) as client:
    request = client.build_request("GET", "https://example.test")

assert (b"x", b"\xc3\xa9") in request.headers.raw
assert request.headers.encoding == "utf-8"

This works unchanged in HTTPX 0.28.1 and current master. Therefore, for normal valid UTF-8 values already constructed through Headers(..., encoding="utf-8"), pre-encoding values, subclassing Client, or adding an event hook is unnecessary.

The real inconsistency is that explicit policy is not preserved. For example:

source = httpx.Headers({"x": "plain"}, encoding="utf-8")
copied = httpx.Headers(source)

assert copied.raw == source.raw
assert source.encoding == "utf-8"
assert copied.encoding == "ascii"

That distinction matters when bytes are ambiguous under multiple encodings. For example, UTF-8 bytes explicitly interpreted as ISO-8859-1 will be reinterpreted as UTF-8 after copying, although the raw bytes remain identical.

So preserving headers._encoding when encoding is None would make construction behave more like Headers.copy(). It changes policy propagation, not raw-byte preservation. Caveat: non-ASCII request-header semantics remain protocol/application-specific; preserving a chosen policy does not make arbitrary UTF-8 header values universally interoperable.