facil.io
Utilities & Parsers

#HTTP/1.x Parser

c
#define FIO_HTTP1_PARSER
#include FIO_INCLUDE_FILE

Small parser, sharp teeth. Define FIO_HTTP1_PARSER to add the static HTTP/1.x request / response parser used by the HTTP layer. It performs no heap allocations, stores only parser state, and reports parsed data through callbacks implemented by the including translation unit.

Nearby context: [IO and HTTP overview](./400 io-overview.md), the higher-level [HTTP module header](./439 http.h), and the neighboring [WebSocket parser header](./004 websocket parser.h).

#What Gets Added

FIO_HTTP1_PARSER exposes:

  • fio_http1_parser_s — parser state.
  • FIO_HTTP1_PARSER_INIT — zero-initializer / reset value.
  • fio_http1_parse — incremental parser entry point.
  • parser state helpers:
    • fio_http1_parser_is_empty
    • fio_http1_parser_is_on_header
    • fio_http1_parser_is_on_body
    • fio_http1_expected
    • fio_http1_parser_skip_body
    • fio_http1_parser_skips_body
  • parse result / expected-body constants:
    • FIO_HTTP1_PARSER_ERROR
    • FIO_HTTP1_EXPECTED_CHUNKED
  • required user callbacks named fio_http1_on_*.

The implementation also declares internal parsing stages named with fio_http1___...; these are private implementation details.

#Parser State

#fio_http1_parser_s

c
typedef struct fio_http1_parser_s fio_http1_parser_s;

struct fio_http1_parser_s {
  int (*fn)(fio_http1_parser_s *, fio_buf_info_s *, void *);
  size_t expected;
  size_t flags;
};

The parser state is intentionally tiny: one function pointer for the current state-machine stage, one expected byte counter / sentinel, and a flags word for per-message state bits (e.g., an accepted Expect: 100-continue). Parser state is never inferred from function pointer identity, so linker identical-code-folding (MSVC / lld /OPT:ICF) cannot alter parsing behavior.

Treat all fields as opaque. Allocate the struct wherever it fits your lifetime (stack, connection object, arena, etc.), initialize it with FIO_HTTP1_PARSER_INIT, and use the helper functions to inspect state.

#FIO_HTTP1_PARSER_INIT

c
#define FIO_HTTP1_PARSER_INIT ((fio_http1_parser_s){0})

Zero-initializes a parser:

c
fio_http1_parser_s parser = FIO_HTTP1_PARSER_INIT;

The parser also resets itself to this empty state after a complete message is reported with fio_http1_on_complete.

#Parsing API

#fio_http1_parse

c
FIO_SFUNC size_t fio_http1_parse(fio_http1_parser_s *p,
                                 fio_buf_info_s buf,
                                 void *udata);

Parses as much HTTP/1.x data as currently possible and invokes callbacks as fields are discovered.

  • p is the parser state.
  • buf is the current readable bytes.
  • udata is passed unchanged to every callback.

Returns the number of bytes consumed from buf, or FIO_HTTP1_PARSER_ERROR ((size_t)-1) on parse / callback error.

A successful return may consume fewer bytes than supplied. Any unconsumed bytes belong to a later parse step or to the next HTTP message on a keep-alive connection.

If the parser needs more bytes before it can make progress, it returns successfully with the bytes consumed so far, which can be 0.

#State Helpers

c
FIO_IFUNC size_t fio_http1_parser_is_empty(fio_http1_parser_s *p);
FIO_IFUNC size_t fio_http1_parser_is_on_header(fio_http1_parser_s *p);
FIO_IFUNC size_t fio_http1_parser_is_on_body(fio_http1_parser_s *p);
FIO_IFUNC size_t fio_http1_expected(fio_http1_parser_s *p);
FIO_IFUNC void fio_http1_parser_skip_body(fio_http1_parser_s *p);
FIO_IFUNC size_t fio_http1_parser_skips_body(fio_http1_parser_s *p);
FIO_IFUNC int fio_http1_version_is_legacy(fio_buf_info_s version);
  • fio_http1_parser_is_empty returns non-zero when the parser is waiting for a new request / response line.
  • fio_http1_parser_is_on_header returns non-zero while reading regular headers or chunked trailer headers.
  • fio_http1_parser_is_on_body returns non-zero while reading a known-length body or while the chunked parser is ready to read the next chunk frame. During a split chunk payload, the internal chunk-read stage may report false even though body bytes are still being drained.
  • fio_http1_expected returns the parser's current expected byte count, returns FIO_HTTP1_EXPECTED_CHUNKED after a chunked body is detected and before the next chunk-size line is parsed, and returns 0 when the parser's internal state marks the message as having no body. During chunked payload delivery, it may expose the current chunk size / remaining chunk bytes.
  • fio_http1_parser_skip_body marks the current message as having no body, regardless of any content-length / transfer-encoding headers. Call it from a parser callback before the headers end (normally on_status), e.g., when parsing the response to a HEAD request — the parser can't know the request method. The mark is cleared when the message completes. See Messages Without a Body below.
  • fio_http1_parser_skips_body returns non-zero when the current message has no body. It is known as soon as the first line is parsed (before any header callback). Test it before reserving body space. See Messages Without a Body.
  • fio_http1_version_is_legacy returns non-zero if a version string (as passed to fio_http1_on_version) is exactly HTTP/1.0 (case-sensitive). HTTP/1.0 messages are not persistent without a keep-alive connection option and never use chunked transfer coding (RFC 9112 §9.3, §6.1). Every other version string gets HTTP/1.1 semantics. HTTP/0.9 is unsupported: its version-less request line is rejected, and a HTTP/0.9 version string is treated as HTTP/1.1 (i.e., host is required).

#Constants

c
#define FIO_HTTP1_PARSER_ERROR ((size_t)-1)
#define FIO_HTTP1_EXPECTED_CHUNKED ((size_t)(-2))

FIO_HTTP1_PARSER_ERROR is the error return value from fio_http1_parse. After a parse error, close / discard the stream state rather than attempting to recover the same parser instance.

FIO_HTTP1_EXPECTED_CHUNKED is the parser's sentinel after transfer-encoding: chunked is accepted and before a chunk-size line is parsed. Once chunk parsing starts, fio_http1_expected may instead report the current chunk size or remaining chunk bytes.

The header also defines FIO___HTTP1_BODY_NOT_ALLOWED as an internal sentinel for methods / states where a body should not be read. User code should rely on fio_http1_expected(p) == 0 instead of using that internal macro.

#Callback Contract

The parser declares these callbacks as static prototypes. The including translation unit must define them.

For portable user code, every int callback should return only 0 to continue or -1 to reject the parse.

The parser's internal checks are not identical for every callback:

  • fio_http1_on_method, fio_http1_on_url, fio_http1_on_version, fio_http1_on_status, and fio_http1_on_body_chunk treat any non-zero return as a parse error.
  • fio_http1_on_header during normal headers and fio_http1_on_header_content_length reject only an exact -1 return.
  • fio_http1_on_header during chunked trailers is passed through; negative values become parse errors and positive values stop the current parse as incomplete.
  • fio_http1_on_expect is special: any non-zero return rejects the expectation, resets the parser, and stops the current parse without calling fio_http1_on_complete.

All fio_buf_info_s values point into the buf memory passed to fio_http1_parse. They are not NUL-terminated unless the input happened to be. Copy or retain the data before the callback returns if it must outlive the input buffer.

Important: the parser lowercases header names in-place. Feed it writable memory, not a string literal or read-only mapping.

#Completion

c
static void fio_http1_on_complete(void *udata);

Called after the request / response line, headers, and any body have been fully parsed. The parser is reset before this callback is invoked, so it is ready for the next message on the same connection.

#Request Line Callbacks

c
static int fio_http1_on_method(fio_buf_info_s method, void *udata);
static int fio_http1_on_url(fio_buf_info_s path, void *udata);
static int fio_http1_on_version(fio_buf_info_s version, void *udata);

For request lines, the parser calls the callbacks in this order:

  1. fio_http1_on_method
  2. fio_http1_on_url
  3. fio_http1_on_version

The version slice is clamped to at most 14 bytes. GET, HEAD, and OPTIONS are recognized case-insensitively and mark the parser as not expecting a body; body-bearing headers for these methods conflict with that marker and are rejected.

#Response Line Callbacks

c
static int fio_http1_on_version(fio_buf_info_s version, void *udata);
static int fio_http1_on_status(size_t istatus,
                               fio_buf_info_s status,
                               void *udata);

For response lines, the parser calls fio_http1_on_version first and then fio_http1_on_status.

istatus is parsed from the numeric status token. status is the remaining status text slice after the numeric token. The version slice is clamped to at most 14 bytes.

The parser decides whether the first line is a response by checking whether the second token starts with a decimal digit.

Responses with a 1xx, 204, or 304 status are automatically marked as having no body (see Messages Without a Body). The parser does not skip interim 1xx responses: each one completes as a separate message, and callers decide whether to wait for the final response.

#Headers

c
static int fio_http1_on_header(fio_buf_info_s name,
                               fio_buf_info_s value,
                               void *udata);

static int fio_http1_on_header_content_length(fio_buf_info_s name,
                                              fio_buf_info_s value,
                                              size_t content_length,
                                              void *udata);

For ordinary headers, fio_http1_on_header receives:

  • name lowercased in-place.
  • value trimmed of leading and trailing spaces / tabs.
  • an empty value as { .buf = NULL, .len = 0 }.

Header names must contain a valid : separator and may not contain the forbidden characters encoded by the parser. NUL bytes in header values are rejected.

content-length is special:

  • empty values are rejected;
  • non-decimal / overflowing values are rejected;
  • values colliding with internal sentinels are rejected;
  • duplicate content-length headers must agree;
  • conflicting content-length and final transfer-encoding: chunked are rejected;
  • the first non-zero accepted value calls fio_http1_on_header_content_length instead of the generic header callback; a repeated matching value is accepted without calling the content-length callback again.

A content-length: 0 value marks the message as having no body and does not call fio_http1_on_header_content_length.

transfer-encoding is also special. Its field lines form a single coding list (RFC 9110 §5.3), and chunked (case-insensitive) must be the final coding, applied exactly once (RFC 9112 §6.3, §7). Anything else can't be framed safely, so it's rejected rather than reading the body as the next message (request smuggling):

  • the parser switches to chunked body decoding when a field line ends with the chunked token;
  • if the value is exactly chunked, no generic header callback is made;
  • if other transfer-coding text appears before the final chunked token, the final chunked token and adjacent separators are stripped before the remaining value is passed to fio_http1_on_header (i.e., gzip, chunked is framed as chunked and forwards transfer-encoding: gzip; decoding it is up to the parser user, since a 501 response is only a SHOULD under RFC 9112 §6.1);
  • a field line without a final chunked token (i.e., gzip) is forwarded unchanged; a later field line may still end the list with chunked;
  • rejected: a list that doesn't end with chunked once the headers end (i.e., gzip, identity, chunked, gzip, an empty value, or a chunked line followed by any other transfer-encoding line), a repeated chunked coding (chunked, chunked), malformed separators before the final chunked token, a content-length header, and requests using GET, HEAD, or OPTIONS.

Requests must carry exactly one host header, as required by RFC 9112 §3.2:

  • an HTTP/1.1 request without a host header is rejected once the headers end;
  • a request with more than one host header line is rejected (for any version);
  • HTTP/1.0 requests may omit host (see fio_http1_version_is_legacy);
  • responses aren't affected. The host header itself is passed to fio_http1_on_header.

expect is special when its value is exactly 100-continue. Any other Expect value is rejected.

#Messages Without a Body

The parser identifies bodyless messages from the first line, before any header callback runs, and reports them through fio_http1_parser_skips_body:

  • requests using GET, HEAD, or OPTIONS (a non-zero content-length on these is still rejected, as described above);
  • responses with a 1xx / 204 / 304 status (RFC 9112 §6.3);
  • messages marked with fio_http1_parser_skip_body (i.e., a HEAD response, marked from on_status).

fio_http1_parser_skips_body is also true after a content-length: 0 header.

A bodyless response completes at the empty line after its headers, and any bytes that follow belong to the next message. Body framing is ignored:

  • content-length is still validated and reported to fio_http1_on_header_content_length, but it is informational. Callbacks must test fio_http1_parser_skips_body before reserving space or enforcing body size limits (a HEAD response may describe a large body that never arrives);
  • transfer-encoding and expect are forwarded unchanged to fio_http1_on_header.

#Expect: 100-continue

c
static int fio_http1_on_expect(void *udata);

Called once, after headers, when an accepted Expect: 100-continue header requires a post-header decision and a body may follow (a Content-Length body or transfer-encoding: chunked). It is not called for messages that cannot carry a body (e.g., GET / HEAD / OPTIONS or a zero Content-Length), since there is nothing to continue (RFC 9110 §10.1.1). Return 0 to continue into the body / completion flow. Return non-zero to reset the parser and stop the current parse without calling fio_http1_on_complete.

#Body Chunks

c
static int fio_http1_on_body_chunk(fio_buf_info_s chunk, void *udata);

Called with decoded body bytes.

For Content-Length bodies, callback chunks follow the supplied input chunks and sum to the accepted content length.

For chunked bodies, framing bytes are removed before callback delivery. Large or split HTTP chunks may be delivered through more than one callback if the input arrives in smaller pieces.

#Parser Flow

text
start line
  ├─ request  -> method -> url -> version
  └─ response -> version -> status
headers
  ├─ no body / body not allowed -> complete   (incl. 1xx/204/304, skip_body)
  ├─ content-length body        -> body chunks -> complete
  └─ chunked body               -> chunk chunks -> trailers -> complete

Details worth keeping in mind:

  • Leading spaces, \r, and \n before the first line are skipped.
  • First lines shorter than the parser's minimum accepted shape are rejected.
  • NUL bytes in the first line are rejected.
  • Header and first-line parsing waits for a newline before making progress.
  • The parser accepts \n line endings and handles an optional preceding \r.
  • Chunk size lines are hexadecimal, capped by the implementation, and do not support chunk extensions.
  • A zero-size chunk either completes immediately when followed by an empty line or enters trailer parsing.

#Chunked Trailers

Allowed trailer headers are reported through fio_http1_on_header after the body's terminating zero-size chunk.

The parser rejects the following trailer names:

  • authorization
  • cache-control
  • content-encoding
  • content-length
  • content-range
  • content-type
  • expect
  • host
  • max-forwards
  • set-cookie
  • te
  • trailer
  • transfer-encoding

#Ownership and Lifetime

  • The parser allocates no memory.
  • The parser does not copy callback data.
  • The parser mutates header names in the input buffer to lowercase.
  • Callback slices are valid only while the input buffer remains valid and unchanged.
  • udata is never owned by the parser; it is simply forwarded.
  • The parser state may live inside a connection object and be reused for keep-alive messages. It resets automatically on complete messages.
  • After FIO_HTTP1_PARSER_ERROR, discard the parser / connection state.

#Minimal Skeleton

c
#define FIO_HTTP1_PARSER
#include FIO_INCLUDE_FILE

static int fio_http1_on_method(fio_buf_info_s method, void *udata) {
  (void)method;
  (void)udata;
  return 0;
}

static int fio_http1_on_url(fio_buf_info_s path, void *udata) {
  (void)path;
  (void)udata;
  return 0;
}

static int fio_http1_on_version(fio_buf_info_s version, void *udata) {
  (void)version;
  (void)udata;
  return 0;
}

static int fio_http1_on_status(size_t status_code,
                               fio_buf_info_s status,
                               void *udata) {
  (void)status_code;
  (void)status;
  (void)udata;
  return 0;
}

static int fio_http1_on_header(fio_buf_info_s name,
                               fio_buf_info_s value,
                               void *udata) {
  (void)name;
  (void)value;
  (void)udata;
  return 0;
}

static int fio_http1_on_header_content_length(fio_buf_info_s name,
                                              fio_buf_info_s value,
                                              size_t content_length,
                                              void *udata) {
  (void)name;
  (void)value;
  (void)udata;
  return content_length > (1UL << 20) ? -1 : 0;
}

static int fio_http1_on_expect(void *udata) {
  (void)udata;
  return 0;
}

static int fio_http1_on_body_chunk(fio_buf_info_s chunk, void *udata) {
  (void)chunk;
  (void)udata;
  return 0;
}

static void fio_http1_on_complete(void *udata) {
  (void)udata;
}

size_t parse_some_http(char *data, size_t len, void *udata) {
  fio_http1_parser_s parser = FIO_HTTP1_PARSER_INIT;
  return fio_http1_parse(&parser, FIO_BUF_INFO2(data, len), udata);
}

For real incremental parsing, keep fio_http1_parser_s with the connection and preserve / retry unconsumed bytes when fio_http1_parse returns less than the available buffer length.