package Fetch;

use 5.008003;
use strict;
use warnings;

our $VERSION = '0.20';

use File::Raw::JSON ();   # JSON encode/decode via its C ABI (ft_json.h / _abi_ptr)

require XSLoader;
XSLoader::load('Fetch', $VERSION);

require Fetch::Future;
require Fetch::Loop;
require Fetch::Loop::Standalone;
require Fetch::Response;
require Fetch::Headers;
require Fetch::CookieJar;
require Fetch::WebSocket;

1;

__END__

=head1 NAME

Fetch - HTTP/2 Future-based user agent

=head1 VERSION

Version 0.16

=head1 SYNOPSIS

    use Fetch;

    # works out of the box - no event loop to set up
    my $res = Fetch->new->get('https://example.com')->get;
    print $res->content if $res->is_success;

    # every call returns a Fetch::Future; ->get awaits it
    my $ua  = Fetch->new(timeout => 10);
    my $f   = $ua->post('https://api/things',
                        headers => { 'Content-Type' => 'application/json' },
                        body    => '{"name":"x"}');
    my $res = $f->get;

    # many requests concurrently on one loop
    my @futs = map { $ua->get($_) } @urls;
    Fetch::Future->needs_all(@futs)->get;
    my @bodies = map { $_->get->content } @futs;

    # a live WebSocket
    my $ws = $ua->websocket('wss://host/socket')->get;
    $ws->send('hello');
    my $reply = $ws->next_message->get;

=head1 DESCRIPTION

Fetch is an HTTP user agent whose socket, TLS, HTTP/2 framing and HTTP/1.1
parsing hot path lives in vendored C, and whose asynchronous results are
L<Fetch::Future> objects that compose with the Hyperman event loop and other
CPAN loops (IO::Async, AnyEvent) - or with nothing at all, since Fetch ships
its own event loop (L<Fetch::Loop::Standalone>) and uses it automatically.

HTTP/1.1 and HTTP/2 (ALPN-negotiated over TLS), over cleartext and TLS, with
keep-alive connection pooling, redirect following, per-request timeouts,
streaming response bodies, a cookie jar, JSON request/response helpers, and
native WebSockets. Every request method returns a
L<Fetch::Future>; C<< ->get >> on one awaits it, pumping whichever event loop
is active (its own if none), so the same code serves both a simple synchronous
call and thousands of requests multiplexed on one loop.

=head1 CONSTRUCTOR

=head2 new(%args)

    my $ua = Fetch->new(
        timeout    => 10,
        tls_verify => 1,
        headers    => { 'Accept' => 'application/json' },
        cookie_jar => 1,
    );

Create a user agent. All arguments are optional:

=over 4

=item C<loop>

The event loop to run on. Omit it and Fetch uses its own
L<Fetch::Loop::Standalone> (so C<< ->get >> just works with no framework). Pass
a raw L<IO::Async::Loop> or L<Hyperman::Loop> and it is wrapped automatically;
pass the string C<'AnyEvent'> to drive AnyEvent; or pass a ready-made
L<Fetch::Loop> adapter. See L</"EVENT LOOPS">.

Omitting it gives every such agent the B<same> loop - one implicit
L<Fetch::Loop::Standalone> per process, rebuilt automatically in a child after
a fork. Agents therefore multiplex: awaiting a request on one drives whatever
the others have in flight rather than stalling them. Pass an explicit loop to
opt out and get an isolated one.

=item C<headers>

Default headers sent on every request, as a hashref, an arrayref of
C<< name => value >> pairs (duplicates preserved), or a L<Fetch::Headers>.
Per-request C<headers> are merged on top (see L</"REQUEST OPTIONS">).

=item C<agent>

The C<User-Agent> string. Defaults to C<"Fetch/$VERSION">.

=item C<tls_verify>

Whether to verify the peer certificate and hostname for C<https>. Default true.
Overridable per request.

=item C<timeout>

Default per-request deadline in seconds (fractional allowed). C<0> (the
default) means no timeout. Overridable per request.

=item C<max_redirects>

How many redirects to follow. Default C<5>; C<0> disables following.
Overridable per request.

=item C<keep_alive>

Reuse connections via a keep-alive pool (default true). Set false to close
every connection after one request.

=item C<pool_size>

Maximum idle connections the keep-alive pool parks (default C<32>).

=item C<cookie_jar>

A L<Fetch::CookieJar> to store and send cookies (applied across redirects), or
a true scalar to create a fresh one. Default: no jar.

=item C<simple_response>

Resolve requests to a plain unblessed hashref C<< { status => ..., headers =>
[k, v, ...], content => ... } >> instead of a blessed L<Fetch::Response>. Read
fields directly (C<< $res->{status} >>, C<< $res->{content} >>) rather than via
methods. This skips the response object's method dispatch on the hot path; the
saving is small (a couple of percent) and only shows when you actually read the
response, so reach for it when you are consuming millions of responses and want
the leanest possible per-response cost. Default false.

=back

=head1 REQUEST METHODS

Each returns a L<Fetch::Future> that resolves to a L<Fetch::Response> (or fails
with an error string). They never block; call C<< ->get >> on the future to
await the result.

=head2 get / head / delete

    my $f = $ua->get($url, %opt);

=head2 post / put

    my $f = $ua->post($url, body => $bytes, %opt);

=head2 request($method, $url, %opt)

    my $f = $ua->request('PATCH', $url, body => $bytes, %opt);

The general form the verb helpers dispatch to; use it for any method.

=head1 REQUEST OPTIONS

Passed as a trailing C<< key => value >> list to any request method:

=over 4

=item C<headers>

Extra headers for this request - a hashref, an arrayref of pairs (keeping
duplicate names, e.g. multiple C<X-*> values), or a L<Fetch::Headers>. Each
named field overrides the agent default of the same name.

=item C<body>

The request body (bytes). Content-Length is added automatically unless you
set it yourself.

=item C<json>

A Perl data structure to send as a JSON body: it is encoded and
C<Content-Type: application/json> is set (unless you gave your own). Takes
precedence over C<body>. Pair it with L<Fetch::Response/json> to decode the
reply. Encoding uses L<Cpanel::JSON::XS> when installed, else core L<JSON::PP>.

    my $res  = $ua->post($url, json => { name => 'x', ok => \1 })->get;
    my $data = $res->json;

=item C<timeout>

Override the agent timeout for this request (seconds; C<0> disables it).

=item C<tls_verify>

Override certificate/hostname verification for this C<https> request.

=item C<max_redirects>

Override how many redirects this request follows.

=item C<on_body>

A coderef called with each body chunk as it arrives, instead of buffering.
Suits large downloads and server-sent events: the buffer is compacted so an
endless stream does not grow memory, and the resolved response body is empty.

    $ua->get($url, on_body => sub { my ($chunk) = @_; print $chunk })->get;

=item C<on_headers>

A coderef called once, as soon as the response status line and headers have
been parsed - before any C<on_body> chunk - with the status code and the header
list:

    $ua->get($url,
        on_headers => sub { my ($status, $headers) = @_; ... },  # $headers: [k,v,...]
        on_body    => sub { my ($chunk) = @_; ... },
    )->get;

This lets a streaming consumer act on the status and headers up front (for
example, to open a downstream writer) rather than waiting for the whole
response. Not fired for a WebSocket upgrade.

=back

=head1 WEBSOCKETS

=head2 websocket($url, %opt)

    my $ws = $ua->websocket('ws://host/echo')->get;   # or wss://

Open a WebSocket (RFC 6455). Returns a L<Fetch::Future> that resolves, after
the C<101> handshake, to a L<Fetch::WebSocket> for sending and receiving
messages. Accepts C<ws://>/C<wss://> (and C<http>/C<https>); C<tls_verify> and
C<timeout> options apply to the handshake.

=head2 clone(%overrides)

    my $shared  = Fetch->new;                        # built once
    my $scoped  = $shared->clone(cookie_jar => 1);   # its own jar, same pool

Another agent over this one's connection pool and event loop, with the given
options replaced. Everything not named is inherited.

The case it exists for is a B<per-request cookie jar>. A jar belongs to the
agent, so a jar on a long-lived agent is shared by every request that agent
serves: fine when the cookies authenticate the application itself, a
cross-request leak the moment they identify an end user. Building a whole fresh
agent per request avoids the leak but throws away the keep-alive pool and
re-resolves the loop adapter, which is most of what C<new> costs. A clone gives
you the isolation without the bill.

C<cookie_jar>, C<headers>, C<agent>, C<timeout>, C<tls_verify>,
C<max_redirects>, C<keep_alive> and C<simple_response> can be overridden;
C<< cookie_jar => undef >> drops an inherited jar. Headers are copied rather
than shared, so a clone cannot write into the parent's defaults.

C<loop> and C<pool_size> cannot be overridden and croak if given: they are what
the clone shares, and an agent on a different loop driving the parent's parked
connections would be reaching into the wrong one.

A clone holds references to the shared pool and loop, so they live until the
last agent using them goes. Keep the parent alive for as long as its clones, as
you would anyway.

=head1 ACCESSORS

=head2 loop

The event-loop adapter this agent runs on.

=head2 cookie_jar

The L<Fetch::CookieJar> in use, or undef.

=head1 OBSERVING OUTBOUND REQUESTS

=head2 Fetch->on_request(\&start, \&done)

Watch every request this B<process> makes, including ones it did not write.

    Fetch->on_request(
        sub {
            my ($method, $url, $headers) = @_;
            push @$headers, traceparent => current_span_id();
            return { at => time, url => $url };        # the token
        },
        sub {
            my ($token, $res, $err) = @_;
            record($token->{url}, time - $token->{at},
                   $err ? "failed: $err" : $res->status);
        },
    );

C<start> fires just before a request goes out, with the merged header list
still B<mutable>: push a name and a value onto C<$headers> and the request
carries them. That is the point of the hook - L</REQUEST OPTIONS>' own
C<headers> is set by whoever made the call, which is no use to a tracing
layer that has to annotate a request the application wrote.

Whatever C<start> returns is the B<token>, handed back to C<done> when that
same request settles. It is an ordinary Perl scalar - a hashref of what you
want to remember is the usual thing - so the two halves correlate without a
lookup table keyed on something that might repeat.

C<done> fires exactly once for every C<start>, with exactly one of C<$res>
(a L<Fetch::Response>) and C<$err> (the failure message). That includes a
timeout, a refused connection and a DNS failure, which are the endings an
instrumented client most wants and the easiest to leak. C<done> is optional.

Three things to know, all of which are the contract rather than an oversight:

=over 4

=item *

B<Per hop, not per call.> A redirect chain is several requests, each of which
went somewhere, and each is observed. One C<< ->get >> that follows two
redirects fires C<start> three times.

=item *

B<Registration is process-global and permanent.> Not per agent: an agent is a
per-worker object and a process may hold several, while an observer is a
property of the process. There is no deregistration, so register at boot -
and note that the callbacks (and everything they close over) live as long as
the process does.

=item *

B<A death is warned, not propagated.> An observer is a bystander, and a
broken bystander must not be able to fail the request it was watching. If
either callback dies, the death becomes a warning naming which half it came
from, and the request carries on - without whatever the callback had not yet
done, such as the header it died before pushing.

=back

Returns 1, or 0 when the observer table is full (C<FETCH_ABI_MAX_OBSERVERS>,
8). Croaks if the callbacks are not code references, at registration, where
the mistake is.

This costs one Perl call per hop, paid only by a process that asked for it: a
request with no observer registered pays a single branch. Where that is too
much - a proxy or a server instrumenting every request it forwards - the same
registry takes C function pointers through the C ABI's L</The table>
C<on_request>, which is the same contract with no Perl on the path.

=head1 THE RESULT

Awaiting a request future yields a L<Fetch::Response> with C<status>,
C<headers> (a L<Fetch::Headers>), C<content>, C<header($name)>, C<is_success>
and C<is_redirect>. A failed request (connection error, timeout, bad TLS, a
rejected WebSocket upgrade) fails the future with a message; C<< ->get >>
rethrows it, or inspect C<< $f->failure >> / C<< $f->is_failed >>.

=head1 EVENT LOOPS

C<< $future->get >> awaits by pumping the active loop, so a bare synchronous
call needs no setup. Hand Fetch a loop to cooperate with an existing async
program - requests then fly concurrently on that one loop without blocking it:

    use IO::Async::Loop;
    my $loop = IO::Async::Loop->new;
    my $ua   = Fetch->new(loop => $loop);       # shares this loop
    my $f    = $ua->get('https://example.com/');
    $loop->loop_once until $f->is_ready;
    my $res  = $f->get;

Supported loops: the built-in L<Fetch::Loop::Standalone>, plus
L<Fetch::Loop::IOAsync>, L<Fetch::Loop::AnyEvent> and L<Fetch::Loop::Hyperman>.

A request future remembers the loop it was issued on, and every future derived
from it by C<then>/C<followed_by>/C<transform> inherits that. So C<< ->get >>
always pumps the loop that owns the socket, however many agents and loops the
program has, and in whatever order they were built. Awaiting a future on a loop
that was never going to resolve it - one with no watchers and no timers - dies
with a diagnostic rather than blocking in the kernel forever.

=head1 C ABI

Fetch exposes a small C ABI so another XS module can drive it from C, with no
per-request Perl round trip on the hot path. It is how L<Reverse::Proxy> builds
its upstream requests entirely in C. This is not part of the Perl API - if you
are writing Perl, use the request methods above; the ABI is only for XS
consumers.

The ABI is a versioned function pointer table. A consumer vendors a copy of the header C<fetch_abi.h> (shipped
in this distribution under C<include/fetch/>) at a pinned C<FETCH_ABI_VERSION>, then at boot resolves the table and checks the version:

    #include "fetch_abi.h"   /* vendored, after the perl.h includes */

    /* in BOOT: */
    IV p = 0;
    if (call_pv("Fetch::_abi_ptr", G_SCALAR) > 0) { SPAGAIN; p = POPi; PUTBACK; }
    const fetch_abi *FETCH = NULL;
    if (p) {
        const fetch_abi *a = INT2PTR(const fetch_abi *, p);
        if (a && a->abi_version >= FETCH_ABI_VERSION) FETCH = a;
    }
    /* FETCH == NULL means the running Fetch is too old; the consumer decides
     * whether that is a hard error or a Perl fallback. The check is >=, never
     * ==: the table only grows at the end, so a newer Fetch still holds every
     * entry the consumer was written against. */

=head2 Fetch::_abi_ptr

Returns the address of Fetch's C<fetch_abi> table as an C<IV>. Call it once, at
boot, and C<INT2PTR> the result to a C<const fetch_abi *>. A version mismatch
must never be treated as a crash - compare C<abi_version> first and fall back.

=head2 The table

C<fetch_abi> (see C<fetch_abi.h> for the exact signatures and ownership rules)
holds, after C<abi_version>:

=over 4

=item C<ua_new(kv, nkv)>

Construct a Fetch user agent from C<nkv> flat key/value SVs (the same options
C<new> takes: C<loop>, C<pool_size>, C<tls_verify>, C<timeout>, C<agent>,
C<headers>, C<cookie_jar>, C<keep_alive>, C<max_redirects>,
C<simple_response>). Returns the blessed Fetch UA SV (+1 owned). A consumer may
instead call C<< Fetch->new >> from Perl once and cache the object; C<ua_new>
just removes that last Perl call.

=item C<request(ua_sv, method, url, hdrs, nhdrs, body, blen, timeout, max_redirects, map, ud)>

Issue one HTTP request on C<ua_sv>, building it entirely from C. The headers
are a flat C<fetch_hdr> array; C<max_redirects> below zero means "use the UA
default". Returns a L<Fetch::Future> SV (+1 owned) - hand it to an awaiting
server or call C<< ->get >> on it. When the request settles, your C<map>
callback shapes the value the future resolves to (for a proxy, a PSGI
C<[ status, \@headers, \@body ]>).

=item C<res_parts(res, status, headers, body)>

Pull the status, the flat header AV and the content SV out of an
already-resolved response (a L<Fetch::Response> or a C<simple_response> hash)
with no method dispatch. Any out-pointer may be C<NULL>; the returned C<headers>
and C<body> are borrowed. For the blocking (awaited) path.

=item C<request_stream(ua_sv, method, url, hdrs, nhdrs, body, blen, timeout, max_redirects, on_headers, on_body, on_done, ud)>

Like C<request>, but streams the response instead of buffering it: C<on_headers>
fires once up front with the status and header AV, C<on_body> once per body
chunk, and C<on_done> at completion (with success/failure). Returns the request
future SV (+1 owned) - keep it alive until C<on_done> fires. Lets a consumer
forward a large download or an endless SSE stream with flat memory.

=item C<tunnel_connect(host, port, tls, verify)> and friends

A raw blocking upstream TCP connection Fetch owns, for a proxy's
Upgrade/WebSocket tunnel where the consumer splices bytes both ways itself.
When C<tls> is true the connection reuses Fetch's own client C<SSL_CTX> (SNI,
and hostname/certificate verification when C<verify> is true), so the consumer
tunnels to a C<wss>/C<https> upstream without linking OpenSSL. This is what lets
L<Reverse::Proxy> tunnel to a TLS upstream.

Unlike the rest of the table, these six entries are pure C - they take no
C<pTHX> and touch no SV, so they can be called directly from inside a
C<select()> splice loop:

=over 4

=item *

C<tunnel_connect(host, port, tls, verify)> - open the connection; returns an
opaque handle, or C<NULL> on failure (DNS, connect, or TLS handshake).

=item *

C<tunnel_fd(conn)> - the underlying socket fd, to hand to C<select()>/C<poll()>.

=item *

C<tunnel_read(conn, buf, len)> - read bytes; returns the count (E<gt>0), C<0> at
EOF, or C<-1> on error.

=item *

C<tunnel_write_all(conn, buf, len)> - write the whole buffer; returns C<0> when
all bytes are written, C<-1> on error.

=item *

C<tunnel_pending(conn)> - bytes already buffered inside the TLS layer; drain
these before trusting C<select()> readiness (always C<0> for a plain
connection).

=item *

C<tunnel_close(conn)> - shut the connection down and free the handle.

=back

=item C<on_request(start, done, ud)>  I<(v2)>

Observe outbound requests.

    static void *start(pTHX_ const char *method, STRLEN mlen,
                       const char *url, STRLEN ulen, AV *headers, void *ud);
    static void  done(pTHX_ void *token, SV *res, SV *err, void *ud);

C<start> fires once per B<hop> with the merged header list still B<mutable> -
push a name and a value onto C<headers> and the request carries them. That is
what the hook is for: L</REQUEST OPTIONS>' C<headers> is set by the caller,
which is no use to anything that wants to annotate a request the application
wrote. It returns an opaque token, handed back to C<done> when that same hop
settles, so the two correlate with no lookup table on the hot path.

C<done> fires exactly once for every C<start>, with exactly one of C<res> and
C<err> - including a timeout, a refused connection and a DNS failure. Both are
borrowed.

Per hop, not per call: a redirect chain is several requests, each of which went
somewhere, and anything measuring them should see all of them.

Registration is process-global, not per agent, and there is no deregistration;
register at boot. Neither callback may croak. Returns 1, or 0 when the table is
full (C<FETCH_ABI_MAX_OBSERVERS>). A request with no observer registered pays
one branch and allocates nothing.

L</Fetch-E<gt>on_request(\&start, \&done)> is the same registry reached from
Perl, for a consumer that is not an XS module: it registers a pair of shims
holding coderefs. Same contract, one Perl call per hop instead of none, and
the one rule C could state and Perl cannot enforce - "neither may croak" -
becomes "a death is warned and the request carries on".

=item C<tunnel_starttls(conn, host, verify)>  I<(v3)>

Upgrade a tunnel opened with C<tls> false to TLS, after the application
protocol has negotiated it - SMTP's C<STARTTLS>, where the connection starts
in plaintext, the server says C<220>, and only then does the handshake begin.
The handshake is the one C<tunnel_connect> performs when C<tls> is true: the
same client C<SSL_CTX>, SNI, and hostname and certificate verification when
C<verify> is true.

Returns C<0> on success, after which C<tunnel_read>, C<tunnel_write_all> and
C<tunnel_pending> speak TLS. Returns C<-1> when C<conn> is C<NULL> or already
TLS, when this build of Fetch has no TLS, or when the handshake fails; after a
failed handshake the socket is in an undefined state and the caller should
C<tunnel_close> it. Pure C, no C<pTHX>, like the rest of the tunnel.

The tunnel carries no timeout of its own. A consumer that cannot wait forever
on a peer sets C<SO_RCVTIMEO> and C<SO_SNDTIMEO> on C<tunnel_fd(conn)>; the
handshake and every later read honour them.

=back

=head1 SEE ALSO

L<Fetch::Response>, L<Fetch::Headers>, L<Fetch::CookieJar>, L<Fetch::WebSocket>,
L<Fetch::Future>, L<Fetch::Loop> and the loop adapters
L<Fetch::Loop::Standalone>, L<Fetch::Loop::IOAsync>, L<Fetch::Loop::AnyEvent>,
L<Fetch::Loop::Hyperman>.

=head1 AUTHOR

LNATION C<< <email@lnation.org> >>

=head1 LICENSE AND COPYRIGHT

This software is Copyright (c) 2026 by LNATION.

This is free software, licensed under the Artistic License 2.0.

=cut
