package Geo::Coder::Free;

use strict;
use warnings;
use autodie qw(:all);

use Carp;
use Config::Auto;
use Geo::Coder::Free::Local;
use Geo::Coder::Free::MaxMind;
use Geo::Coder::Free::OpenAddresses;
use Geo::Coder::Free::Utils qw(_abbreviate _normalize);
use Object::Configure;
use Params::Get;
use Readonly;
use Scalar::Util;

=head1 NAME

Geo::Coder::Free - Geocoding using free, locally-hosted databases

=head1 VERSION

Version 0.43

=cut

our $VERSION = '0.43';

=encoding utf-8

=head1 SYNOPSIS

    use Geo::Coder::Free;

    my $geo = Geo::Coder::Free->new();
    my $pt  = $geo->geocode(location => 'Ramsgate, Kent, UK');
    printf "%.6f, %.6f\n", $pt->lat(), $pt->long();

    # With OpenAddresses/WhoOnFirst data:
    my $geo2 = Geo::Coder::Free->new(openaddr => $ENV{OPENADDR_HOME});
    my $pt2  = $geo2->geocode(location => '1600 Pennsylvania Avenue NW, Washington DC, USA');

    # Free-text scanning:
    my @hits = $geo2->geocode(scantext => 'She grew up in Ramsgate, Kent.',
                              region   => 'GB');

=head1 DESCRIPTION

C<Geo::Coder::Free> translates addresses into latitude/longitude coordinates
using local SQLite databases built from free data sources - MaxMind/GeoNames,
OpenAddresses, Who's On First, OpenStreetMap, and dr5hn's countries/states/cities
database.  It deliberately avoids paid or rate-limited online geocoding services.
The module is designed to be flexible, supporting both command-line and programmatic usage.
It also includes a sample CGI script for a web-based geocoding service.

Geocoding dispatch order depends on whether C<OPENADDR_HOME> (or C<openaddr>) is set:

B<With OpenAddresses data:>

=over 4

=item 1. C<Geo::Coder::Free::OpenAddresses> - requires C<OPENADDR_HOME>

=item 2. C<Geo::Coder::Free::Local> - user-curated CSV entries (tried as fallback)

=item 3. C<Geo::Coder::Free::MaxMind> - bundled, always available

=back

B<Without OpenAddresses data:>

=over 4

=item 1. C<Geo::Coder::Free::MaxMind> only - Local is not consulted.

=back

The C<cgi-bin> directory contains a simple DIY geo-coding website:

    cgi-bin/page.fcgi page=query q=1600+Pennsylvania+Avenue+NW+Washington+DC+USA

The sample website is currently down while a new host is sought.
When it returns, you will be able to test it with:

    curl 'https://geocode.nigelhorne.com/cgi-bin/page.fcgi?page=query&q=1600+Pennsylvania+Avenue+NW+Washington+DC+USA'

=head1 LIMITATIONS

=over 4

=item * C<scantext> mode only finds locations in OpenAddresses; it falls back
silently when C<OPENADDR_HOME> is not set (B<FIXME>: should warn).

=item * The C<__DATA__> alternatives table is hard-coded; it should live in a
user-editable config file.

=item * The address-regex scantext path misses birth-year sentences such as
C<"She was born May 21, 1937 in Noblesville, IN."> because the regex requires
a preceding capital-letter word directly before the city.

=item * C<reverse_geocode> is only partially implemented; the MaxMind path does
not return meaningful results.

=item * The C<alternatives> map loop uses C<each %{$alt}>, which retains its
iterator position across calls.  After a successful match and early C<return>,
the next C<geocode()> call on the same input starts iterating from the key
B<after> the matched one, potentially missing the match entirely until C<each>
wraps around.  Workaround: call C<keys %{$alt}> once to reset the iterator
before iterating.

=back

=cut

# -----------------------------------------------------------------------
# Module-level singletons — initialised once, shared across all instances.
# Using 'our' so that test code can reset them between test runs if needed.
# -----------------------------------------------------------------------
our $alternatives;

# -----------------------------------------------------------------------
# Error-message table.  All user-facing strings live here so that a future
# i18n layer only needs to swap this hash, not touch every call site.
# -----------------------------------------------------------------------
my %_MESSAGES = (
	usage_geocode       => 'Usage: %s::geocode(location => $location|scantext => $text)',
	usage_reverse       => 'Usage: %s::reverse_geocode(latlng => "$lat,$long")',
	invalid_location    => '%s: invalid location to geocode(), %s',
	invalid_scantext    => '%s: invalid scantext to geocode(), %s',
	geocoding_failed    => '%s: geocoding failed',
	reverse_unsupported => 'Reverse lookup is not yet supported',
	use_arrow_new       => '%s: use ->new() not ::new() to instantiate',
	bad_ref_arg         => 'Usage: %s — do not pass a non-hash reference',
);

# Confidence thresholds for scantext results, expressed as named constants
# rather than magic numbers so call sites document intent.
Readonly::Scalar my $CONF_TRIPLET => 0.8;
Readonly::Scalar my $CONF_DUPLET  => 0.7;
Readonly::Scalar my $CONF_REGEX   => 0.7;

# Stopwords excluded from word-window scantext searches.
my %_COMMON_WORDS = map { $_ => 1 } qw(
	a age an and at be by cross for how i in is more of on or over pm
	road she side some the to was with
);

=head1 METHODS

=head2 new

=head3 SYNOPSIS

    my $geo = Geo::Coder::Free->new();
    my $geo = Geo::Coder::Free->new(openaddr => '/data/openaddr');
    my $geo = Geo::Coder::Free->new(directory => '/data/maxmind');

=head3 DESCRIPTION

Constructor.  Accepts a hash or hashref of options.  If called without
C<openaddr>, the module checks C<$ENV{OPENADDR_HOME}> before giving up.

If called on an existing object instance (C<< $clone = $geo->new() >>), returns
a B<shallow clone>.  All scalar fields are copied by value, but reference-type
fields (C<alternatives>, C<scantext_misses>, C<maxmind>, C<openaddr>) share the
same underlying object or hashref between the original and the clone.  Mutations
to those shared references are immediately visible in both objects.

=head3 API SPECIFICATION

=head4 input

    # Input schema (Params::Validate::Strict)
    openaddr  => { type => 'scalar', optional => 1 }                          # path to OpenAddresses/WOF data dir
    directory => { type => 'scalar', optional => 1 }                          # path to MaxMind/GeoNames files
    cache     => { type => 'object', optional => 1, can => ['get', 'set'] }   # CHI-compatible cache object

=head4 output

    # Output schema (Return::Set)
    { type => 'object', isa => 'Geo::Coder::Free' }

=head3 EXAMPLE

    use Geo::Coder::Free;

    # Minimal - uses only the bundled MaxMind data:
    my $geo = Geo::Coder::Free->new();

    # Full - also searches OpenAddresses/WOF:
    my $geo = Geo::Coder::Free->new(openaddr => $ENV{OPENADDR_HOME});

=head3 MESSAGES

    use ->new() not ::new()   Called as a function; use arrow syntax.

=cut

sub new {
	my $class = shift;

	my $params = Params::Get::get_params(undef, @_) // {};

	# Called as a function (Geo::Coder::Free::new) rather than a method
	if (!defined($class)) {
		if (keys %{$params}) {
			Carp::carp(_i18n('use_arrow_new', __PACKAGE__));
			return;
		}
		$class = __PACKAGE__;	# FIXME: only works with no arguments
	} elsif (Scalar::Util::blessed($class)) {
		return bless { %{$class}, %{$params} }, ref($class);
	}

	# Populate the alternatives map once from __DATA__ and cache it
	# across all instances.  Config::Auto parses the INI-like DATA block.
	if (!$alternatives) {
		my $keep = $/;
		local $/ = undef;
		my $data = <DATA>;
		$/ = $keep;

		$alternatives = Config::Auto->new(source => $data)->parse();
		# Config::Auto turns multi-value keys into arrayrefs; flatten them.
		while (my ($key, $value) = each %{$alternatives}) {
			$alternatives->{$key} = join(', ', @{$value});
		}
	}

	# Resolve OPENADDR_HOME before Object::Configure so plug-ins can see it
	if (!defined($params->{'openaddr'}) && $ENV{'OPENADDR_HOME'}) {
		$params->{'openaddr'} = $ENV{'OPENADDR_HOME'};
	}

	$params = Object::Configure::configure($class, $params);

	my $rc = {
		%{$params},
		maxmind      => Geo::Coder::Free::MaxMind->new($params),
		alternatives => $alternatives,
	};

	if ($params->{'openaddr'}) {
		$rc->{'openaddr'} = Geo::Coder::Free::OpenAddresses->new(id => 'md5', %{$params});
	}
	if (my $cache = $params->{'cache'}) {
		$rc->{'cache'} = $cache;
	}

	return bless $rc, $class;
}

=head2 geocode

=head3 SYNOPSIS

    # Standard lookup (returns a Geo::Location::Point or undef)
    my $pt = $geo->geocode(location => 'Ramsgate, Kent, UK');
    printf "lat=%.6f lon=%.6f\n", $pt->lat(), $pt->long();

    # Scantext - returns a list of Geo::Location::Point objects
    my @hits = $geo->geocode(
        scantext     => 'She lived in Ramsgate, Kent.',
        region       => 'GB',
        ignore_words => [qw(lived)],
    );

    # Invocation flexibility (all equivalent)
    $geo->geocode('Ramsgate, Kent, UK');
    $geo->geocode({ location => 'Ramsgate, Kent, UK' });
    $geo->geocode(location => 'Ramsgate, Kent, UK');

=head3 API SPECIFICATION

=head4 input

    # Input schema (Params::Validate::Strict) - exactly one of location or scantext is required
    location     => { type => 'scalar',   optional => 1 }  # address string (exclusive with scantext)
    scantext     => { type => 'scalar',   optional => 1 }  # free text to scan for place names
    region       => { type => 'scalar',   optional => 1 }  # ISO 3166-1 alpha-2 country code hint
    ignore_words => { type => 'arrayref', optional => 1 }  # words to suppress during scantext scan

=head4 output

    # Output schema (Return::Set)
    # scalar context: { type => 'object',   isa => 'Geo::Location::Point', optional => 1 }
    # list context:   { type => 'arrayref', of  => { isa => 'Geo::Location::Point' } }

=head3 MESSAGES

    Usage: ...::geocode(...)        No location or scantext argument given.
    invalid location to geocode()   location is purely numeric.
    invalid scantext to geocode()   scantext is purely numeric.

=head3 PSEUDOCODE

    if self is not a blessed object → delegate to new()->geocode(@args)
    normalise @_ into %params
    validate: location is not purely numeric; scantext is not purely numeric
    if openaddr backend is available:
        if scantext:
            try the raw scantext string as a direct location
            build stopword set from %_COMMON_WORDS + ignore_words param
            try 3-word windows (triplets) at confidence 0.8
            try 2-word windows (duplets) at confidence 0.7
            try the address-pattern regex at confidence 0.7
            try region-specific address finders (GB / US / CA)
            mark scantext as a miss; return undef
        else:
            try openaddr backend
            try local backend
            try __DATA__ alternatives map
    try maxmind backend for location lookups
    croak if no scantext and no location

=cut

sub geocode {
	my $self = shift;

	# Handle being called as a function rather than a method
	if (!ref($self)) {
		if (scalar @_) {
			return __PACKAGE__->new()->geocode(@_);
		} elsif (!defined($self)) {
			Carp::croak(_i18n('usage_geocode', __PACKAGE__));
		} elsif ($self eq __PACKAGE__) {
			Carp::croak(_i18n('usage_geocode', $self));
		}
		return __PACKAGE__->new()->geocode($self);
	} elsif (ref($self) eq 'HASH') {
		return __PACKAGE__->new()->geocode($self);
	}

	my %params = _normalize_args(_i18n('usage_geocode', __PACKAGE__), 'location', @_);

	# Reject pure-numeric inputs early — they are never valid addresses
	if (defined($params{'location'}) && $params{'location'} !~ /\D/) {
		Carp::croak(_i18n('invalid_location', __PACKAGE__, $params{'location'}))
			if length($params{'location'});
		return;
	}
	if (defined($params{'scantext'}) && $params{'scantext'} !~ /\D/) {
		Carp::croak(_i18n('invalid_scantext', __PACKAGE__, $params{'scantext'}))
			if length($params{'scantext'});
		return;
	}

	if ($self->{'openaddr'}) {
		if (my $scantext = $params{'scantext'}) {
			return if $self->{'scantext_misses'}{$scantext};

			# First try the whole scantext as a direct location lookup —
			# saves work when the text happens to be a valid address string.
			$self->{'local'} ||= Geo::Coder::Free::Local->new();
			my @direct = grep { defined }
				$self->{'local'}->geocode($scantext),
				$self->{'openaddr'}->geocode($scantext),
				$self->{'maxmind'}->geocode($scantext);
			return @direct if @direct;

			my $region = $params{'region'};

			# Build the effective stopword set.  When the caller supplies no
			# ignore_words we avoid copying %_COMMON_WORDS (which is immutable
			# at runtime) by using a reference to it directly.  Only when extra
			# words are needed do we allocate and populate a merged hash.
			my $ignore_words;
			if (my $iw = $params{'ignore_words'}) {
				my %merged = %_COMMON_WORDS;
				$merged{lc $_} = 1 for @{$iw};
				$ignore_words = \%merged;
			} else {
				$ignore_words = \%_COMMON_WORDS;
			}

			my @rc;

			# 3-word window search
			my @triplets = _find_word_ngrams($scantext, 3, $ignore_words);
			my $res = $self->_resolve_scan_candidates(\@triplets, $region, $CONF_TRIPLET, $scantext);
			if (@{$res}) {
				return wantarray ? @{$res} : $res->[0];
			}

			# 2-word window search
			my @duplets = _find_word_ngrams($scantext, 2, $ignore_words);
			$res = $self->_resolve_scan_candidates(\@duplets, $region, $CONF_DUPLET, $scantext);
			if (@{$res}) {
				return wantarray ? @{$res} : $res->[0];
			}

			# Regex-based address pattern — catches "City, ST"-style fragments.
			# Note: misses sentences like "born May 21, 1937 in Noblesville, IN"
			# because the pattern requires a capitalised word before the city.
			my @regex_matches = $scantext =~
				/\b(?:\d+\s+)?(?:[A-Z][a-z]+(?:\s+[A-Z][a-z]+)*\.?),\s*
				 (?:[A-Z][a-z]+(?:\s+[A-Z][a-z]+)*(?:,\s*[A-Z]{2,})*)\b/gx;
			my @places = grep { defined && $_ ne '' } @regex_matches;
			$res = $self->_resolve_scan_candidates(\@places, $region, $CONF_REGEX, $scantext);
			if (@{$res}) {
				return wantarray ? @{$res} : $res->[0];
			}

			# Region-specific structured address patterns
			if ($region) {
				my @candidates;
				if    ($region eq 'GB') { @candidates = _find_gb_addresses($scantext) }
				elsif ($region eq 'US') { @candidates = _find_us_addresses($scantext) }
				elsif ($region eq 'Canada') { @candidates = _find_ca_addresses($scantext) }

				if (@candidates) {
					my @regional;
					for my $candidate (@candidates) {
						next if $ignore_words->{lc $candidate};
						my @hits = grep { defined }
							$self->{'openaddr'}->geocode("$candidate, $region");
						push @regional, @hits if @hits;
					}
					return @regional if @regional;
				}
			}

			$self->{'scantext_misses'}{$scantext} = 1;
			return;
		}

		# Standard (non-scantext) lookup path
		if (wantarray) {
			my @rc = $self->{'openaddr'}->geocode(\%params);
			return @rc if @rc && $rc[0];
			$self->{'local'} ||= Geo::Coder::Free::Local->new();
			@rc = $self->{'local'}->geocode(\%params);
			return @rc if @rc && $rc[0];
		} else {
			if (my $rc = $self->{'openaddr'}->geocode(\%params)) {
				return $rc;
			}
			$self->{'local'} ||= Geo::Coder::Free::Local->new();
			if (my $rc = $self->{'local'}->geocode(\%params)) {
				return $rc;
			}
		}

		# Try the alternatives table — hand-curated mappings for locations that
		# the databases have under slightly different names.
		# M3 (transitive reduction): every scantext path inside this block returned
		# early above; at this point $params{'scantext'} is provably falsy, so the
		# former `if (!$params{'scantext'})` guard was a tautology — removed.
		if (my $alt = $self->{'alternatives'}) {
			my $location = $params{'location'};
			while (my ($key, $value) = each %{$alt}) {
				next unless $location =~ $key;
				(my $new_loc = $location) =~ s/$key/$value/;
				$params{'location'} = $new_loc;
				if (my $rc = $self->geocode(\%params)) {
					return $rc;
				}
				# Also try the comma-free variant ("Tyne and Wear" etc.)
				if ($value =~ /, /) {
					(my $flat = $value) =~ s/,//g;
					($new_loc = $location) =~ s/$key/$flat/;
					$params{'location'} = $new_loc;
					if (my $rc = $self->geocode(\%params)) {
						return $rc;
					}
				}
				# Restore for the next iteration
				$params{'location'} = $location;
			}
		}
	}

	# Final fallback: MaxMind for location lookups.
	# Scantext without OPENADDR_HOME will silently reach here and return undef;
	# a proper fix would warn here (see LIMITATIONS).
	# M8 (tautology elimination): both arms of the former wantarray ternary were
	# identical — collapsed to a single call; context propagates implicitly.
	if ($params{'location'}) {
		return $self->{'maxmind'}->geocode(\%params);
	}

	Carp::croak(_i18n('usage_geocode', __PACKAGE__)) unless $params{'scantext'};
	return;
}

=head2 reverse_geocode

=head3 SYNOPSIS

    my $loc = $geo->reverse_geocode(latlng => '51.3341,-1.4159');

=head3 DESCRIPTION

Translates a latitude/longitude pair back to a place name.
B<Partially implemented>: the MaxMind backend does not return meaningful results.
OpenAddresses is attempted first when available.

=head3 API SPECIFICATION

=head4 input

    # Input schema (Params::Validate::Strict) — latlng required
    latlng => { type => 'scalar' }  # "$lat,$long" comma-separated decimal degrees
    # NOTE: separate lat/lon/long keys are NOT supported at the Geo::Coder::Free
    # (facade) level.  When no OpenAddresses backend is configured, passing
    # lat/lon/long instead of latlng will croak "not yet supported".
    # To use separate coordinates call Geo::Coder::Free::Local::reverse_geocode.

=head4 output

    # Output schema (Return::Set)
    { type => 'object', isa => 'Geo::Location::Point', optional => 1 }

=cut

sub reverse_geocode {
	my $self = shift;

	if (!ref($self)) {
		if (scalar @_) {
			return __PACKAGE__->new()->reverse_geocode(@_);
		} elsif (!defined($self)) {
			Carp::croak(_i18n('usage_reverse', __PACKAGE__));
		} elsif ($self eq __PACKAGE__) {
			Carp::croak(_i18n('usage_reverse', $self));
		}
		return __PACKAGE__->new()->reverse_geocode($self);
	} elsif (ref($self) eq 'HASH') {
		return __PACKAGE__->new()->reverse_geocode($self);
	}

	my %params = _normalize_args(_i18n('usage_reverse', __PACKAGE__), 'latlng', @_);

	# M8 (tautology — same pattern fixed in geocode): both arms are identical;
	# context propagates implicitly through return.
	if ($self->{'openaddr'}) {
		return $self->{'openaddr'}->reverse_geocode(\%params);
	}
	if ($params{'latlng'}) {
		return $self->{'maxmind'}->reverse_geocode(\%params);
	}

	Carp::croak(_i18n('reverse_unsupported'));
}

=head2 ua

Does nothing.  Present for drop-in compatibility with other Geo::Coder::* modules.

=cut

sub ua { }

=head2 run

Command-line entry point.  Use as:

    perl lib/Geo/Coder/Free.pm 1600 Pennsylvania Avenue NW, Washington DC

=cut

__PACKAGE__->run(@ARGV) unless caller();

sub run {
	require Data::Dumper;

	my $class    = shift;
	my $location = join ' ', @_;

	my @rc = $ENV{'OPENADDR_HOME'}
		? $class->new(openaddr => $ENV{'OPENADDR_HOME'})->geocode($location)
		: $class->new()->geocode($location);

	Carp::croak(_i18n('geocoding_failed', $0)) unless @rc;

	print Data::Dumper->new([\@rc])->Dump();
}

# -----------------------------------------------------------------------
# Private helpers
# -----------------------------------------------------------------------

# Purpose:  Return a formatted message string from the message table.
#           Falls back to the raw key so call sites never die on a missing key.
# Entry:    $key — message key; @args — sprintf format arguments.
# Exit:     Formatted string.
sub _i18n {
	my ($key, @args) = @_;
	my $fmt = $_MESSAGES{$key} // $key;
	return @args ? sprintf($fmt, @args) : $fmt;
}

# Purpose:  Normalise the four calling conventions accepted by geocode/reverse_geocode:
#             hashref, even-length key-val list, odd-length list, single bare string.
# Entry:    $error_msg — message to croak if a non-hash reference is passed.
#           $bare_key  — hash key to assign to a single bare string argument
#                        ('location' for geocode, 'latlng' for reverse_geocode).
#           @args — raw @_ after $self has been shifted.
# Exit:     Flat key-value %params hash.
# Side Effects: None; croaks on unsupported reference argument.
sub _normalize_args {
	my ($error_msg, $bare_key, @args) = @_;
	return %{$args[0]}              if ref($args[0]) eq 'HASH';
	Carp::croak($error_msg)         if ref($args[0]);
	return @args                    if @args && @args % 2 == 0;
	return ($bare_key => $args[0])  if @args == 1;
	# M7 (logic-gap closure): odd-count > 1 is not a valid calling convention.
	# The former code silently returned () here, discarding the trailing elements.
	# Fail fast instead of propagating a malformed argument list downstream.
	Carp::croak($error_msg)         if @args;
	return ();
}

# Purpose:  Try to geocode each candidate place string via the openaddr backend,
#           annotating every hit with location/text/confidence metadata and
#           memoising misses to avoid re-querying the same failing string.
# Entry:    $self       — geocoder instance with {openaddr} and {scantext_misses}.
#           $candidates — arrayref of place-name strings to probe.
#           $region     — optional ISO country code appended to each candidate.
#           $confidence — numeric confidence score assigned to all hits.
#           $scantext   — original free text (stored inside each result object).
# Exit:     Arrayref of Geo::Location::Point objects; empty arrayref on no hits.
# Side Effects: Populates $self->{scantext_misses} for every non-matching place.
sub _resolve_scan_candidates {
	my ($self, $candidates, $region, $confidence, $scantext) = @_;
	my @results;
	for my $place (@{$candidates}) {
		my $location = $region ? "$place, $region" : $place;
		next if $self->{'scantext_misses'}{$location};
		my @res = grep { defined } $self->{'openaddr'}->geocode($location);
		if (@res) {
			for my $entry (@res) {
				$entry->{'location'}   = $location;
				$entry->{'text'}       = $scantext;
				$entry->{'confidence'} = $confidence;
			}
			push @results, @res;
		} else {
			$self->{'scantext_misses'}{$location} = 1;
		}
	}
	return \@results;
}

# Purpose:  Slide a window of $n words across $text and return all comma-joined
#           n-grams, excluding pure-numeric tokens and stopwords.
#           Replaces the former _find_word_triplets ($n=3) and
#           _find_word_duplets ($n=2) which were identical except for window size
#           and the duplets version was missing the lc() normalisation on stopwords.
# Entry:    $text — raw string; $n — window size; $stop — stopword hashref.
# Exit:     Flat list of n-gram strings.
sub _find_word_ngrams {
	my ($text, $n, $stop) = @_;
	$text =~ s/,+/ /g;
	$text =~ s/\s+/ /g;
	$text =~ s/^\s+|\s+$//g;
	my @words = grep { !/^\d+$/ && !$stop->{lc $_} } split /\s+/, $text;
	my @ngrams;
	for my $i (0 .. $#words - ($n - 1)) {
		push @ngrams, join ', ', @words[$i .. $i + $n - 1];
	}
	return @ngrams;
}

# Purpose:  Extract structured US street addresses from free text.
# Entry:    $text — arbitrary string.
# Exit:     List of full address strings matching the US pattern.
sub _find_us_addresses {
	my $text = shift;
	my @addresses;
	my $re = qr/
		\b \d{1,5} \s+                                      # house number
		(?:[A-Za-z0-9]+(?:\s+[A-Za-z0-9]+){0,6}) \s+       # street name: 1–7 words (bounded)
		(?:Avenue|Ave\.?|Boulevard|Blvd\.?|Road|Rd\.?|Lane|Ln\.?|Drive|Dr\.?|Street|St\.?)
		(?:\s+[A-Za-z]{2})? ,\s*
		(?:[A-Za-z]+(?:\s+[A-Za-z]+){0,3}) ,\s*            # city: 1–4 words
		[A-Z]{2} \s* (?:\d{5}(?:-\d{4})?)? \b              # state + optional zip
	/x;
	while ($text =~ /$re/g) {
		push @addresses, $&;
	}
	return @addresses;
}

# Purpose:  Extract British-style addresses from free text.
# Entry:    $text — arbitrary string.
# Exit:     List of trimmed address strings.
sub _find_gb_addresses {
	my $text = shift;
	my @addresses;
	# ReDoS fix: the former pattern used \s*,?\s* between all parts and
	# [A-Za-z\s'-]+ groups — with commas optional, the engine must try all
	# possible splits of a long string among 5 overlapping space-containing
	# groups, causing exponential backtracking.  Making commas mandatory
	# eliminates the ambiguity entirely: UK postal addresses always have commas.
	# All groups are non-capturing (only $& is used).
	my $re = qr/
		\b
		(?:\d{1,5} | [\w'-]+)                   # house number or single-word name
		\s+
		(?:[\w'-]+(?:\s+[\w'-]+){0,5})           # street name (up to 6 words)
		\s*,\s*                                   # COMMA REQUIRED — kills the ambiguity
		(?:[\w'-]+(?:\s+[\w'-]+){0,3})           # town (up to 4 words)
		\s*,\s*                                   # COMMA REQUIRED
		(?:[\w'-]+(?:\s+[\w'-]+){0,3})           # county (up to 4 words)
		\s*,\s*                                   # COMMA REQUIRED
		(?:[\w'-]+(?:\s+[\w'-]+){0,2})           # country (up to 3 words)
		\b
	/x;
	while ($text =~ /$re/g) {
		(my $addr = $&) =~ s/[,\s]+$//;
		push @addresses, $addr;
	}
	return @addresses;
}

# Purpose:  Extract Canadian street addresses from free text.
# Entry:    $text — arbitrary string.
# Exit:     List of full address strings matching the Canadian pattern.
sub _find_ca_addresses {
	my $text = shift;
	my @addresses;
	my $re = qr/
		\b \d{1,5} \s+                                         # house number
		(?:[A-Za-z0-9]+(?:\s+[A-Za-z0-9]+){0,6}) \s+          # street name: 1–7 words (bounded)
		(?:Avenue|Ave\.?|Boulevard|Blvd\.?|Road|Rd\.?|Lane|Ln\.?|Drive|Dr\.?|Street|St\.?|Circle|Crescent|Cres\.?)
		\s*,\s*
		(?:[A-Za-z]+(?:\s+[A-Za-z]+){0,3}) \s*,\s*            # city: 1–4 words
		[A-Z]{2} \s*,?\s*
		(?:[A-Z]\d[A-Z]\s?\d[A-Z]\d)? \b                      # optional postal code
	/x;
	while ($text =~ /$re/g) {
		push @addresses, $&;
	}
	return @addresses;
}

# _normalize and _abbreviate are imported from Geo::Coder::Free::Utils.

=head1 GETTING STARTED

To download, import and set up the local database:
before running C<make>, but after running C<perl Makefile.PL>, follow these instructions.

Optionally set C<OPENADDR_HOME> to point to an empty directory and download the data from
L<http://results.openaddresses.io> into that directory; and
optionally set C<WHOSONFIRST_HOME> to point to an empty directory and download the data using
L<https://github.com/nigelhorne/NJH-Snippets/blob/master/bin/wof-clone>.
The script C<bin/download_databases> (see below) will do those for you.
You do not need to download the MaxMind data — that is downloaded automatically.

You will need to create the database used by C<Geo::Coder::Free>.

Install L<App::csv2sqlite> and L<https://github.com/nigelhorne/NJH-Snippets>.
Run C<bin/create_sqlite> — this converts the MaxMind "cities" database from CSV to SQLite.

To use with MariaDB, set C<MARIADB_SERVER="$hostname;$port"> and
C<MARIADB_USER="$user;$password"> (TODO: username/password should be asked for interactively).
The code will use a database called C<geo_code_free>, which will be dropped and recreated if it exists.
C<$user> needs only DROP, CREATE, SELECT, INSERT, and INDEX privileges on that database.

The following optional steps download and install large databases.
This will take a long time and use a lot of disc space.

=over 4

=item 1

C<mkdir $WHOSONFIRST_HOME; cd $WHOSONFIRST_HOME> then run C<wof-clone> from NJH-Snippets.

This can take a long time because it contains many nested directories, which filesystem drivers
can be slow to navigate (particularly on EXT4 and ZFS).

=item 2

Install L<https://github.com/dr5hn/countries-states-cities-database.git> into C<$DR5HN_HOME>.
This data covers cities only, so it is not used when C<OSM_HOME> is set (OSM is far more
comprehensive).  Only Australia, Canada, and the US are imported, as the UK data is difficult
to parse.

=item 3

Run C<bin/download_databases> — this downloads the Who's On First, OpenAddr, OpenStreetMap,
and dr5hn databases.
OpenStreetMap now uses PBF files, so you will need C<apt install osmium-tool> first.
Check the values of C<OSM_HOME>, C<OPENADDR_HOME>, C<DR5HN_HOME> and C<WHOSONFIRST_HOME>
within that script and adjust them for your setup.
The C<Makefile.PL> file downloads the MaxMind database automatically, as it is not optional.

=item 4

Run C<bin/create_db> — this creates the database used by C<Geo::Coder::Free> from the data you
have just downloaded.
The database is called C<openaddr.sql> for historical reasons (before Who's On First was added);
it actually contains data from all sources above.

=back

Now you are ready to run C<make>.
See the comment at the start of C<createdatabase.PL> for further details.

=head1 MORE INFORMATION

I have written several Perl genealogy programs including
L<gedcom|https://github.com/nigelhorne/gedcom> and
L<ged2site|https://github.com/nigelhorne/ged2site>.
One of the things these do is check the validity of a family tree, including verifying place-names.
Of course places do change names and spelling becomes more consistent over the years, but the vast
majority remain the same — enough to make computerised verification worthwhile.

=head1 BUGS

Some lookups fail.  Please file a bug report at
L<https://rt.cpan.org/NoAuth/Bugs.html?Dist=Geo-Coder-Free>.

The MaxMind data contains cities only.
The OpenAddresses data does not cover the whole globe.
C<London, England> cannot be parsed yet.

=head1 SEE ALSO

=over 4

=item * L<Configure an Object at Runtime|Object::Configure>

=item * L<Test Dashboard|https://nigelhorne.github.io/Geo-Coder-Free/coverage/>

=back

L<Geo::Coder::Free::Local>, L<Geo::Coder::Free::MaxMind>,
L<Geo::Coder::Free::OpenAddresses>,
L<https://openaddresses.io/>, L<https://www.maxmind.com/>,
L<https://www.geonames.org/>, L<https://www.whosonfirst.org/>.

=head1 AUTHOR

Nigel Horne C<< <njh@nigelhorne.com> >>

=head1 FORMAL SPECIFICATION

=head2 new

    GeoCoderFreeState ::= ⟨⟨ maxmind     : MaxMind_Geocoder;
                              openaddr    : OpenAddr_Geocoder | undef;
                              alternatives: Map[STRING → STRING];
                              cache       : Cache | undef ⟩⟩

    Init : Params → GeoCoderFreeState
    ∀ p : Params •
      let oa_path == p.openaddr ∨ env.OPENADDR_HOME •
      GeoCoderFreeState.openaddr = if oa_path ≠ ∅ then OpenAddresses(oa_path) else undef fi

=head2 geocode

    Geocode : Address × Region? → Point?
    ∀ addr : Address; r : Region? •
      let backends == (openaddr ≠ undef ⟹ [OpenAddresses, Local, MaxMind])
                    ∧ (openaddr = undef ⟹ [MaxMind]) •
      result = first { defined } map { b.geocode(addr, r) } backends

=head1 LICENSE AND COPYRIGHT

Copyright 2017-2026 Nigel Horne.  Licensed under GPL2 for personal use.

This product uses GeoLite2 data created by MaxMind,
available from L<https://www.maxmind.com/>.

=cut

1;

# Common mappings for looser lookups.  A future version should load these from
# an external, user-editable config file.  See also Local.pm's %alternatives.
__DATA__
St Lawrence, Thanet, Kent = Ramsgate, Kent
St Peters, Thanet, Kent = Broadstairs, Kent
Minster, Thanet, Kent = Ramsgate, Kent
Tyne and Wear = Borough of North Tyneside
