Skip to content

feat: add asyncio support with HTTPX - #43

Closed
Joseph-Mutua wants to merge 1 commit into
serpapi:masterfrom
Joseph-Mutua:feat/async-httpx-client
Closed

Joseph-Mutua wants to merge 1 commit into
serpapi:masterfrom
Joseph-Mutua:feat/async-httpx-client

Conversation

@Joseph-Mutua

@Joseph-Mutua Joseph-Mutua commented Sep 16, 2026

Copy link
Copy Markdown

Summary

  • replace the Requests transport with persistent httpx.Client / httpx.AsyncClient transports while preserving the existing synchronous serpapi.Client and module-level helpers
  • add serpapi.AsyncClient support for search, archive, account, locations, image upload, concurrent calls, and async pagination
  • add explicit close() / aclose() and sync/async context managers, shared parameter behavior, redirects, timeout and error translation, and compatibility handling for the existing per-call proxy/TLS options
  • document pip3 and uv installation, async lifecycle and concurrency, and include a deterministic local transport benchmark
  • align package metadata with HTTPX and the existing CI matrix by requiring Python 3.8+

Fixes #27

Compatibility notes

The synchronous call pattern remains unchanged. timeout=None keeps the SDK's historical no-timeout behavior, redirects remain enabled, pagination URLs retain existing query parameters, and SerpApi exceptions retain status_code, error, request, and response details.

HTTPX configures proxy and TLS settings at the client level. Constructor-level proxy, verify, and cert settings therefore reuse the connection pool; legacy per-call proxies, verify, cert, and stream arguments remain accepted through a scoped compatibility path.

HTTPX 0.28 requires Python 3.8+, which matches this repository's current Python 3.8-3.14 CI matrix.

Verification

  • 119 passed, 42 deselected in the deterministic offline suite
  • 34 passed for the transport/client suite on Python 3.8.20 with HTTPX 0.28.1
  • Ruff format and lint checks pass with a Python 3.8 target
  • Sphinx HTML build passes with warnings treated as errors
  • wheel and sdist build successfully and pass twine check
  • clean Python 3.8 wheel install exposes Client and AsyncClient and depends on HTTPX, not Requests

Live API tests were not run locally to avoid consuming account searches; the existing CI workflow can exercise those with the repository secret.

Controlled benchmark

20 requests against a local threaded endpoint with a 50 ms response delay, using the median of three runs and excluding client setup/shutdown:

Client Median Observed range
requests.Session sequential 1.295 s 1.091-1.338 s
serpapi.Client sequential HTTPX 1.105 s 1.076-1.121 s
serpapi.AsyncClient concurrent 0.122 s 0.121-0.160 s

That is a 9.05x throughput improvement for this concurrent I/O workload. It does not imply that a single search becomes faster.

Review context

@adarshdigievo @jvmvik - I followed the separate AsyncClient direction discussed in #27. The two decisions I would especially appreciate confirmation on are raising the declared minimum to Python 3.8 and retaining legacy per-call proxy/TLS options via scoped clients.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Proposal] Support Python asyncio by migrating to httpx from requests

1 participant