Skip to main content

PyPI

This page guides you through the process of setting up the PyPI source connector.

Setup guide

Get package name from PyPI

This is the name given in pip install package_name box. For example, airbyte-cdk is the package name for airbyte-cdk.

Optianlly, provide a version name. If not provided, the release stream, containing data for particular version, cannot be used. The project stream is as same as release stream but contains data for all versions.

Supported streams and sync modes

Performance considerations

Due to the heavy caching and CDN use, there is currently no rate limiting of PyPI APIs at the edge.

In addition, PyPI reserves the right to temporarily or permanently prohibit a consumer based on irresponsible activity.

Try not to make a lot of requests (thousands) in a short amount of time (minutes). Generally PyPI can handle it, but it’s preferred to make requests in serial over a longer amount of time if possible.

Changelog

Expand to review
VersionDatePull RequestSubject
0.2.12024-08-1644196Bump source-declarative-manifest version
0.2.02024-08-1444082Refactor connector to manifest-only format
0.1.152024-08-1043643Update dependencies
0.1.142024-08-0343137Update dependencies
0.1.132024-07-2742790Update dependencies
0.1.122024-07-2042210Update dependencies
0.1.112024-07-1341844Update dependencies
0.1.102024-07-1041464Update dependencies
0.1.92024-07-0941078Update dependencies
0.1.82024-07-0640842Update dependencies
0.1.72024-06-2540459Update dependencies
0.1.62024-06-2239952Update dependencies
0.1.52024-06-0639152[autopull] Upgrade base image to v1.2.2
0.1.42024-05-2838702Make connector compatible with the builder
0.1.32024-04-1937237Upgrade to CDK 0.80.0 and manage dependencies with Poetry.
0.1.22024-04-1537237Base image migration: remove Dockerfile and use the python-connector-base image
0.1.12024-04-1237237schema descriptions
0.1.02022-10-2918632Initial Release