05. pip 101¶

Overview¶

pip is the **P**ackage **I**nstaller for **P**ython. It's the tool recommended by PyPA for managing dependencies for application sections. Starting from Python 3.4, pip is the preferred installer program and is included by default with the Python binary installers.

Documentation: https://pip.pypa.io/en/latest/getting-started/

Usage¶

pip has total of 14 commands.

Command List¶

PIP search package¶

For the internal design for pip

https://pip.pypa.io/en/latest/development/architecture/package-finding/

Collect together the various network and file system locations containing project package files. These locations are derived, for example, from pip’s --index-url (with default https://pypi.org/simple/ ) setting and any configured --extra-index-url locations. Each of the project page URL’s is an HTML page of anchor links, as defined in PEP 503, the “Simple Repository API.”

For each project page URL, fetch the HTML and parse out the anchor links, creating a Link object from each one. The LinkCollector class is responsible for both the previous step and fetching the HTML over the network.

Determine which of the links are minimally relevant, using the LinkEvaluator class. Create an InstallationCandidate object (aka candidate for install) for each of these relevant links.

Further filter the collection of InstallationCandidate objects (using the CandidateEvaluator class) to a collection of “applicable” candidates.

If there are applicable candidates, choose the best candidate by sorting them (again using the CandidateEvaluator class).

Cache strategy¶

Target:

The reason why we need cached:

• Using for dev-mode to reduced built time

• Using for slim CICD for internal testing purpose

• Some when deduplication on prod when not changing

Strategy:

• [1] Follow the Git SHA

• [2] Follow Git Branch

Implement:

ref: https://pip.pypa.io/en/stable/topics/caching/

Recommendation¶

Using with Hash

Reference¶

https://docs.python.org/3/installing/index.html

PIP - Wheel https://realpython.com/python-wheels/

Ref: https://pip.pypa.io/en/stable/topics/secure-installs/

Appendix¶

# Build
wheel>=0.34.1,<=0.41.2
hatch==1.7.0

# Global
typing-extensions==4.6.0

# Datetime
python-dateutil>=2.8.2,<2.9.0
pytz==2023.3
tzdata>=2023.3
ciso8601==2.3.0

# [API Framework]
fastapi==0.98.0,<0.100.0

# [Server Framework]
starlette>=0.27.0,<0.28.0

# [Deployment Server]
uvicorn==0.23.1

Alternative for pip

Process

When install seaborn[stats]==0.13

Traceback (most recent call last): File "C:\Python3\lib\site-packages\pip_vendor\urllib3\response.py", line 438, in _error_catcher yield File "C:\Python3\lib\site-packages\pip_vendor\urllib3\response.py", line 561, in read data = self._fp_read(amt) if not fp_closed else b"" File "C:\Python3\lib\site-packages\pip_vendor\urllib3\response.py", line 527, in _fp_read return self._fp.read(amt) if amt is not None else self._fp.read() File "C:\Python3\lib\site-packages\pip_vendor\cachecontrol\filewrapper.py", line 90, in read data = self.__fp.read(amt) File "C:\Python3\lib\http\client.py", line 459, in read n = self.readinto(b) File "C:\Python3\lib\http\client.py", line 503, in readinto n = self.fp.readinto(b) File "C:\Python3\lib\socket.py", line 704, in readinto return self._sock.recv_into(b) File "C:\Python3\lib\ssl.py", line 1241, in recv_into return self.read(nbytes, buffer) File "C:\Python3\lib\ssl.py", line 1099, in read return self._sslobj.read(len, buffer) socket.timeout: The read operation timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "C:\Python3\lib\site-packages\pip_internal\cli\base*command.py", line 180, in exc_logging_wrapper status = run_func(*args) File "C:\Python3\lib\site-packages\pip_internal\cli\req_command.py", line 248, in wrapper return func(self, options, args) File "C:\Python3\lib\site-packages\pip_internal\commands\install.py", line 377, in run requirement_set = resolver.resolve( File "C:\Python3\lib\site-packages\pip_internal\resolution\resolvelib\resolver.py", line 161, in resolve self.factory.preparer.prepare_linked_requirements_more(reqs) File "C:\Python3\lib\site-packages\pip_internal\operations\prepare.py", line 565, in prepare_linked_requirements_more self._complete_partial_requirements( File "C:\Python3\lib\site-packages\pip_internal\operations\prepare.py", line 479, in _complete_partial_requirements for link, (filepath, ) in batch_download: File "C:\Python3\lib\site-packages\pip_internal\network\download.py", line 183, in **call* for chunk in chunks: File "C:\Python3\lib\site-packages\pip_internal\cli\progress_bars.py", line 53, in _rich_progress_bar for chunk in iterable: File "C:\Python3\lib\site-packages\pip_internal\network\utils.py", line 63, in response_chunks for chunk in response.raw.stream( File "C:\Python3\lib\site-packages\pip_vendor\urllib3\response.py", line 622, in stream data = self.read(amt=amt, decode_content=decode_content) File "C:\Python3\lib\site-packages\pip_vendor\urllib3\response.py", line 587, in read raise IncompleteRead(self._fp_bytes_read, self.length_remaining) File "C:\Python3\lib\contextlib.py", line 135, in exit self.gen.throw(type, value, traceback) File "C:\Python3\lib\site-packages\pip_vendor\urllib3\response.py", line 443, in _error_catcher raise ReadTimeoutError(self._pool, None, "Read timed out.") pip._vendor.urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='files.pythonhosted.org', port=443): Read timed out.

Solved python -m pip install seaborn[stats]==0.13 --default-timeout=100

https://stackoverflow.com/questions/43298872/how-to-solve-readtimeouterror-httpsconnectionpoolhost-pypi-python-org-port

PIP - Wheel https://realpython.com/python-wheels/

https://learnpython.com/blog/python-requirements-file/

https://snyk.io/advisor/python/check-requirements-txt

Using pip compile: https://rednafi.com/python/dependency_management_redux/