Requests¶

Overview¶

https://stackoverflow.com/questions/2018026/what-are-the-differences-between-the-urllib-urllib2-urllib3-and-requests-modul

In the Python 2 standard library there were two HTTP libraries that existed side-by-side. Despite the similar name, they were unrelated: they had a different design and a different implementation.

urllib was the original Python HTTP client, added to the standard library in Python 1.2. Earlier documentation for urllib can be found in Python 1.4.

urllib2 was a more capable HTTP client, added in Python 1.6, intended as a replacement for urllib:

urllib2 - new and improved but incompatible version of urllib (still experimental).

Earlier documentation for urllib2 can be found in Python 2.1.

The Python 3 standard library has a new urllib which is a merged/refactored/rewritten version of the older modules.

urllib3 is a third-party package (i.e., not in CPython's standard library). Despite the name, it is unrelated to the standard library packages, and there is no intention to include it in the standard library in the future.

Finally, requests internally uses urllib3, but it aims for an easier-to-use API.

I think all answers are pretty good. But fewer details about urllib3.urllib3 is a very powerful HTTP client for python. For installing both of the following commands will work,

urllib3 using pip,

pip install urllib3 or you can get the latest code from Github and install them using,

$ git clone git://github.com/urllib3/urllib3.git $ cd urllib3 $ python setup.py install Then you are ready to go,

Just import urllib3 using,

import urllib3 In here, Instead of creating a connection directly, You’ll need a PoolManager instance to make requests. This handles connection pooling and thread-safety for you. There is also a ProxyManager object for routing requests through an HTTP/HTTPS proxy Here you can refer to the documentation. example usage :

>>> from urllib3 import PoolManager
>>> manager = PoolManager(10)
>>> r = manager.request('GET', 'http://google.com/')
>>> r.headers['server']
'gws'
>>> r = manager.request('GET', 'http://yahoo.com/')
>>> r.headers['server']
'YTS/1.20.0'
>>> r = manager.request('POST', 'http://google.com/mail')
>>> r = manager.request('HEAD', 'http://google.com/calendar')
>>> len(manager.pools)
2
>>> conn = manager.connection_from_host('google.com')
>>> conn.num_requests

As mentioned in urrlib3 documentations,urllib3 brings many critical features that are missing from the Python standard libraries.

Thread safety. Connection pooling. Client-side SSL/TLS verification. File uploads with multipart encoding. Helpers for retrying requests and dealing with HTTP redirects. Support for gzip and deflate encoding. Proxy support for HTTP and SOCKS. 100% test coverage. Follow the user guide for more details.

Response content (The HTTPResponse object provides status, data, and header attributes) Using io Wrappers with Response content Creating a query parameter Advanced usage of urllib3 requests requests uses urllib3 under the hood and make it even simpler to make requests and retrieve data. For one thing, keep-alive is 100% automatic, compared to urllib3 where it's not. It also has event hooks which call a callback function when an event is triggered, like receiving a response In requests, each request type has its own function. So instead of creating a connection or a pool, you directly GET a URL.

For install requests using pip just run

pip install requests

or you can just install from source code,

$ git clone git://github.com/psf/requests.git $ cd requests $ python setup.py install Then, import requests

Here you can refer the official documentation, For some advanced usage like session object, SSL verification, and Event Hooks please refer to this url.

https://docs.python.org/3/library/urllib.request.html#module-urllib.request

Reference¶

https://stackoverflow.com/questions/2018026/what-are-the-differences-between-the-urllib-urllib2-urllib3-and-requests-modul