---
description: In this tutorial, learn how to access Internet data in Python. Learn how to get HTML Data from URL using Urllib.Request and urlopen() examples.
title: Python Internet Access using Urllib.Request and urlopen()
image: https://www.guru99.com/images/python-internet-access-urllib-urlopen.png
---

 

[Skip to content](#main) 

**⚡ Smart Summary**

urllib is the standard Python module for accessing internet data, letting programs open URLs and retrieve HTML, JSON, or binary content. The urllib.request.urlopen() function connects to a web address and reads the server response directly into Python.

* 🔘 **Standard library:** urllib ships with Python and bundles the request, error, parse, and robotparser modules for URL tasks.
* ☑️ **Open a URL:** The urllib.request.urlopen() function opens a connection, and getcode() returns the HTTP status such as 200.
* ✅ **Read the response:** The read() method returns raw bytes, which decode(“utf-8”) converts into readable text.
* 🧪 **Handle errors:** Catch HTTPError and URLError from urllib.error so failed requests never crash the program.
* 🛠️ **Download files:** The urlretrieve() function saves any URL straight to disk, ideal for images, PDFs, and datasets.
* 🤖 **AI ready:** urllib fetches datasets and API data that feed machine learning models and AI pipelines.

[ Read More ](javascript:void%280%29;) 

![Python Internet Access using Urllib](https://www.guru99.com/images/python-internet-access-urllib-urlopen.png)

The sections below explain what urllib is, how to open a URL and read its HTML, and how to handle errors, download files, and choose between urllib and the requests library.

## What is urllib?

urllib is a Python module that can be used for opening URLs. It defines functions and classes to help in URL actions.

With Python, you can also access and retrieve data from the internet, such as XML, HTML, and JSON, and then work with this data directly. In the sections below, we will see how to retrieve data from the web. For example, here we use a Guru99 video URL, access it using Python, and print the HTML file of this URL.

The urllib package groups several modules: urllib.request for opening URLs, urllib.error for the exceptions that requests can raise, urllib.parse for building and parsing URLs, and urllib.robotparser for reading robots.txt files.

## How to Open URL using Urllib

Before we run the code to connect to internet data, we need to import the URL library module, or “urllib”.

[](https://www.guru99.com/images/Pythonnew/python19%5F1.png)

* Import urllib
* Define your main function
* Declare the variable webUrl
* Then call the urlopen function on the urllib library
* The URL we are opening is the Guru99 tutorial on YouTube
* Next, we print the result code
* The result code is retrieved by calling the getcode function on the webUrl variable we have created
* We convert that to a string, so that it can be concatenated with our string “result code”
* This will be a regular HTTP code “200”, indicating the HTTP request is processed successfully

## How to Get an HTML File from a URL in Python

You can also read the HTML file by using the read() function in Python. When you run the code, the HTML file will appear in the console.

[](https://www.guru99.com/images/Pythonnew/python19%5F2.png)

* Call the read() function on the webUrl variable
* The read() method allows you to read the contents of data files
* Read the entire content of the URL into a variable called data
* Run the code and it will print the data in HTML format

Here is the complete code:

### RELATED ARTICLES

* [Python DateTime, TimeDelta, Strftime(Format) with Examples ](https://www.guru99.com/date-time-and-datetime-classes-in-python.html "Python DateTime, TimeDelta, Strftime(Format) with Examples")
* [Python abs() Function: Absolute Value Examples ](https://www.guru99.com/abs-in-python.html "Python abs() Function: Absolute Value Examples")
* [type() and isinstance() in Python with Examples ](https://www.guru99.com/type-isinstance-python.html "type() and isinstance() in Python with Examples")
* [Python Program to Find the Factorial of a Number ](https://www.guru99.com/python-factorial-example.html "Python Program to Find the Factorial of a Number")

### Python 2 Example

#  
# read the data from the URL and print it
#
import urllib2

def main():
# open a connection to a URL using urllib2
   webUrl = urllib2.urlopen("https://www.youtube.com/user/guru99com")
  
#get the result code and print it
   print "result code: " + str(webUrl.getcode()) 
  
# read the data from the URL and print it
   data = webUrl.read()
   print data
 
if __name__ == "__main__":
  main()

### Python 3 Example

#
# read the data from the URL and print it
#
import urllib.request
# open a connection to a URL using urllib
webUrl  = urllib.request.urlopen('https://www.youtube.com/user/guru99com')

#get the result code and print it
print ("result code: " + str(webUrl.getcode()))

# read the data from the URL and print it
data = webUrl.read()
print (data)

## How to Handle urllib Errors: URLError and HTTPError

A request does not always succeed. When something goes wrong, urllib raises an exception instead of returning data, so it is important to handle these errors. The urllib.error module defines two main exception classes.

* **HTTPError** is raised when the server returns an error status code, such as 404 Not Found or 500 Internal Server Error. It carries the status code and the response body.
* **URLError** is raised when the server cannot be reached at all, for example because of a wrong domain name or no internet connection. It carries a reason attribute that explains the failure.

Because HTTPError is a subclass of URLError, always catch HTTPError first so that each error type is handled separately:

from urllib.request import urlopen
from urllib.error import HTTPError, URLError

try:
    response = urlopen("https://www.guru99.com/no-such-page.html")
    data = response.read()
except HTTPError as error:
    print("HTTP error:", error.code)
except URLError as error:
    print("URL error:", error.reason)

Handling these exceptions keeps your program from crashing and lets you log the problem, retry the request, or fall back to cached data.

## How to Download a File from a URL Using urllib

Reading a URL into memory works for small pages, but for images, PDF documents, or datasets it is easier to save the response straight to disk. The urllib.request module offers two simple ways to do this.

The urlretrieve() function downloads a URL directly to a local file in a single line:

from urllib.request import urlretrieve

urlretrieve("https://www.python.org/static/img/python-logo.png", "logo.png")

For more control, open the URL and write the bytes yourself. Open the local file in binary mode because urlopen() returns bytes, not text:

from urllib.request import urlopen

with urlopen("https://example.com/data.zip") as response:
    with open("data.zip", "wb") as file:
        file.write(response.read())

For very large files, read and write in a loop using response.read(8192) so the whole file is never held in memory at once.

## urllib vs. the requests Library in Python

urllib is part of the Python standard library, so it works everywhere without installation. The third-party requests library is not built in, but many developers prefer it for everyday HTTP work because its syntax is shorter and more readable.

The main differences are:

* **Installation:** urllib ships with Python, while requests must be installed with pip.
* **Decoding:** urllib returns raw bytes that you decode yourself, while requests decodes text and JSON automatically.
* **Convenience:** requests handles sessions, cookies, and custom headers with less code.
* **Dependencies:** urllib adds none, while requests relies on urllib3 underneath.

Choose urllib when you want zero dependencies or only need a quick, one-off request. Choose requests for larger projects that involve authentication, sessions, or frequent API calls. Both send the same underlying HTTP requests, so the decision comes down to convenience versus a smaller dependency list.

## FAQs

⚡ What does urllib.request.urlopen() return?

urlopen() returns an HTTP response object, usually an http.client.HTTPResponse. You call read() to get the raw bytes, getcode() to read the status code, and headers or info() to inspect metadata such as content type and length.

🔄 What is the difference between urllib2 and urllib.request?

urllib2 belonged to Python 2\. In Python 3 it was reorganized: its features were split into urllib.request for opening URLs and urllib.error for exceptions. Code written for urllib2 must be updated to import urllib.request instead.

📥 Why must you call .decode() on the response data?

read() returns bytes, not a string, because urlopen() does not know the encoding in advance. Calling decode(“utf-8”) converts those bytes into readable text so you can print, search, or parse the HTML or JSON content.

🧾 How do you read JSON data from a URL with urllib?

Open the URL with urlopen(), read the bytes, decode them, then parse with the json module: json.loads(response.read().decode(“utf-8”)). This returns a Python dictionary or list you can use directly, which is common when calling web APIs.

🕵️ How do you set a custom User-Agent header in urllib?

Wrap the URL in a urllib.request.Request object and add the header before opening it: request.add\_header(“User-Agent”, “Mozilla/5.0”), then pass the request to urlopen(). Custom headers help when a server blocks the default Python user agent.

🤖 Can urllib collect data for machine learning projects?

Yes. urllib can fetch datasets, CSV files, and JSON from web APIs, which you then decode and load into pandas or NumPy. It is a lightweight, dependency-free way to pull training data into a machine learning pipeline.

🧠 Can GitHub Copilot help write urllib code?

Yes. GitHub Copilot autocompletes common urllib patterns such as urlopen() calls, Request objects, and try/except error handling. It speeds up boilerplate, but you should still verify the URLs, headers, and exception handling the AI assistant suggests.

📤 How do you send a POST request with urllib?

Encode your parameters with urllib.parse.urlencode(), convert them to bytes with encode(), then pass them as the data argument to urlopen() or a Request object. Supplying data automatically makes urllib send a POST instead of a GET request.

#### Summarize this post with:

ChatGPT Perplexity Grok Google AI 

**Stay Updated on AI** **Get Weekly AI Skills, Trends, Actionable Advice.** 

##### Sign up for the newsletter

Subscribe for Free 

You have successfully subscribed.  
Please check your inbox. 

![AI-Newsletter]() Chosen by over **350,000+** professionals 

[Scroll to top ](#wrapper)Scroll to top 

× 

Toggle Menu Close 

Search for: 

Search

```json
{"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://www.guru99.com/#organization","name":"Guru99","sameAs":["https://www.facebook.com/Guru99Official","https://twitter.com/guru99com"],"logo":{"@type":"ImageObject","@id":"https://www.guru99.com/#logo","url":"https://www.guru99.com/images/guru99-logo-v1-150x59.png","contentUrl":"https://www.guru99.com/images/guru99-logo-v1-150x59.png","caption":"Guru99","inLanguage":"en-US"}},{"@type":"WebSite","@id":"https://www.guru99.com/#website","url":"https://www.guru99.com","name":"Guru99","publisher":{"@id":"https://www.guru99.com/#organization"},"inLanguage":"en-US"},{"@type":"ImageObject","@id":"https://www.guru99.com/images/python-internet-access-urllib-urlopen.png","url":"https://www.guru99.com/images/python-internet-access-urllib-urlopen.png","width":"700","height":"250","caption":"Python Internet Access Using Urllib.Request and Urlopen()","inLanguage":"en-US"},{"@type":"BreadcrumbList","@id":"https://www.guru99.com/accessing-internet-data-with-python.html#breadcrumb","itemListElement":[{"@type":"ListItem","position":"1","item":{"@id":"https://www.guru99.com","name":"Home"}},{"@type":"ListItem","position":"2","item":{"@id":"https://www.guru99.com/python","name":"Python"}},{"@type":"ListItem","position":"3","item":{"@id":"https://www.guru99.com/accessing-internet-data-with-python.html","name":"Python Internet Access using Urllib.Request and urlopen()"}}]},{"@type":"WebPage","@id":"https://www.guru99.com/accessing-internet-data-with-python.html#webpage","url":"https://www.guru99.com/accessing-internet-data-with-python.html","name":"Python Internet Access using Urllib.Request and urlopen()","dateModified":"2026-07-11T11:29:53+05:30","isPartOf":{"@id":"https://www.guru99.com/#website"},"primaryImageOfPage":{"@id":"https://www.guru99.com/images/python-internet-access-urllib-urlopen.png"},"inLanguage":"en-US","breadcrumb":{"@id":"https://www.guru99.com/accessing-internet-data-with-python.html#breadcrumb"}},{"@type":"Person","@id":"https://www.guru99.com/author/anna","name":"Anna Blake","description":"I'm Anna Blake, specializing in Python tutorials, offering clear and concise lessons to help you master Python programming efficiently.","url":"https://www.guru99.com/author/anna","image":{"@type":"ImageObject","@id":"https://www.guru99.com/images/anna-blake-author.png","url":"https://www.guru99.com/images/anna-blake-author.png","caption":"Anna Blake","inLanguage":"en-US"},"worksFor":{"@id":"https://www.guru99.com/#organization"}},{"articleSection":"Python","headline":"Python Internet Access using Urllib.Request and urlopen()","description":"In this tutorial, learn how to access Internet data in Python. Learn how to get HTML Data from URL using Urllib.Request and urlopen() examples.","keywords":"python","speakable":{"@type":"SpeakableSpecification","cssSelector":[".entry-title",".summary"]},"@type":"Article","author":{"@id":"https://www.guru99.com/author/anna","name":"Anna Blake"},"dateModified":"2026-07-11T11:29:53+05:30","image":{"@id":"https://www.guru99.com/images/python-internet-access-urllib-urlopen.png"},"copyrightYear":"2026","name":"Python Internet Access using Urllib.Request and urlopen()","subjectOf":[{"@type":"FAQPage","mainEntity":[{"@type":"Question","name":"What does urllib.request.urlopen() return?","acceptedAnswer":{"@type":"Answer","text":"urlopen() returns an HTTP response object, usually an http.client.HTTPResponse. You call read() to get the raw bytes, getcode() to read the status code, and headers or info() to inspect metadata such as content type and length."}},{"@type":"Question","name":"What is the difference between urllib2 and urllib.request?","acceptedAnswer":{"@type":"Answer","text":"urllib2 belonged to Python 2. In Python 3 it was reorganized: its features were split into urllib.request for opening URLs and urllib.error for exceptions. Code written for urllib2 must be updated to import urllib.request instead."}},{"@type":"Question","name":"Why must you call .decode() on the response data?","acceptedAnswer":{"@type":"Answer","text":"read() returns bytes, not a string, because urlopen() does not know the encoding in advance. Calling decode(\"utf-8\") converts those bytes into readable text so you can print, search, or parse the HTML or JSON content."}},{"@type":"Question","name":"How do you read JSON data from a URL with urllib?","acceptedAnswer":{"@type":"Answer","text":"Open the URL with urlopen(), read the bytes, decode them, then parse with the json module: json.loads(response.read().decode(\"utf-8\")). This returns a Python dictionary or list you can use directly, which is common when calling web APIs."}},{"@type":"Question","name":"How do you set a custom User-Agent header in urllib?","acceptedAnswer":{"@type":"Answer","text":"Wrap the URL in a urllib.request.Request object and add the header before opening it: request.add_header(\"User-Agent\", \"Mozilla/5.0\"), then pass the request to urlopen(). Custom headers help when a server blocks the default Python user agent."}},{"@type":"Question","name":"Can urllib collect data for machine learning projects?","acceptedAnswer":{"@type":"Answer","text":"Yes. urllib can fetch datasets, CSV files, and JSON from web APIs, which you then decode and load into pandas or NumPy. It is a lightweight, dependency-free way to pull training data into a machine learning pipeline."}},{"@type":"Question","name":"Can GitHub Copilot help write urllib code?","acceptedAnswer":{"@type":"Answer","text":"Yes. GitHub Copilot autocompletes common urllib patterns such as urlopen() calls, Request objects, and try/except error handling. It speeds up boilerplate, but you should still verify the URLs, headers, and exception handling the AI assistant suggests."}},{"@type":"Question","name":"How do you send a POST request with urllib?","acceptedAnswer":{"@type":"Answer","text":"Encode your parameters with urllib.parse.urlencode(), convert them to bytes with encode(), then pass them as the data argument to urlopen() or a Request object. Supplying data automatically makes urllib send a POST instead of a GET request."}}]}],"@id":"https://www.guru99.com/accessing-internet-data-with-python.html#schema-1140766","isPartOf":{"@id":"https://www.guru99.com/accessing-internet-data-with-python.html#webpage"},"publisher":{"@id":"https://www.guru99.com/#organization"},"inLanguage":"en-US","mainEntityOfPage":{"@id":"https://www.guru99.com/accessing-internet-data-with-python.html#webpage"}}]}
```