Headless Browser Selenium (HTMLUnitDriver)

A headless browser is a web browser without a graphical user interface. It can run in the background without visual distractions. It offers an efficient and effective way to test web applications while saving time and resources. In this tutorial, we will dive into what a headless browser is, when to use headless browser testing, and how to perform it using Selenium.

What is Headless Browser?

A headless browser is a web-browser without a graphical user interface. This program will behave just like a browser but will not show any GUI.

Some of the examples of Headless Drivers include

• HtmlUnit
• Ghost
• PhantomJS
• ZombieJS
• Watir-webdriver

When to Use Headless Browser Testing?

In today’s digital age, web applications are developed to be compatible with a wide range of devices and platforms. This often presents a challenge for website developers who need to ensure their applications work seamlessly across these platforms. Headless browser testing is an ideal solution for this problem as it allows developers to test their web applications without the need for a graphic user interface. By using headless browser testing, developers can easily test complex web applications with multiple components and dependencies, paving the way for faster development, bug-free code, and happy users.

Headless Browser Testing with Selenium

Selenium is a powerful tool for headless browser testing, allowing developers to run automated tests without the need for a visible user interface. By running tests in the background, Selenium can save time and resources while also helping identify issues that may not be apparent in a traditional UI-based testing environment. This includes performance-related issues and layout problems that may only become evident in a headless setting. However, it’s important to keep in mind the limitations of headless testing and balance it with traditional UI-based methods to ensure comprehensive test coverage.

Popular Examples of Headless Browsers

There are many headless browsers available each with its own unique features and benefits, making them suitable for different use cases. We discuss them below:-

PhantomJS

PhantomJS is a headless browser that utilizes WebKit as its rendering engine and supports various web standards such as HTML5, CSS3, and JavaScript. It can be used for screen capture and page automation tasks. It is open-source and compatible with multiple operating systems.

Example of Selenium with Headless PhantomJS in Python

from selenium import webdriver
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
# Set up PhantomJS options
phantomjs_options = webdriver.DesiredCapabilities.PHANTOMJS.copy()
phantomjs_options['phantomjs.page.settings.userAgent'] = 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.3'
# Set up the PhantomJS driver
driver = webdriver.PhantomJS('/path/to/phantomjs', desired_capabilities=phantomjs_options)
# Perform actions using the driver
driver.get('https://www.example.com')
print(driver.title)
# Close the driver
driver.quit()

Chrome

Chrome is the most popular browser on the planet and offers a headless feature as well. It can be used across multiple platforms and supports programming languages and frameworks. Its built-in debugging tools and extensive documentation make it easy to use and troubleshoot any issues that may arise during testing.

Example of Headless Chrome with Selenium in Python

from selenium import webdriver
from selenium.webdriver.chrome.options import Options
# Set up Chrome options
chrome_options = Options()
chrome_options.add_argument('--headless')  # Run Chrome in headless mode
chrome_options.add_argument('--no-sandbox')
chrome_options.add_argument('--disable-dev-shm-usage')
# Set up the Chrome driver
driver = webdriver.Chrome('/path/to/chromedriver', options=chrome_options)
# Perform actions using the driver
driver.get('https://www.example.com')
print(driver.title)
# Close the driver
driver.quit()

Firefox

Firefox is a popular web browser that can also be used as a headless browser for testing purposes. One of the benefits of using Firefox as a headless browser is its lightweight nature, making it a versatile option for testing on various operating systems. Additionally, with its extensive documentation and community support, Firefox is an excellent choice for those looking to experiment with headless browser technology.

Example of Headless Firefox with Selenium in Python

from selenium import webdriver
from selenium.webdriver.firefox.options import Options
# Set up Firefox options
firefox_options = Options()
firefox_options.add_argument('--headless')  # Run Firefox in headless mode
# Set up the Firefox driver
driver = webdriver.Firefox(options=firefox_options)
# Perform actions using the driver
driver.get('https://www.example.com')
print(driver.title)
# Close the driver
driver.quit()

Advantages of Headless Browser Testing

• Faster test execution
• Cost-effective testing
• Better testing coverage
• Flexibility in running tests
• Integration with CI/CD pipelines

Disadvantages of Headless Browser Testing

• Lack of GUI
• Difficulties in debugging
• Limited browser support

HTMLUnitDriver

HTML UnitDriver is the most light weight and fastest implementation headless browser for of WebDriver. It is based on HtmlUnit. It is known as Headless Browser Driver. It is same as Chrome, IE, or FireFox driver, but it does not have GUI so one cannot see the test execution on screen.

Features of HTML unit driver

• Support for the HTTPS and HTTP protocols
• Support for HTML responses ( clicking links, submitting forms, walking the DOM model of the HTML document etc.)
• Support for cookies
• Proxy server support
• Support for basic and NTLM authentication
• Excellent JavaScript support
• Support for submit methods GET and POST
• Ability to customize the request headers being sent to the server
• Ability to determine whether failing responses from the server should throw exceptions or should be returned as pages of the appropriate type

Steps to Use HTMLUnit Driver with Selenium

Step 1) In Eclipse, copy the following code. Add the standard selenium library files to the project. No additional jar files are required.

package  htmldriver;
import org.openqa.selenium.By;		
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;	
import org.openqa.selenium.htmlunit.HtmlUnitDriver;		
public class htmlUnitYest {				
		public static void main(String[] args) {
                     // Creating a new instance of the HTML unit driver
                      
                     WebDriver driver = new HtmlUnitDriver();
                      
           		     // Navigate to Google		
                     driver.get("http://www.google.com");					
          
					 // Locate the searchbox using its name		
                     WebElement element = driver.findElement(By.name("q"));	
                     
                    // Enter a search query		
                    element.sendKeys("Guru99");	
                   
		            // Submit the query. Webdriver searches for the form using the text input element automatically		
                    // No need to locate/find the submit button		
                    element.submit();			
                    
            		// This code will print the page title		
                    System.out.println("Page title is: " + driver.getTitle());		
                    
                    driver.quit();			
         }		
}

Step 2) Run the code. You will observer no browser is launched and results are shown in console.

Benefits of Html Unit Driver:

• Since it is not using any GUI to test, your tests will run in background without any visual interruption
• Compared to all other instances execution is faster
• To run your tests through HtmlUnit driver you can also select other browser versions
• It is platform independent and easier to run several tests concurrently. Ideal for Load Testing.

Limitations:

• It cannot emulate other browsers JavaScript behavior

PhantomJS

PhantomJS is a headless browser with JavaScript API. It is an optimal solution for Headless Website Testing, access and manipulate webpages & comes with the standard DOM API.

In order to use PhantomJS with Seleniun, one has to use GhostDriver. GhostDriver is a implementation of Webdriver Wire protocol in simple JS for PhantomJS.

The latest release of PhatomJS has integrated GhostDriver and there is no need to separately install it.

Here is how the system works-

Steps to run Selenium with PhatomJS

Step 1) You need Eclipse with Selenium installed

Step 2) Download PhantomJS here

Step 3) Extract the downloaded folder to Program Files

Step 4) Download the PhantomJS Driver from here. Add the jar to your project

Step 5) Paste the following code in eclipse

package htmldriver;	
import java.io.File;
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.phantomjs.PhantomJSDriver;	

public class phantom {				
		public static void main(String[] args) {
        	        File file = new File("C:/Program Files/phantomjs-2.0.0-windows/bin/phantomjs.exe");				
                    System.setProperty("phantomjs.binary.path", file.getAbsolutePath());		
                    WebDriver driver = new PhantomJSDriver();	
                    driver.get("http://www.google.com");         
                    WebElement element = driver.findElement(By.name("q"));	
                    element.sendKeys("Guru99");					
                    element.submit();         			
                    System.out.println("Page title is: " + driver.getTitle());		
                    driver.quit();			
           }		
}

Step 6) Run the code. You will observe the output is shown in console and no browser is launched.

NOTE: At first run, based on your settings, you may get security warning from Windows to allow to run PhantomJS. Click on Allow Access.

Many organization uses Phantom.JS for various purpose, for example,

• Headless Testing
• Screen Capture
• Page Automation
• Network Monitoring
• To render dashboard screenshots for their users
• To run Unit tests on command line
• To generate employee handbooks from HTML to PDF
• Combined with QUnit for the test suite

Summary

To test application rapidly in various browsers and without any visual interruption, headless browser Testing is used. Due to its speed, accuracy and easy to access features, HTML unit driver and PhantomJS are gaining popularity for headless browser automation testing. By following some simple steps you get to know how easily these tools can be integrated with other tools and can execute the test code.