Introduction
Ever been in a situation where you needed to get down to the nitty-gritty of network activity on your web application? Well, you're in the right place. Today, I’m going to walk you through how to capture network logs in the HAR (HTTP Archive) format using Selenium and the Chrome DevTools Protocol (CDP) in Java. This will be a game-changer for debugging and performance analysis.
Prerequisites
Before we dive in, make sure you’ve got these essentials:
- Java Development Kit (JDK) - the latest version is preferred.
- Selenium library (selenium-java).
- ChromeDriver - ensure it matches your version of Chrome.
- A build tool like Maven or Gradle to manage dependencies.
Step 1: Add Selenium Dependency
If you’re using Maven, add this to your pom.xml:
<dependency>
<groupId>org.seleniumhq.selenium</groupId>
<artifactId>selenium-java</artifactId>
<version>4.1.0</version>
</dependency>
For Gradle users, add this to your build.gradle:
dependencies {
testImplementation 'org.seleniumhq.selenium:selenium-java:4.1.0'
}Step 2: Set Up ChromeDriver
Time to set up ChromeDriver and make it work with CDP. Here’s a simple code snippet to get you started:
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.chrome.ChromeOptions;
import org.openqa.selenium.devtools.DevTools;
import org.openqa.selenium.devtools.v85.network.Network;
import org.openqa.selenium.devtools.v85.network.model.RequestWillBeSent;
import org.openqa.selenium.devtools.v85.network.model.ResponseReceived;
import org.openqa.selenium.devtools.v85.network.model.Response;
public class CaptureHAR {
public static void main(String[] args) {
// Set the path to the ChromeDriver executable
System.setProperty("webdriver.chrome.driver", "/path/to/chromedriver");
// Set Chrome options
ChromeOptions options = new ChromeOptions();
// Initialize ChromeDriver
ChromeDriver driver = new ChromeDriver(options);
// Create DevTools session
DevTools devTools = driver.getDevTools();
devTools.createSession();
// Enable Network
devTools.send(Network.enable(Optional.empty(), Optional.empty(), Optional.empty()));
// Add event listener to capture network requests
devTools.addListener(Network.requestWillBeSent(), request -> {
System.out.println("Request: " + request.getRequest().getUrl());
});
// Add event listener to capture network responses
devTools.addListener(Network.responseReceived(), response -> {
Response res = response.getResponse();
System.out.println("Response: " + res.getUrl() + " - " + res.getStatus());
});
// Navigate to the website
driver.get("https://example.com");
// Perform any actions on the website here
// Close the browser
driver.quit();
}
}
Step 3: Capture and Save HAR Logs
import com.google.gson.Gson;
import com.google.gson.GsonBuilder;
import org.openqa.selenium.devtools.v85.network.model.ResponseReceived;
import java.io.FileWriter;
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
import java.util.Optional;
public class CaptureHAR {
public static void main(String[] args) {
System.setProperty("webdriver.chrome.driver", "/path/to/chromedriver");
ChromeOptions options = new ChromeOptions();
ChromeDriver driver = new ChromeDriver(options);
DevTools devTools = driver.getDevTools();
devTools.createSession();
devTools.send(Network.enable(Optional.empty(), Optional.empty(), Optional.empty()));
Gson gson = new GsonBuilder().setPrettyPrinting().create();
List<ResponseReceived> responses = new ArrayList<>();
devTools.addListener(Network.responseReceived(), responses::add);
driver.get("https://example.com");
// Wait for some time to capture network activity
try {
Thread.sleep(10000);
} catch (InterruptedException e) {
e.printStackTrace();
}
try (FileWriter writer = new FileWriter("network_logs.har")) {
gson.toJson(responses, writer);
} catch (IOException e) {
e.printStackTrace();
}
driver.quit();
}
}
Wrapping Up
And there you have it! Capturing network logs using Selenium and Chrome DevTools Protocol in Java is a powerful tool for web developers. Whether you’re debugging an issue or optimizing performance, these HAR files will provide invaluable insights into your web application's network activity.
