Navigate to a Web Page

The first step in the web scraping procedure is navigating to the web page we plan to scrape. When we do this manually, what we do is:

1. Launch the browser.

2. Open a new browser page.

3. Navigate to the web page.

When we use Puppeteer, we must follow the same steps. Let’s see how we can instruct Puppeteer to perform the above tasks:

Launch the web browser

We can open the browser using the launch function in Puppeteer, as shown in the code snippet below. With default configurations, this will launch a Chromium browser bundled with Puppeteer.

We can use the returned reference variable from the launch function to interact with it further.

We can use the default configuration to do our scraping work most of the time. However, if we want to customize the browser, we can customize it by providing some configurations. Here are ...