Blocking Unnecessary Resources in Web Scraping

Tense Turtle answered on November 1, 2024 Popularity 6/10 Helpfulness 1/10

answer Blocking Unnecessary Resources in Web Scraping

related Automated Backup Script

Blocking Unnecessary Resources in Web Scraping

Comment

{
2 
3    // Launching a new browser instance
4    const browser = await chromium.launch({
5        headless: true
6    });
7 
8    // Opening a new page
9    const page = await browser.newPage();
10 
11    // Block unnecessary resources (images, stylesheets, fonts)
12    await page.route("**/*", route => {
13 
14        const url = route.request().url();
15 
16        if (url.endsWith(".png") || url.endsWith(".jpg") || url.endsWith(".jpeg") || url.endsWith(".gif") ||
17 
18        	url.endsWith(".css") || url.endsWith(".woff") || url.endsWith(".woff2") || url.endsWith(".ttf")) {
19 
20        	route.abort();
21 
22        } else {
23        	route.continue();
24        }
25 
26    });
27 
28    // Navigate to the target URL
29    await page.goto("https://browserless.io/");
30    console.log("Page loaded");
31 
32    // Perform any scraping tasks
33    const data = await page.evaluate(() => {
34 
35        return document.title;
36 
37    });
38 
39    console.log(`Page title: ${data}`);
40 
41    // Closing the page
42    await page.close();
43 
44    console.log("Page closed");
45 
46    // Closing the browser instance
47    await browser.close();
48 
49    console.log("Browser closed");
50 
51})();
COPY

Popularity 6/10 Helpfulness 1/10 Language whatever

Source: www.browserless.io

Tags: blocking resources web-scraping whatever

Link to this answer
Share Copy Link

Contributed on Nov 01 2024

Tense Turtle

0 Answers Avg Quality 2/10

Closely Related Answers

Automated Backup Script

Comment

#!/bin/bash
# Backup a directory and store it in a backup folder with a timestamp
SOURCE="/path/to/important/data"
DEST="/path/to/backup/location"
TIMESTAMP=$(date +"%Y%m%d%H%M%S")
tar -czvf $DEST/backup_$TIMESTAMP.tar.gz $SOURCE
echo "Backup completed: $DEST/backup_$TIMESTAMP.tar.gz"

Popularity 3/10 Helpfulness 1/10 Language shell

Source: blog.devops.dev

Tags: shell script backup

Link to this answer
Share Copy Link

Contributed on Nov 01 2024

barisx

0 Answers Avg Quality 2/10

Blocking Unnecessary Resources in Web Scraping

Contents

More Related Answers

Blocking Unnecessary Resources in Web Scraping

Closely Related Answers

Automated Backup Script

Grepper

Documentation

Social

Legal

Contact

Oops, You will need to install Grepper and log-in to perform this action.