how to scrap data in python

Akhlaq Altaf answered on March 10, 2022 Popularity 9/10 Helpfulness 2/10

answer how to scrap data in python

related python scraping

related Python Script to Scrape Data From Website

how to scrap data in python

Comment

-1

import requests

from bs4 import BeautifulSoup
import pandas as pd
import time

# URL of the website to scrape
url = "https://www.imdb.com/chart/top"

# Send an HTTP GET request to the website
response = requests.get(url)

# Parse the HTML code using BeautifulSoup
soup = BeautifulSoup(response.content, 'html.parser')

# Extract the relevant information from the HTML code
movies = []
for row in soup.select('tbody.lister-list tr'):
title = row.find('td', class_='titleColumn').find('a').get_text()
year = row.find('td', class_='titleColumn').find('span', class_='secondaryInfo').get_text()[1:-1]
rating = row.find('td', class_='ratingColumn imdbRating').find('strong').get_text()
movies.append([title, year, rating])

# Store the information in a pandas dataframe
df = pd.DataFrame(movies, columns=['Title', 'Year', 'Rating'])

# Add a delay between requests to avoid overwhelming the website with requests
time.sleep(1)

Popularity 9/10 Helpfulness 2/10 Language python

Source: nanonets.com

Tags: python

Link to this answer
Share Copy Link

Contributed on Mar 14 2024

Akhlaq Altaf

0 Answers Avg Quality 2/10

Closely Related Answers

python scraping

Comment

#Scrapes Python's URL, version number and logo from its Wikipedia page:

# $ pip3 install requests beautifulsoup4
import requests, bs4, os, sys

URL = 'https://en.wikipedia.org/wiki/Python_(programming_language)'
try:
    html       = requests.get(URL).text
    document   = bs4.BeautifulSoup(html, 'html.parser')
    table      = document.find('table', class_='infobox vevent')
    python_url = table.find('th', text='Website').next_sibling.a['href']
    version    = table.find('th', text='Stable release').next_sibling.strings.__next__()
    logo_url   = table.find('img')['src']
    logo       = requests.get(f'https:{logo_url}').content
    filename   = os.path.basename(logo_url)
    with open(filename, 'wb') as file:
        file.write(logo)
    print(f'{python_url}, {version}, file://{os.path.abspath(filename)}')
except requests.exceptions.ConnectionError:
    print("You've got problems with connection.", file=sys.stderr)

Popularity 10/10 Helpfulness 4/10 Language python

Source: gto76.github.io

Tags: beautifulsoup be

Link to this answer
Share Copy Link

Contributed on Mar 10 2022

Michael

0 Answers Avg Quality 2/10

Python Script to Scrape Data From Website

Comment

import os
import requests
from bs4 import BeautifulSoup
url = "https://www.google.com/"
reponse = requests.get(url)
if reponse.ok:
 soup = BeautifulSoup(reponse.text, "lxml")
 title = str(soup.find("title"))
 title = title.replace("<title>", "")
 title = title.replace("</title>", "")
 print("The title is : " + str(title))
os.system("pause")

Popularity 10/10 Helpfulness 3/10 Language python

Source: medium.com

Tags: python py

Link to this answer
Share Copy Link

Contributed on Jun 16 2022

Harendra

0 Answers Avg Quality 2/10

how to scrap data in python

Contents

More Related Answers

how to scrap data in python

Closely Related Answers

python scraping

Python Script to Scrape Data From Website

Grepper

Documentation

Social

Legal

Contact

Oops, You will need to install Grepper and log-in to perform this action.