Beautiful Soup — a library for extracting data from HTML and XML files, ideal for web scraping.
🔹 Installation
pip install beautifulsoup4
🔹 Import
from bs4 import BeautifulSoup
import requests
🔹 Basic Parsing
html_doc = "<html><body><p class='text'>Hello, world!</p></body></html>"
soup = BeautifulSoup(html_doc, 'html.parser') # or 'lxml', 'html5lib'
print(soup.p.text) # Hello, world!
🔹 Element Search
# First found element
first_p = soup.find('p')
# Search by class or attribute
text_elem = soup.find('p', class_='text')
text_elem = soup.find('p', {'class': 'text'})
# All elements
all_p = soup.find_all('p')
all_text_class = soup.find_all(class_='text')
🔹 Working with Attributes and Text
a_tag = soup.find('a')
print(a_tag['href']) # value of the href attribute
print(a_tag.get_text()) # text inside the tag
print(a_tag.text) # alternative🔹 Navigating the Tree
# Moving to parent, children, siblings
parent = soup.p.parent
children = soup.ul.children
next_sibling = soup.p.next_sibling
# Finding the previous/next element
prev_elem = soup.find_previous('p')
next_elem = soup.find_next('div')
🔹 Parsing a Real Page
response = requests.get('https://example.com')
soup = BeautifulSoup(response.text, 'html. parser')
title = soup.title.text
links = [a['href'] for a in soup.find_all('a', href=True)]🔹 CSS Selectors
# More powerful and concise search
items = soup.select('div.content > p.text')
first_item = soup.select_one('a.button')
💡 Where it's useful:
🟢 Web scraping and data collection
🟢 Processing HTML/XML reports
🟢 Automating data extraction from websites
🟢 Preparing data for analysis and machine learning
👩💻 @CodeProgrammer