Check links website
You can use the requests library in Python to check if a link is valid, and you can use the BeautifulSoup library to parse HTML and extract links from a webpage. Here's a simple example using these libraries to check for broken links on a website:
import requests
from bs4 import BeautifulSoup
from urllib.parse import urljoin
def get_all_links(url):
response = requests.get(url)
soup = BeautifulSoup(response.text, 'html.parser')
links = [a.get('href') for a in soup.find_all('a', href=True)]
return links
def check_links(links):
for link in links:
full_url = urljoin(base_url, link)
try:
response = requests.head(full_url)
response.raise_for_status()
print(f"Link {full_url} is valid.")
except requests.exceptions.HTTPError as errh:
print(f"HTTP Error: {errh} for {full_url}")
except requests.exceptions.ConnectionError as errc:
print(f"Error Connecting: {errc} for {full_url}")
except requests.exceptions.Timeout as errt:
print(f"Timeout Error: {errt} for {full_url}")
except requests.exceptions.RequestException as err:
print(f"An error occurred: {err} for {full_url}")
# Replace 'your_website_url' with the actual URL of your website
base_url = 'https://your_website_url'
all_links = get_all_links(base_url)
check_links(all_links)
In this example, the get_all_links function fetches all the links from a given webpage. The check_links function then iterates over these links, sends a HEAD request to each link to check if it's valid, and prints the results. You should replace 'your_website_url' with the actual URL of your website.