Few days ago NewsFromNepal was asking me for some help on some URL validation code. The task was to verify a list of URLs and print the title of the page if it's valid/working. Here's a simple Python script to do that. The URLs are stored per line in url.txt
#! /usr/bin/python
# URL Validity
import urllib2
import re
urls = open('urls.txt', 'r').readlines()
badurls = []
print "Working URLs:",
for url in urls:
try:
handle = urllib2.urlopen(url)
html = handle.read()
match = re.search("<title>(.*)</title>",html)
print "\n", url.strip(), " -> ",
if match:
print match.groups()[0],
except:
badurls.append(url)
if badurls:
print "\nBad URLs:"
for bad in badurls:
print bad.strip()