Web Crawler / Web Bot

Using Python3 and Beautiful Soup, Xidel, or other library or tool, create a program to find all of the external links in a web page and print statistics. (this is the start of a web crawler or web bot.)

Are there different Python modules you could use?

Project #1

Create a program to find all of the external links in a web page and print statistics. See the instructor for the complete requirements. (this is the start of a web crawler or web bot)

Project #2

Go down some of those links and do it all over again. Limit the depth you go to. (This is just a programming exercise.)

Links

Beautiful Soup Tutorial - How to Parse Web Data With Python

Top 6 Alternatives to Xidel for Web Scraping and Data Extraction