计算机科学导论

Build a Web Crawler

The program is just for communication with computer.

The language of program is to avoid and more effective.

Buid a web crawler needs three steps. There are finding data, building an index and ranking pages.

 

Unit 1

Getting started--Extracting the first link on a web page.

A Web crawler finds web pages for our search engine by starting from a "seed" page and following links on that page to find other pages.

The program gives us a way to tell the computer what steps to take. In this lecture, python is the language to used. And it is a nice high-level language.