Introduction - If you have any usage issues, please Google them yourself
As an excellent guide for using Python to crawl network data, it explains how to crawl data from static pages and how to use caching to manage server loads. In addition, the book also introduces how to use AJAX URL and Firebug extensions to crawl data, as well as more truths about crawling techniques, such as using browsers to render, managing cookie, and submitting forms to extract data from complex sites protected by a validation code. This book uses Scrapy to create a high-level web crawler and crawls some real Web sites.