Frontera 0.3.0 documentation

This documentation contains everything you need to know about Frontera.

First steps

Frontera at a glance
Understand what Frontera is and how it can help you.
Installation Guide
Get Frontera installed on your computer.

Basic concepts

What is a Frontera?
Learn what Frontera is and how to use it.
Architecture overview
See how Frontera works and its different components.
Frontier objects
Understand the classes used to represent links and pages.
Frontera API
Learn how to use the frontier.
See how to configure Frontera.

Extending Frontera

Filter or alter information for links and pages.
Define your own crawling logic.

Built-in services and tools

Using the Frontier with Scrapy
Learn how to use Frontera with Scrapy.
Using the Frontier with Requests
Learn how to use Frontera with Requests.
Graph Manager
Define fake crawlings for websites to test your frontier.
Testing a Frontier
Test your frontier in an easy way.
Recording a Scrapy crawl
Create Scrapy crawl recordings and reproduce them later.
Scrapy Seed Loaders
Scrapy middlewares for seed loading

All the rest

Some example projects and scripts using Frontera.
Best practices
The best practices of Frontera usage.
How to run and write Frontera tests.
Release Notes
See what has changed in recent Frontera versions.