Yet another Python package for scraping FanFiction.Net…

pip install ffscraper

Authors:Alexander L. Hayes
Documentation:Module Index
Search:Search Page
Bugtracker:GitHub Issues

License Travis CI continuous integration build status Code coverage status Documentation status

FanFiction.Net was established in 1998 and is among the world’s largest collection of user-submitted fanfiction (works of fanfiction authored by fans of existing stories; such as movies, books, or TV shows). Recently the large amount of easily-available user content has drawn interest in analyzing the content and creative differences between original works and their fanfiction counterparts [1], and [2] created an anonymized dataset of the metadata.

This project is twofold: creating open-source systems for scraping content, and using that content to build open-source systems which can be used by the FanFiction.Net community.

Installation and Usage

Interact with the scraper from the commandline:

$ pip install ffscraper
$ python -m ffscraper --help
$ python -m ffscraper -s 123

Or import the Python package and start building your own systems:

import ffscraper as ffs

sids = ['123', '124', '125']

for id in sids:
    story = ffs.fanfic.story.scraper(id)
[1]Milli, Smitha and David Bamman, “Beyond Canonical Texts: A Computational Analysis of Fanfiction.” Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing.
[2]Yin, K., Aragon, C., Evans, S. and Katie Davis. “Where No One Has Gone Before: A Meta-Dataset of the World’s Largest Fanfiction Repository.” Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. ACM, 2017.