Tools for ecosystem monitoring work
Whenever possible, we publish our collection and analysis code to allow others to reproduce our findings and hold our methods to account.
All repositories · 6
pytok collects TikTok videos, comments and account metadata for research corpora: user timelines, hashtag feeds, keyword search and sounds, with helpers that flatten results into video, comment and user tables. Our most-used tool, tested in CI and citable through a Zenodo DOI. Install it from this repository; the name on PyPI belongs to unrelated software. Released under the PolyForm Noncommercial licence: source-available, for non-commercial use.
fbscrape collects public Facebook data — page and group timelines, keyword search, comments, post details and page-transparency records — and returns it in Facebook’s own schema rather than parsed HTML. Built for long collections that can resume where they stopped, with output to CSV, JSONL or Parquet and a guide for every endpoint. Released under the PolyForm Noncommercial licence: source-available, for non-commercial use.
igscrape collects Instagram user timelines, profiles, individual posts and keyword-search results, from Python or the command line, as the JSON records the app itself receives. Results stream to JSONL and flatten to CSV or Parquet, and images and videos can be downloaded from a saved result file. Released under the PolyForm Noncommercial licence: source-available, for non-commercial use.
aiscrape audits what AI products answer and what they cite: it sends a prompt to Google’s AI Mode and to the ChatGPT, Claude, Gemini and Meta AI web interfaces, and returns the answer together with the web sources behind it. It is the collection layer behind our audits of AI answer engines. Released under the PolyForm Noncommercial licence: source-available, for non-commercial use.
The meo command line is the client for MEO’s digital trace database: search, count, aggregate and bulk-export posts across Twitter/X, Facebook, Instagram, YouTube, TikTok, Telegram and Bluesky, with results as JSONL, JSON, CSV or a table. It requires an approved MEO account.
A topic-modelling pipeline that pulls posts from the MEO API for a query, a date range and a set of platforms, and clusters them with BERTopic or Toponymy using multilingual embeddings. It is a research pipeline rather than a library, and it requires MEO API access. Released under the PolyForm Noncommercial licence: source-available, for non-commercial use.