Skip to main content
100%

Script to download all fox news press releases. Prints verbose output to console and file.

✓ Published0🌍 Public
Aabehmiel
Last edited Aug 9, 2017
Created on Aug 8, 2017

This example scrapes Fox News press releases from the press archive for years 2011–2017, organized by month, and saves each release as a separate text file. It uses BeautifulSoup and the `requests` library to parse HTML pages, extract article text from elements with class `hentry`, and prints verbose download progress and content to the console while writing files to a `press-releases/` directory. The script iterates through URL patterns for each year and month, handling missing pages silently.

AI-generated description

Similar vizzes

Loading thumbnail…

Script to download all the CBS press releases. Prints verbose output to console and to file.

A script downloads every CBS Corporation press release from 2009 through 2017, saving each article’s text as a separate file in a `press-releases/` directory. It uses BeautifulSoup to parse HTML from both year-index pages and individual article pages, extracting the main content from the `entry-content` class. The code iterates through paginated year listings, printing each URL and the downloaded text to the console, and writes the text to numbered files. Requests handles the HTTP fetching, while `urljoin` and glob organize the file paths.

Aabehmiel
78% match
Loading thumbnail…

Press release scrape for pressreleasepoint.com - verbose output to console and saves to text file

This example scrapes press releases from pressreleasepoint.com, saving each article’s text to individual files while printing both the download URL and content to the console. It demonstrates web scraping with BeautifulSoup and requests, iterating through 2,497 paginated index pages and extracting text from the `content-press_release` class. The script handles missing links with a broad exception pass, outputs verbose progress logs, and organizes saved files by page and index.

Aabehmiel
76% match
Loading thumbnail…

A simple news crawler.

This example demonstrates a news crawler that fetches headlines from four Malaysian RSS feeds, extracts the full article text, and stores each story as a JSON file organized by publication date. The crawler uses `feedparser` to parse RSS feeds, `requests` to download article pages, and the `readability` library to isolate main content from HTML. `BeautifulSoup` cleans the extracted text, while `dateutil.parser` converts publication timestamps into structured day, month, and year fields. Each JSON file, saved in a local `collection` folder, contains the article’s cleaned content, original link, and parsed date components.

Aamirothman
56% match
Loading thumbnail…

A vanilla py-script to download all Pocket.com data via API.

This example demonstrates how to download a complete archive of all Pocket.com saved articles using the Pocket API. It shows the OAuth authentication flow, which opens a browser for user approval, then retrieves every article in batches of ten, displaying each article’s title and URL as it processes them. The script uses Python’s `requests` library to make HTTP POST calls to the Pocket API endpoints, along with the `json` module to parse responses and `subprocess` to trigger browser opening. It saves the full article list as a JSON file.

BBastiTee
53% match
Loading thumbnail…

Gist d003ece0b6ca171be8e095e0bc90af7e

This notebook example demonstrates how to programmatically scrape, parse, and structure a corpus of New York Times articles into a pandas DataFrame. The code loads a pre-saved pickle file of 50 article links, iterates through each URL using the `newspaper` library's `Article` class, and extracts the article’s text, title, publication date, keywords, and summary. It then stores the results in a list of dictionaries and converts that list into a structured DataFrame. The script includes error handling for unparseable articles, inserting `NaN` values, and prints the total execution time, which took roughly 38.5 seconds for this run. The displayed table shows the resulting columns: date, keywords, link, summary, text, and title.

GGeorgeMcIntire
51% match
Loading thumbnail…

Buzzfeed article scraper for NLP

This example scrapes a Buzzfeed article and converts its text into clean, tokenized sentences for natural language processing. It shows the pipeline from raw HTML to normalized, sentence-segmented data. The scraper uses BeautifulSoup and requests to extract paragraph elements with the class `subbuzz-text`. Text cleaning relies on the `unicodedata` module and regex via `re`, while sentence tokenization uses NLTK’s Punkt tokenizer. The final output is a single string of joined sentences, ready for further NLP analysis.

Aabehmiel
51% match
Loading thumbnail…

BLS Unemployment Scraper

This example fetches monthly unemployment data from the U.S. Bureau of Labor Statistics public API for two series IDs and writes formatted text tables to separate files. It uses `node-fetch` to POST the series IDs and date range, `prettytable` to build a table with columns for series ID, year, period, value, and footnotes, and `fs` to save each table as a `.txt` file. The script filters out non-monthly periods, concatenates footnote texts, and logs completion after generating the files.

CCurran Kelleher
51% match
Loading thumbnail…

Gist 7e0def20499d0f9fa328

This example visualizes 15 years of U.S. federal contract data for a specified company, scraped from USASpending.gov. The code uses bash scripting with `curl` to download XML files by fiscal year, then converts them to JSON using `xml2json`, and cleans the output with `sed` to remove namespace prefixes. The resulting JSON files are stored per year, ready for further analysis or visualization.

Aamccartney
50% match