Skip to main content
100%

Script to download all the CBS press releases. Prints verbose output to console and to file.

✓ Published0🌍 Public
Aabehmiel
Last edited Aug 9, 2017
Created on Aug 8, 2017

A script downloads every CBS Corporation press release from 2009 through 2017, saving each article’s text as a separate file in a `press-releases/` directory. It uses BeautifulSoup to parse HTML from both year-index pages and individual article pages, extracting the main content from the `entry-content` class. The code iterates through paginated year listings, printing each URL and the downloaded text to the console, and writes the text to numbered files. Requests handles the HTTP fetching, while `urljoin` and glob organize the file paths.

AI-generated description

Similar vizzes

Loading thumbnail…

Script to download all fox news press releases. Prints verbose output to console and file.

This example scrapes Fox News press releases from the press archive for years 2011–2017, organized by month, and saves each release as a separate text file. It uses BeautifulSoup and the `requests` library to parse HTML pages, extract article text from elements with class `hentry`, and prints verbose download progress and content to the console while writing files to a `press-releases/` directory. The script iterates through URL patterns for each year and month, handling missing pages silently.

Aabehmiel
78% match
Loading thumbnail…

Press release scrape for pressreleasepoint.com - verbose output to console and saves to text file

This example scrapes press releases from pressreleasepoint.com, saving each article’s text to individual files while printing both the download URL and content to the console. It demonstrates web scraping with BeautifulSoup and requests, iterating through 2,497 paginated index pages and extracting text from the `content-press_release` class. The script handles missing links with a broad exception pass, outputs verbose progress logs, and organizes saved files by page and index.

Aabehmiel
73% match
Loading thumbnail…

A vanilla py-script to download all Pocket.com data via API.

This example demonstrates how to download a complete archive of all Pocket.com saved articles using the Pocket API. It shows the OAuth authentication flow, which opens a browser for user approval, then retrieves every article in batches of ten, displaying each article’s title and URL as it processes them. The script uses Python’s `requests` library to make HTTP POST calls to the Pocket API endpoints, along with the `json` module to parse responses and `subprocess` to trigger browser opening. It saves the full article list as a JSON file.

BBastiTee
55% match
Loading thumbnail…

A simple news crawler.

This example demonstrates a news crawler that fetches headlines from four Malaysian RSS feeds, extracts the full article text, and stores each story as a JSON file organized by publication date. The crawler uses `feedparser` to parse RSS feeds, `requests` to download article pages, and the `readability` library to isolate main content from HTML. `BeautifulSoup` cleans the extracted text, while `dateutil.parser` converts publication timestamps into structured day, month, and year fields. Each JSON file, saved in a local `collection` folder, contains the article’s cleaned content, original link, and parsed date components.

Aamirothman
51% match
Loading thumbnail…

A Bash script to download files of a Gist answered by Gemini model

This example demonstrates a Bash script that downloads all files from a specified GitHub Gist using its unique ID. It first fetches the Gist’s metadata as JSON via the GitHub API with `curl`, then extracts raw file URLs using `jq`’s `.files[].raw_url` filter. The script iterates over each URL, extracts the filename with `basename`, and downloads the file into a chosen directory using a second `curl` call, printing a confirmation for each download. The rendering approach is purely command-line based, relying on the GitHub REST API and standard Unix text-processing tools.

VVioletVivirand
51% match
Loading thumbnail…

download using "Copy All as cURL" from chrome dev tools (https://www.quora.com/How-can-I-download-all-images-from-Google-image-search-engine-at-once)

This example shows a shell-based pipeline that downloads a batch of images from a web page, using Chrome DevTools’ “Copy All as cURL” command to capture network requests. It pipes copied cURL commands through `pbpaste`, then `awk` to strip the trailing character and append a sequential filename (e.g., `>>1.jpg`) before executing each command with `bash`. The rendering relies entirely on standard Unix utilities—`pbpaste`, `awk`, and `bash`—with no JavaScript libraries or visualization frameworks, and the data source is the clipboard contents from the DevTools network tab.

RRasinj
51% match
Loading thumbnail…

Sync NBA team data from stats.nba.com

A shell script named `syncteams.sh` synchronizes NBA team statistics from the stats.nba.com API by iterating over seasons from 1996–97 through 2013–14. For each season, it fetches six JSON files—Base, Advanced, Misc, Four Factors, Scoring, and Opponent—using `curl` with a URL that specifies parameters like `MeasureType`, `PerMode=Totals`, and `rowsPerPage=30`. The script checks for existing files to avoid redundant downloads, re-fetching only the latest season (2013–14) each run. Data is stored in a `data/teams/` directory organized by season, with filenames lowercased from the stat type.

CCaged
51% match
Loading thumbnail…

Gist d003ece0b6ca171be8e095e0bc90af7e

This notebook example demonstrates how to programmatically scrape, parse, and structure a corpus of New York Times articles into a pandas DataFrame. The code loads a pre-saved pickle file of 50 article links, iterates through each URL using the `newspaper` library's `Article` class, and extracts the article’s text, title, publication date, keywords, and summary. It then stores the results in a list of dictionaries and converts that list into a structured DataFrame. The script includes error handling for unparseable articles, inserting `NaN` values, and prints the total execution time, which took roughly 38.5 seconds for this run. The displayed table shows the resulting columns: date, keywords, link, summary, text, and title.

GGeorgeMcIntire
51% match