Gist 7e0def20499d0f9fa328
✓ Published0🌍 Public
Aamccartney
Last edited Nov 10, 2015
Created on Nov 9, 2015
This example visualizes 15 years of U.S. federal contract data for a specified company, scraped from USASpending.gov. The code uses bash scripting with `curl` to download XML files by fiscal year, then converts them to JSON using `xml2json`, and cleans the output with `sed` to remove namespace prefixes. The resulting JSON files are stored per year, ready for further analysis or visualization.
AI-generated descriptionNew and easier scraper!
Ok, all those functionalities are now rolled into one script! Get 15 years of contract data by running this on the command line. Returns a separate file for each year.
Takes the name of a company as an argument. If you want Hewlett-Packard data, for instance, run
$ bash scraper.sh Hewlett-Packard