Skip to main content
SimilarExport ZIP
100%

pywb proxy mods

✓ Published0🌍 Public
NN0taN3rd
Last edited Apr 22, 2017
Created on Apr 22, 2017

This visualization shows the internal logic of the pywb web archiving proxy, displaying how HTML, CSS, and JavaScript content is rewritten during replay. The animation illustrates the code paths for rewriting HTML tags and attributes via the `HTMLRewriterMixin`, and content sanitization through `RewriteContent` using regex patterns for meta refresh and charset detection. The wombat.js script implements client-side URL rewriting with a JavaScript state machine that handles absolute, relative, and proxy-mode URL transformations. The gist demonstrates the interaction between these Python and JavaScript modules in the pywb rewriting pipeline, emphasizing the regex-based parsing and URL manipulation strategies.

AI-generated description

Similar vizzes

Loading thumbnail…

Gist 2d272134b57b02eb8cdd0c5904249591

This example tests the archival fidelity of a web page by loading a series of images and scripts that mimic dynamic behavior, including DOM manipulation, AJAX requests, and delayed updates. It uses JavaScript’s `XMLHttpRequest` to fetch remote images and replaces them via base64 data URLs, while `setTimeout` and scroll events trigger asynchronous content changes. The page is rendered within an iframe by the pywb web archiving tool, with a banner and rewrite layer applied. The visualization demonstrates how archived sites handle live interactions and resource loading over time.

NN0taN3rd
68% match
Loading thumbnail…

View CNN.com by iframe

This example demonstrates a historical snapshot of CNN.com’s homepage replayed inside a web iframe, using the pywb web-archiving framework. The visualization shows how the original page’s JavaScript-driven content, including lazy-loaded zones and carousels, is reconstructed from archived assets. The code references CNN’s custom inline scripts—such as `CNN.Zones`, `CNN.contentModel`, and `pageReveal`—alongside the iframe’s `seamless` attribute. The pywb replay uses `wb.js`, `wb_frame.js`, and `default_banner.js` to embed the captured page, with the source being a captured timestamp from the archive.

NN0taN3rd
63% match
Loading thumbnail…

PYTHONPATH=<path to pywb dir>pywb/ python -v -u pywb/apps/cli.py --live &> out.txt

This example visualizes Python module import traces generated by running `pywb` with Python’s verbose flag, capturing output to a text file. It shows the dependency chain of `gevent` and related libraries as they are loaded, highlighting both source and extension modules. The visualization uses a network graph layout likely built with `d3.forceSimulation` to connect modules to their import origins. Each node represents a module or code object, with edges indicating dependency relationships, sourced from the parsed `out.txt` log lines.

NN0taN3rd
60% match
Loading thumbnail…

https://web.archive.org/web/20170209205035/http://www.soufeel.com/

This visualization tracks the client-side rewriting of a single webpage archived in the Wayback Machine, logging every URL modification made by JavaScript functions. It tallies 357 total rewrites and 13 HTTP 404 errors, breaking down calls by API (such as `Element.setAttribute`, `Node.appendChild`, `Element.innerHTML`, and `document.write`) and by element and attribute type, with counts for each combination. Raw server logs and rewrite logs are appended, showing specific requests and error responses. The data is rendered as plain text tables and log excerpts, providing a quantitative audit of archive-injected behavior.

NN0taN3rd
59% match
Loading thumbnail…

CS 891 Web Archiving Seminar

This visualization demonstrates the process of web archiving by showing the WARC file records produced when using `wget` to archive two websites: `http://example.com` and the React Router documentation. The code uses `wget` with extensive arguments including `--mirror`, `--warc-cdx`, and `--page-requisites` to capture complete page snapshots. The example displays the resulting WARC file structure, containing request, response, metadata, and resource records, along with a shell script that orchestrates the archiving workflow and manages the Wayback Machine collections.

NN0taN3rd
56% match
Loading thumbnail…

Internet Archive Incorrectly Wrapping Valid HTML

This example demonstrates how the Internet Archive’s Wayback Machine inserts its banner and analytics scripts into an archived page, wrapping a Polymer web component’s `<dom-module>` and its script in unexpected ways. The archived HTML shows the original valid structure disrupted by added script tags and a `textarea` element containing code, breaking the component’s expected parsing. It uses Polymer’s element registration and observers, plus XMLHttpRequest for fetching data, highlighting the mismatch between archived and original HTML.

NN0taN3rd
55% match
Loading thumbnail…

Mat Kelly, PhD Student

This visualization examines the structure and contents of two WARC files from different crawls of Mat Kelly's academic homepage. It animates a series of frames that map the sequence of HTTP requests and responses, using a force-directed layout via `d3.forceSimulation` to position nodes by their relationships. The rendering combines SVG, Canvas, and WebGL to display the animation, with timing controlled by `transition.attrTween`. The data is parsed directly from the provided `heritrix.warc` and `wget.warc` files, visualizing crawl behavior and server responses like redirects and robots.txt blocks.

NN0taN3rd
54% match
Loading thumbnail…

{p.title}

This animated visualization illustrates the structure and interactive features of a FastHTML-based blogging application, rendering the flow from notebook files to styled HTML pages. The code demonstrates how Jupyter notebooks are parsed and converted into markdown with embedded HTML, using `mistletoe` for markdown processing and `fastcore.nbio` for file handling. The application manages subscriber data through a SQLite database via `fastlite`, while the user interface is built with `fasthtml.common` and `monsterui.all` components. The animation highlights the templating system that transforms content into a complete blog layout, including navigation, article rendering, and social media links.

深深度碎片
53% match