Slow indexing, elasticsearch/django/haystack

100%

Slow indexing, elasticsearch/django/haystack

✓ Published0🌍 Public

Last edited May 10, 2016

Created on May 10, 2016

This example demonstrates how to build a search index for contract records using Elasticsearch, Django, and Haystack. It shows the configuration of a Haystack connection to an AWS Elasticsearch instance with request signing via `requests_aws4auth.AWS4Auth`, and defines a `ContractIndex` class that indexes fields like fiscal year, dollars obligated, and transaction status. The code uses Haystack’s `indexes.CharField` and `indexes.DecimalField` to map model attributes, with a template-based `text` field for full-text search. It also includes Django REST Framework and `drf-haystack` for API integration, though the visualization behavior itself is not depicted.

AI-generated description

Similar vizzes

Loading thumbnail…

Gist 671a9b9586430094aa6d

A web development contract between a cartographer and a turf products company is visualized as a structured, legal timeline. The example uses a custom JavaScript visualization built with D3.js, laying out contract clauses, payment schedules, and milestones as interactive nodes and links. It renders the contract’s text into a diagrammatic flow, with d3.forceSimulation positioning clauses and deadlines, while hover states reveal the original document’s wording. The data source is the raw Markdown contract file, parsed to extract sections, parties, and dates.

SStephen

48% match

Loading thumbnail…

Gist 6198f227114a9aeabdbc

This example demonstrates a Selenium-based automated test of the Django admin interface for the Hireable HQ application, specifically exercising the booking, stock request, and consignment workflows. The test uses the Selenium Python API to launch a Chrome browser, log in with credentials, and navigate through administrative pages by clicking links and buttons. It interacts with form elements using `sel.type` and `sel.select`, and verifies page loads via `sel.wait_for_page_to_load`. The rendering approach is plain HTML pages accessed through the Selenium WebDriver, with no visualization libraries involved.

Aandybak

46% match

Loading thumbnail…

Gist 9ea11dbcc52b4bb1c048

This example shows performance analysis for a data engineering capstone project, comparing a legacy Python log parser against a proposed Scala implementation on Spark and Mesos. The visualization tracks transfer metrics from ASCP log data, specifically examining throughput and packet rates. Using d3.js for charting, it evaluates how efficiently the new streaming solution handles regex parsing of unstructured logs. The project also demonstrates integration with Redis for persistent storage and FaspStream for live data ingestion, with the D3 visualizations presenting benchmark results from test log datasets.

BBBischof

45% match

Loading thumbnail…

Gist 7e0def20499d0f9fa328

This example visualizes 15 years of U.S. federal contract data for a specified company, scraped from USASpending.gov. The code uses bash scripting with `curl` to download XML files by fiscal year, then converts them to JSON using `xml2json`, and cleans the output with `sed` to remove namespace prefixes. The resulting JSON files are stored per year, ready for further analysis or visualization.

Aamccartney

45% match

Loading thumbnail…

Gist dca53ad15bb437468b52eb951a09f356

This visualization shows the data pipeline for managing Fantasy Premier League player data, fetching gameweek deadlines and team rosters from the official FPL API. It retrieves data using Python's `requests` library, converts JSON responses into DataFrames with `pandas`, and then persists the processed data to Amazon S3 through the `boto3` client. The code also reads configuration parameters from AWS SSM and sends notifications via SNS topics, demonstrating an automated workflow for updating and storing fantasy football statistics.

CConorAspell

45% match

Loading thumbnail…

Gist d57b4594607d7185a88e

This example demonstrates parent-child relationships in Elasticsearch by indexing stores, departments, and products as nested document types. It shows how to query hierarchical data using the `has_child` filter to find stores containing departments whose products match a specific trademark. The code uses Elasticsearch’s REST API with `curl` commands to define mappings, index sample data, and execute filtered queries. It illustrates the `_parent` field for linking document types and `_routing` to enforce shard placement, while the final search combines term queries with nested `has_child` filters to retrieve relevant parent documents.

CCBasis

45% match

Loading thumbnail…

Gist fd2387f597641c4a52a7

This visualization displays the movement of consignments over time, plotting each item’s journey from start to end location based on timestamps. The code queries logistics data with Django ORM, filtering out consignments linked to bookings with a specific status, then renders the results as a temporal flow. It uses D3.js’s `d3.forceSimulation` to position nodes representing locations and consignments, while `transition.attrTween` animates the paths between them. The rendering approach relies on SVG elements, with the data sourced from a PostgreSQL database via the raw SQL query shown.

Aandybak

45% match

Loading thumbnail…

Deploying a static site with s3

This example walks through the process of building a static website with Jekyll and deploying it to Amazon Web Services’ S3 hosting. It shows a step-by-step tutorial covering environment setup, theme installation, local testing, and customization. The visualization illustrates the workflow with a series of screenshots and code snippets, making the deployment process clear and accessible.

Aamccartney

45% match