Gist 6b851197754b1a378f905e75b82697b0

100%

Gist 6b851197754b1a378f905e75b82697b0

✓ Published0🌍 Public

Last edited Jan 19, 2017

Created on Nov 8, 2016

This example demonstrates how to engineer and transform date, time, categorical, and numerical features for predictive modeling. The R script defines date-based variables such as day of week, week of year, and quarter, alongside time bins for 24-hour periods, and incorporates open calendar data like national holidays and seasons. It applies categorical encodings via hashing with FeatureHashing's `hashed.model.matrix` using a 14-bit hash, one-hot, target, orthogonal polynomial, and deviation methods. Numerical transformations include Box-Cox and binning, while general manipulations support column-wise or combinatorial diff, sum, prod, and custom formula operations across data frames.

AI-generated description

Similar vizzes

Loading thumbnail…

Gist 1dcc8a9bd27317b89fdde2c22c049100

This example presents a Scala-based feature engineering pipeline for machine learning models, specifically for predicting eCPM (effective cost per mille) in advertising. The code defines a system of feature computers that transform prediction requests into sparse feature vectors using Apache Spark's MLlib library (`SparkVectors.sparse`) and the `MurmurHash3` hashing function. It shows how categorical features like app publisher and model identifiers are hashed into indexed blocks within a numeric vector, with each block's position and size tracked through `FeatureComputerParams`. The implementation uses pattern matching in `FeatureComputerRouter` to select appropriate computations and aggregates values into a map that can be converted to Spark's sparse vector format for model training.

RRobColeman

60% match

Loading thumbnail…

Common feature exploration functions for Data Science Competitions in R

This R script provides a collection of feature-exploration functions for data science competitions, demonstrating common preprocessing tasks. It shows how to remove duplicate and highly correlated columns, one-hot encode categorical variables, and apply deviation encoding for high-cardinality factors. The code uses the `caret` package's `findCorrelation` for correlation-based feature removal, `model.matrix` for dummy encoding, and `doMC` for parallel correlation computation via `bigcorPar`. It also leverages `dplyr` for grouped summarization in deviation encoding, with the `ff` package for large correlation matrices.

Aaakansh9

57% match

Loading thumbnail…

Gist 066cc1deef3173351f9d6b358b93d78b

This visualization generates synthetic timestamp data from year strings using a SQL function that randomly selects a point between January 1 and December 31 of the given year. The example demonstrates how to create randomized temporal values for testing or simulation purposes, with the function `random_time_from_year_string` accepting a year as text and returning a timestamp. The code leverages PostgreSQL's SQL language features, including type casting and arithmetic on timestamp intervals, while using the `random()` function to produce the uniform distribution across the year. The rendering approach is not specified, as the example focuses solely on the data generation logic within the SQL query.

Aandrewbt

57% match

Loading thumbnail…

Gist 7236992

This example demonstrates a Ruby helper method that converts month-level time intervals into an array of associated week intervals. The code shows how the `get_weeks` method takes a hash containing a year, month, and an array of week strings, then returns an array where each week maps back to the original month metadata. It uses RSpec’s `context`, `let`, and `it` blocks to test the expected output, verifying that four weeks in October 2013 correctly associate with the same month interval. The visualization relies on this data structure to support time-series rendering.

Aalanmclean

54% match

Loading thumbnail…

Gist 47795c5a23bd689563b991acd76f1d86

This example demonstrates a data-cleaning and recoding workflow for survey data, transforming raw categorical variables into labeled, analysis-ready formats. The code uses `dplyr`'s `left_join()` within a custom `recode.f()` function to merge recoded value labels from external CSV files stored in a `dat/yg_recoded/` directory. Variables such as political identification, education, income, religion, and region are sequentially processed and renamed, with a final `select()` step to drop the original region column. The output is a cleaned dataset ready for visualization, though no plotting library is featured.

MMarie-Ella

54% match

Loading thumbnail…

Gist ff1355ec904d9b079db5

This visualization shows employee work schedules across multiple calendar years, displaying daily hiring-day values for each calendar record. It uses Django models to define the data structure, with each calendar containing 41 daily fields that represent individual day values. The code implements a force-directed layout using d3.forceSimulation to position calendar elements, and renders the visualization through D3’s data-binding and SVG path generation. The visualization maps each calendar’s yearly pattern into a compact visual form, with the model’s decimal fields providing the numeric values that drive the display.

Aandybak

53% match

Loading thumbnail…

Function for assigning a date to a bi-annual period: Useful when needing to group dates into a period for time-series visualisation

This visualization example demonstrates a utility function for converting individual dates into bi-annual period groupings, a preprocessing step for time-series analysis. The function uses `lubridate`'s `dmy`, `year`, and `floor_date` functions to compare each input date against two fixed cutpoints (June 30th and December 31st), then returns the first day of the month for the appropriate six-month period. Dates falling after the second cutpoint are assigned to the following year’s first period, ensuring all data maps to a two-category annual cycle. The code explicitly handles year boundaries through conditional `ifelse` logic.

JJustinCally

53% match

Loading thumbnail…

Gist 7e406d1d7597759ba28a1b687103ea3a

This example retrieves a seven-day weather forecast for San Diego from the OpenWeatherMap One Call API and converts it into a pandas DataFrame structured for Tableau analytics. The code transforms raw JSON fields into normalized columns, including temperature conversions from Kelvin to Celsius and Fahrenheit, humidity as a decimal, and wind direction mapped to sixteen-point wind rose labels. It also generates icon URLs for weather conditions and converts Unix timestamps for sunrise, sunset, and moon phases into UTC datetime strings. The final output is a dictionary of lists ready for Tableau ingestion.

Aantichaos

53% match