An Approximate Distribution wrapper class for TDunnings java TDigest library
✓ Published0🌍 Public
RRobColeman
Last edited Jan 29, 2016
Created on Jan 29, 2016
This example demonstrates how to estimate and visualize approximate probability distributions from large datasets using a wrapper class around TDunning's Java TDigest library. The code shows both local and Spark-based aggregation paths, where data is converted into TreeDigest instances and merged via reduction operations. It generates CDF and PDF sequences by sampling support points between observed minimum and maximum values, then plots these distributions for comparison. The example also illustrates Kryo serialization configuration for Spark clusters, using 100,000 samples drawn from a normal distribution to validate the approach.
AI-generated description