Writing 10M lines of a query result to a table in Power Query via R

100%

Writing 10M lines of a query result to a table in Power Query via R

✓ Published0🌍 Public

Last edited Mar 9, 2017

Created on Mar 9, 2017

This example demonstrates how to write 10 million rows of a query result to a CSV file from Power Query by invoking an R script. It shows the integration of Power Query’s table generation with R’s file-export capabilities, specifically using the `R.Execute` function to run `write.csv` on a dataset. The Power Query code creates a list of numbers from 1 to 10,000,000 via `{1..10000000}`, converts it to a table with `Table.FromList`, and passes that table as the `dataset` parameter to the R script. The R script then writes the data to `d:\Downloads\out.csv`.

AI-generated description

Similar vizzes

Loading thumbnail…

Using R.Execute() in Power Query to export the contents of a table to CSV

This example demonstrates how to use R.Execute() in Power Query to export a table’s contents to a CSV file. The code generates a seed list of numbers from 1 to 1,000,000, converts it into a table, and then passes that dataset to R’s write.csv() function via the R.Execute() API. The R script writes the data to a specified file path. The visualization highlights the integration between Power Query and R, showing how R’s data-export capabilities can be invoked directly within a query pipeline, without additional manual steps.

HHugoberry

83% match

Loading thumbnail…

Power Query export data to SQL Server

This example demonstrates exporting data from a SQL Server database by running a large SQL script stored in a local file. It shows how Power Query can leverage native database connectivity to execute a bulk insert operation with high parallelism. The code uses `File.Contents` to read the SQL file, then passes its binary content to `Value.NativeQuery`, which targets a SQL Server instance via `Sql.Database`. Key parameters include `EnableBulkInsert` for efficient data loading, `MaxDegreeOfParallelism` to speed up execution, and explicit timeouts for command and connection. The rendered output is the result set from the query, effectively transferring data into Power BI’s data model.

HHugoberry

70% match

Loading thumbnail…

R.Execute example without the use of R.Execute() in Power Query [behind the scenes this gets executed bin\x64\Rscript.exe" --encoding=UTF-8 RScriptWrapper.R"]

This example demonstrates how to execute an R script from within Power Query without explicitly invoking the `R.Execute` function, instead relying on the `AdoDotNet.DataSource` with the `R.Provider` to run a native query. The code passes a string containing an `output` variable that captures the result of `system2("whoami", stdout=TRUE)`, which retrieves the current username. The resulting table is then converted from binary R data using `RData.FromBinary`. This approach showcases the underlying use of `Rscript.exe` via the provider, highlighting an alternative integration path for R workflows in Power Query.

HHugoberry

69% match

Loading thumbnail…

Reading a Zip file in Power Query via R

This example demonstrates how to read a compressed CSV file directly from a local ZIP archive using Power Query’s R integration. The visualization shows the result of an R script that extracts the file `R_Twitter_sample.csv` from `R_Twitter_sample.zip` and loads it into a table. The Power Query `R.Execute` function runs the R code, which relies on R’s base `read.csv` and `unz` functions to handle the archive and file parsing. The output is then returned to Power Query via the `[Name="datacsv"]` reference, making the data available for further transformation or charting. No external visualization library is used; the focus is purely on data ingestion.

HHugoberry

67% match

Loading thumbnail…

Gist faec7eb6c03589ff608bab991699407b

This example demonstrates how to generate and execute a dynamic SQL script in Power Query (M language) to bulk-insert data into a SQL Server table using a table-valued parameter (TVP). It builds a textual TVP declaration and populates it by chunking source rows into sets of 1,000 entries, then constructs a multi-statement script that inserts each chunk and calls a stored procedure. The code relies on M functions like `Table.TransformRows`, `List.Generate`, `Text.Combine`, and `Value.NativeQuery` to programmatically assemble and run the query. The visualization highlights the data flow from a source table to a SQL stored procedure, emphasizing the chunking logic and the use of native query execution.

HHugoberry

66% match

Loading thumbnail…

Power Query embedded in an excel odc connection file

This example shows how a Power Query formula is embedded inside an Excel ODBC connection file, specifically within a Mashup.xml document. It demonstrates the structure of a single query item that generates a simple list of numbers from 1 to 20 using the `let` and `in` syntax. The code, written in Power Query’s M language, defines a source as a list range and then outputs that list. The file includes metadata such as client type (EXCEL), version, and culture settings, and the query is stored as a CDATA block within the XML, allowing for direct execution by Excel’s data mashup engine.

HHugoberry

65% match

Loading thumbnail…

Power Query incremental load example for querypower.com

This example demonstrates a Power Query incremental load setup for querypower.com, showing how a simple list source is transformed step by step. The code defines a query named `Query1` that generates a numeric range from 0 to 9, serving as the foundational data source. It uses the Power Query M language’s `let` and `in` syntax to structure the transformation pipeline, with `Source` assigned the output of `{0..9}`. The visualization highlights the initial stage of an incremental refresh pattern, where only new or changed rows would be processed in subsequent runs. The example focuses on the query definition itself, not on external libraries or rendering, as the M code contains no visual output or advanced APIs.

HHugoberry

65% match

Loading thumbnail…

Power Query script to get msmdsrv.exe port number via R

This example shows how to retrieve the port number of the msmdsrv.exe process by combining R scripts with Power Query’s M language. It first runs a system command via `system2` to capture the task list and netstat output, then parses the results using `read.fwf` with fixed-width columns. The M code invokes the R script through `R.Execute`, formats the resulting tables with `Table.TransformColumns` and `Table.PromoteHeaders`, and finally joins the process ID from the task list with the matching netstat entry to extract the local address port. The visualization relies on text-based data processing rather than graphical libraries.

HHugoberry

65% match