BlazingSQL is a GPU-accelerated SQL engine constructed on high of the RAPIDS ecosystem. BlazingSQL permits customary SQL queries to be distributed throughout GPU clusters, and the outcomes to be fed instantly into GPU-accelerated visualization and machine studying libraries. Basically, BlazingSQL supplies the ETL portion of an all-GPU information science workflow.
RAPIDS is a collection of open supply software program libraries and APIs, incubated by Nvidia, that makes use of CUDA and relies on the Apache Arrow columnar reminiscence format. CuDF, a part of RAPIDS, is a Pandas-like DataFrame library for loading, becoming a member of, aggregating, filtering, and in any other case manipulating information on GPUs.
For distributed SQL question execution, BlazingSQL attracts on Dask, which is an open supply instrument that may scale Python packages to a number of machines. Dask can distribute information and computation over a number of GPUs, both in the identical system or in a multi-node cluster. Dask integrates with RAPIDS cuDF, XGBoost, and RAPIDS cuML for GPU-accelerated information analytics and machine studying.
BlazingSQL is a SQL interface for cuDF, with numerous options to help large-scale information science workflows and enterprise datasets, together with help for the dask-cudf library maintained by the RAPIDS mission. BlazingSQL permits you to question information saved externally (similar to in Amazon S3, Google Storage, or HDFS) utilizing easy SQL; the outcomes of your SQL queries are GPU DataFrames (GDFs), that are instantly accessible to any RAPIDS library for information science workloads.
The BlazingSQL code is an open supply mission launched underneath the Apache 2.0 License. The BlazingSQL Notebooks web site is a service utilizing BlazingSQL, RAPIDS, and JupyterLab, constructed on AWS. It presently makes use of g4dn.xlarge situations and Nvidia T4 GPUs. There are plans to improve among the bigger BlazingSQL Notebooks cluster sizes to A100 GPUs sooner or later.
In a nutshell, BlazingSQL enables you to ETL uncooked information instantly into GPU reminiscence as GPU DataFrames. Once you’ve GPU DataFrames in GPU reminiscence, you should use RAPIDS cuML for machine studying, or convert the DataFrames to DLPack or NVTabular for in-GPU deep studying with PyTorch or TensorFlow.
BlazingSQL structure
As we are able to see within the figures under, BlazingSQL integrates SQL into the RAPIDS ecosystem. The first diagram exhibits the BlazingSQL stack, and the second diagram exhibits how BlazingSQL matches with different parts of the RAPIDS ecosystem.
Looking on the first diagram, BlazingSQL connects to Apache Calcite by way of JPype, and makes use of…
