With the two.1 launch, Apache Flink additionally now helps Process Table Functions (PTFs), probably the most highly effective type of perform for Flink SQL and Table API, the PMC mentioned. Conceptually, a PTF is a superset of all different user-defined features, mapping zero, one, or a number of tables to zero, one, or a number of rows. This allows implementing user-defined operators that may be as feature-rich as built-in operations, the PMC mentioned. PTFs have entry to Flink’s managed state, event-time, desk change logs, and timer companies.
Apache Flink 2.1 additionally provides VARIANT as a knowledge kind for semi-structured information akin to JSON. This new kind helps storing any semi-structured information together with ARRAY, MAP (with STRING keys), and scalar varieties whereas preserving subject kind info in a JSON-like construction. Unlike the ROW and STRUCTURED varieties, VARIANT offers superior flexibility for dealing with deeply nested and evolving schemas. Users can use PARSE_JSON or TRY_PARSE_JSON to transform JSON-formatted VARCHAR information to VARIANT.
Also in Apache Flink 2.1:
