BigQuery Engine for Apache Flink
Serverless Apache Flink integrated with BigQuery
Build and run real-time streaming applications with a fully managed Flink service that is integrated with BigQuery. Now in Preview.
Apache Flink® is a trademark owned by the Apache Software Foundation.
Features
Bring real time data to your unified data and AI platform
Make business decisions based on real-time insights with a well integrated and scalable streaming platform based on familiar technologies—Apache Flink and Apache Kafka. Combined with Google’s differentiated AI/ML capabilities in BigQuery, you can unlock the full potential of your data. Iterate rapidly without being slowed down by infrastructure management while having the ability to scale effectively with security and governance built-in.
Save time and resources with a serverless Flink engine
Organizations want to get value from real-time data by building streaming applications on Google Cloud. However, they are often bogged down by the operational burden of running self-managed Flink, fine-tuning countless configurations, meeting demands of diverse workloads while managing costs, and keeping up with upgrades. BigQuery Engine for Apache Flink is serverless; it reduces this operational burden and allows customers to focus on what they do best—their business innovation.
Compatible with open source Apache Flink
BigQuery Engine for Apache Flink makes it easier to lift and shift existing streaming applications relying on open source Apache Flink to Google Cloud, without rewriting code or relying on third-party services. Combined with Google Managed Service for Apache Kafka (now GA), it is easy to migrate and modernize your streaming analytics on Google Cloud.
How It Works
BigQuery Engine for Apache Flink lets you run Apache Flink in a fully managed environment within Google Cloud and connect it to BigQuery. It uploads your executable code and dependencies and creates an Apache Flink deployment or job. It then allocates resources to run the pipeline.
BigQuery Engine for Apache Flink lets you run Apache Flink in a fully managed environment within Google Cloud and connect it to BigQuery. It uploads your executable code and dependencies and creates an Apache Flink deployment or job. It then allocates resources to run the pipeline.
Common Uses
Streamling ETL
Streaming ETL for your AI-ready data platform
Streaming ETL for your AI-ready data platform
Apache Flink provides an open and flexible platform for real-time ETL, allowing you to ingest data streams from sources like Kafka, perform transformations (filtering, aggregations, and more), and instantly load them into BigQuery for analysis and storage. This enables rapid data analysis and faster decision-making with the benefits of open source extensibility and adaptability to diverse data sources.
Tutorials, quickstarts, & labs
Streaming ETL for your AI-ready data platform
Streaming ETL for your AI-ready data platform
Apache Flink provides an open and flexible platform for real-time ETL, allowing you to ingest data streams from sources like Kafka, perform transformations (filtering, aggregations, and more), and instantly load them into BigQuery for analysis and storage. This enables rapid data analysis and faster decision-making with the benefits of open source extensibility and adaptability to diverse data sources.
Build event-driven applications
Build event-driven applications
Build event-driven applications
Event-driven apps help enterprises solve for fraud detection models, recommendation engines, marketing personalization, and more. You can use Google Cloud's managed Apache Kafka service to capture real-time event streams like user activity or payments from various sources. The Apache Flink engine then processes these streams with low latency, enabling complex operations like real-time.
Tutorials, quickstarts, & labs
Build event-driven applications
Build event-driven applications
Event-driven apps help enterprises solve for fraud detection models, recommendation engines, marketing personalization, and more. You can use Google Cloud's managed Apache Kafka service to capture real-time event streams like user activity or payments from various sources. The Apache Flink engine then processes these streams with low latency, enabling complex operations like real-time.
Streaming analytics
Build a real-time data and AI platform
Build a real-time data and AI platform
BigQuery Engine for Apache Flink lets you run Flink for stream analytics without managing infrastructure. Analyze real-time data with Flink's SQL or DataStream API. Build dashboards by streaming your data to BigQuery and connecting it to visualization tools. Monitor job performance and leverage Flink's libraries for streaming ML.
Tutorials, quickstarts, & labs
Build a real-time data and AI platform
Build a real-time data and AI platform
BigQuery Engine for Apache Flink lets you run Flink for stream analytics without managing infrastructure. Analyze real-time data with Flink's SQL or DataStream API. Build dashboards by streaming your data to BigQuery and connecting it to visualization tools. Monitor job performance and leverage Flink's libraries for streaming ML.
Pricing
| BigQuery Engine for Apache Flink pricing | Usage is billed for resources that your Flink jobs consume, including compute slots and state storage. | |
|---|---|---|
| Service and usage | Description | Price (USD) |
Compute slot | Compute slots measure Flink application resource usage, billed per second. Usage is displayed in hours for hourly pricing. | Starting at $0.12 per hour |
State storage | State storage rate is a monthly rate. | Starting at $0.04 per GiB per month |
For full pricing details, visit the BigQuery Engine for Apache Flink pricing page.
BigQuery Engine for Apache Flink pricing
Usage is billed for resources that your Flink jobs consume, including compute slots and state storage.
Compute slot
Compute slots measure Flink application resource usage, billed per second. Usage is displayed in hours for hourly pricing.
Starting at
$0.12
per hour
State storage
State storage rate is a monthly rate.
Starting at
$0.04
per GiB per month
For full pricing details, visit the BigQuery Engine for Apache Flink pricing page.
PRICING CALCULATOR
Custom Quote
Getting started
Available in Preview today
Learn more about how it works
Works with Managed Service for Apache Kafka
2023 Forrester Wave: Streaming Data Platforms
