The need to enrich a fast, high volume data stream with slow-changing reference data is probably one of the most wide-spread requirements in stream processing applications. Apache Flink's built-in join functionalities and its flexible lower-level APIs support stream enrichment in various ways depending on the specific requirements of the use case at hand. In this webinar, I like to provide an overview of the basic methods to enrich a data stream with Apache Flink and highlight use cases, limitations, advantages and disadvantages of each.
WEBINAR: Detecting row patterns with Flink SQL.
I will discuss the new MATCH RECOGNIZE feature and present use cases that benefit from pattern matching support in streaming SQL, such as process monitoring or anomaly detection. I will demonstrate the feature with a few example queries.
Kubernetes has rapidly established itself as the de facto standard for orchestrating containerized infrastructures. And with the recent completion of the refactoring of Flink's deployment and process model known as FLIP-6, Kubernetes has become a natural choice for Flink deployments. In this talk we will walk through how to get Flink running on Kubernetes.
Talk Abstract: Introduction to SQL on streams concepts, explain how the new SQL MATCH_RECOGNIZE clause brings the power of pattern matching to (streaming) SQL and demonstrate advanced stream analytics with Flink SQL.
In my talk, I will discuss the new MATCH RECOGNIZE feature and present use cases that benefit from pattern matching support in streaming SQL, such as process monitoring or anomaly detection. I will demonstrate the feature with a few example queries and take a brief look under the hood to discuss a few details of the implementation.
Join the first meetup in Seattle of 2019 to hear talks about Apache Flink and Apache Hive integration and E2E Flink SQL Experience.
Hear latest Flink use cases in academia and industry from ETH Zürich and Teralytics.
Description: As data processing becomes more real-time, stream processing is becoming more important. Apache Flink makes it easier to build and manage stream processing applications. Flink's new SQL interface is not only a great way to get started with Flink, but also to build and maintain production applications.
This hands-on session introduces Flink via the SQL interface. You will receive an overview of stream processing, and a survey of Apache Flink with its various modes of use. Then we'll use Flink to run SQL queries on data streams and contrast this with the Flink data stream API.
Flink Forward speakers are experts from global companies like Airbnb, Alibaba, Lyft, Netflix, Splunk, Uber, and many more, who have built scalable streaming infrastructure and enterprise-grade applications.
Hear why and how they use Flink as the stream processing engine of choice for large-scale stateful applications, including real-time analytics, real-time search and content ranking, fraud/anomaly/threat detection.
Chat with the experts about Apache Flink, the stream processing platform for real-time analytics and real-time applications. Apache Flink provides efficient, fast, accurate, and fault tolerant handling of massive streams of events. Learn about the new Application Manager in dA Platform which makes deploying and running Flink in production much easier.
Talk Abstract: Complex Event Processing (CEP) helps at detecting patterns over continuous streams of data. DNA sequencing, fraud detection, shipment tracking with specific characteristics (e.g. contaminated goods) and user activity analysis fall into this category. In this talk, Kostas Kloudas will present Flink's CEP library and the benefits of its integration with Flink.
Talk Abstract: When working with Apache Flink users, we see many different types of stream processing applications being implemented on top of Apache Flink. Over time, we noticed common patterns and saw how most streaming applications can be reduced to a few core archetypes or "application blueprints". In this talk Aljoscha Krettek will outline a few of these "stream processing application blueprints" in a practical, hands-on format, and also take a look at operational considerations.
Talk abstract: Apache Flink is known today largely as a stream processor for high-volume streams with low latency, event time aware, supporting large and consistent state. Following the "batch is a special case of streaming" paradigm, Flink doubles also as a fast batch processor.
Talk Abstract: Come learn how the combination of Apache Kafka and Apache Flink is making stateful stream processing even more expressive and flexible to support applications in streaming that were previously not considered streamable.
The new world of applications and fast data architectures has broken up the database: Raw data persistence comes in the form of event logs, and the state of the world is computed by a stream processor. Apache Kafka provides a strong solution for the event log, while Apache Flink forms a powerful foundation for the computation over the event streams.
In this talk Stephan will discuss how Flink's abstraction and management of application state have evolved over time and how Flink's snapshot persistence model and Kafka's log work together to form a base to build 'versioned applications'.
Talk Abstract: Stephan Ewen will present an open source streaming data stack consisting of Pravega (stream storage) and Apache Flink (computation on streams). He will discuss how the combination of these two systems offers an unprecedented way of handling "everything as a stream", while dynamically accommodating workload variations in a novel way.
Pravega offers a permanent streaming storage, exposing an API than enables applications to access data in either near-realtime or at any arbitrary time in the future in a uniform fashion. Apache Flink's SQL and streaming APIs provide a common interface for processing continuous near-real-time data and set of historic data, or combinations of both. A deep integration between these two systems gives end-to-end exactly-once semantics for pipelines of streams and stream processing, and lets both systems jointly scale and adjust automatically to changing data rates.
Flink Forward is the conference for Apache Flink and stream processing communities. Join core Flink committers, thought leaders, and new and experienced users from companies like Google, Yelp, eBay, Lyft, Drivetribe, Alibaba, American Express, Netlfix, Capital One, Uber and many more to share experiences and best practices in stream processing, real-time analytics, event-driven applications, and managing mission-critical Flink deployments in production.
Check out the schedule and register here.
Join the second Apache Flink meetup in Seattle.
Talks by Fabian Hueske (data Artisans) on 'Why, when, and how to leverage the power and simplicity of SQL on Flink', and by Flavio Junqueira (DellEMC) on Pravega, a storage system designed and built to store unbounded amounts of stream data.
Talk Abstract: Stephan Ewen and Flavio Junqueira detail an open source streaming data stack consisting of Pravega (stream storage) and Apache Flink (computation on streams) that offers an unprecedented way of handling "everything as a stream" that includes unbounded streaming storage and unified batch and streaming abstraction and dynamically accommodates workload variations in a novel way.
Talk Abstract: Fabian Hueske and Shuyi Chen explore SQL's role in the world of streaming data and its implementation in Apache Flink and cover fundamental concepts, such as streaming semantics, event time, incremental results. They also share their experience using Flink SQL in production at Uber, explaining how Uber leverages Flink SQL to solve its unique business challenges.
Talk Abstract: This talk will start with brief introduction to streaming processing and Flink itself. Next, we will take a look at some of the most interesting recent improvements in Flink such as incremental checkpointing, end-to-end exactly-once processing guarantee and network latency optimizations. We'll discuss real problems that Flink's users were facing and how they were addressed by the community and dataArtisans.
Join the first Apache Flink London meetup of 2018.
Talks by Alex Garella & Aris Koliopoulos (DriveTribe) on designing applications with distributed streams, and by Patrick Lucas (data Artisans) on best practices around Flink state types (List/Map/ValueState et al). Patrick Lucas has expertise in containerised Flink Apps, so he is the person to ask, if you have any relevant questions.
Join us for our first Apache Flink meetup in Israel hosted by Clicktale and learn more about how their team uses Apache Flink.
Talks by Robert Metzger (data Artisans) on building a stream processor for fast analytics, event-driven applications, event time, and tons of state, and by Daniel Miodownik (Clicktale) & Gennady Gilin (Clicktale) on why Clicktale needs stateful processing and their use-case on using Flink at scale to build data products.