Schedule

Martijn Visser

Customers and companies are becoming more digital and the amount of available data is growing everyday. Traditionally, data engineers were needed to implement business logic via data pipelines before business users can start using it. In this talk, I will explain how data analysts and non-engineers can use only Flink SQL to explore and transform data into insights and actions, without writing any Java, Scala or Python code. This includes: * Exploring data sources, such as a realtime clickstream and a table with historical clickstream results * Displaying a fraud detection notification based on the customers clickstream pattern * Combining multiple data sources to display actionable insights on your website

Jelte Fennema

On certain workloads a single Postgres server cannot deliver the desired performance. If the traffic is read heavy, you could already resolve this by adding read replicas. If the traffic is write heavy then the upcoming Citus 11 release comes to the rescue. In both cases the queries of your application need to be sent to a randomly chosen Postgres server, to make these servers share the load. A TCP load balancer can do this easily. However, such a load balancer has a hidden downside. When you use it and try to cancel a query, you'll notice that cancelling only works some of the time. This talk will explain why this problem with cancelations occurs. And it shows a few ways to work around it, including changes that I proposed to Postgres and PgBouncer. If any of the following topics sound interesting then this talk is for you: 1. Postgres read replicas 2. Scaling writes with Citus 11 3. Running PgBouncer on multiple CPU cores 4. Implementation details of query cancellations

Francesco Tisiot

GDPR, data lineage, impact assessment, security scans are only a few situations where a proper metadata handling can turn a month-long project into a simple query. The talk is about the hidden gold stored in metadata: starting by defining use cases, we'll then focus on point solutions making metadata accessible for a precise technology. But companies' data efforts are rarely limited to a single tech, therefore we'll discuss how to design a system that collects, merges and makes available metadata coming from different tools. Finally we'll show an Open Source tool, the Aiven metadata parser, that can help harvesting and querying the metadata from a variety of different Open Source technologies. If you want to programmatically keep control over your dynamic data assets, this session is for you!

Henrietta Dombrovskaya

It’s a well-known fact, that although the database performance is great, and each query is executed in milliseconds, the overall application response time may be slow, making the users wait for a response for extended periods of time. The problem is not the database, but the way the application developers communicate with the database - using Object-Relational Mappers (ORM). The only way to change this behavior is to provide application developers with a tool, which is as easy to use, as an ORM, but which will allow escaping the common ORM pitfalls. That’s why we developed NORM – No-ORM Framework. To facilitate the adoption of the No-ORM approach, we developed a set of generators of user-defined types and functions. The input data is presented in a form of annotated JSON schema, which makes them easy to use by both database and application developers.

Steve Flanders

It has been said that open source is eating the world and in the observability space, the project behind this movement is OpenTelemetry. The project is quickly becoming the standard for instrumentation and collection of observability data. Why is an open standard and open-source approach to instrumentation and data collection so compelling? This talk will provide five reasons why OpenTelemetry is disrupting the observability market.

Dave Stokes

Percona Monitoring and Measuring (PMM) is an open source tool to help simply your MySQL, Postegresql, or MongoDB databases. This session covers what you need to know to get started with PMM and how to measure the performance of your databases. We will start with the starting PMM in a container, starting the agents for various database instances, and then introduce monitoring you instances. PMM can help streamline your operation of instances, enhance security, and automate administrative functions.

Francesco Tisiot

There's a rule in Italy that states: "pineapple doesn't belong to pizza". Yet it's a common choice around the world and a big discussion topic online. We'll use this funny example to show the power of the best streaming open source duo: Apache Kafka and Flink. We will initially showcase how data can flow in streaming mode through Kafka topics, and then add Flink on top to detect anomalies (yep pineapple, I'm looking at you), calculate aggregations, and enrich our pipelines with data coming from external systems like a PostgreSQL database. If you want to see the creation of a streaming data pipeline for anomaly detection in 10 minutes, this talk is for you.

Gunnar Morling

The saying goes that there are only two hard things in Computer Science: cache invalidation, and naming things. Well, turns out the first one is solved actually ;) Join us for this session to learn how to keep read views of your data in distributed caches close to your users, always kept in sync with your primary data stores via change data capture. You will learn how to - Implement a low-latency data pipeline for cache updates based on Debezium, Apache Kafka, and Infinispan - Create denormalized views of your data using Kafka Streams and make them accessible from a cache cluster close by - Propagate updates between cache clusters using cross-site replication In a live demo we'll show how all the pieces come together, of course connected via Apache Kafka.

Kirk Roybal

Join us as we take the mystery out of shared memory in PostgreSQL. This talk will give an overview of how the PostgreSQL shared memory system interacts with the underlying database components, and provide insight into the historical reasons for it being that way. You can expect to walk away with clarity on when data is loaded into memory, why it's kept there, and when it's evicted.

Cheuk Ting Ho

At the beginning of the talk, we will explore the role of revision control in promoting open-source software development. Cheuk will talk about how open-source software contribution is made possible with tools like git and GitHub. Next Cheuk will review how most database is lacking this revision control feature. She will also talk about why that is a problem and how we can improve. In this talk, Cheuk may demo some Python code with the use of TerminusDB, which is an open-source graph database. However, no Python knowledge will be needed to understand the topic and discussion in this talk. This talk is for those who work with data within a team and is seeking a better solution to collaborate and share data, making the data pipeline more efficient and benefiting data science teams in their productivity.

Gilles Barbier

Introducing the FLiPN stack which combines Apache Flink, Apache NiFi, Apache Pulsar and other Apache tools to build fast applications for IoT, AI, rapid ingest with Java, C#, Python or Golang. FLiPN provides a quick set of tools to build applications at any scale for any streaming and IoT use cases. Apache Pulsar enables Java applications to communicate asynchronously at any scale, geo-replicate and interact with non JVM applications. Pulsar also acts as a function mesh to run Java functions as a FaaS triggered by Events. All of this is open source and includes an integrated Schema Registry with support for JSON, Avro, Text and ProtoBuf schemas. Tools Java, Golang, Python, C#, Apache Flink, Apache Pulsar, Apache NiFi, MiNiFi, Apache MXNet, DJL.AI

Olena Kutsenko

A columnar database ClickHouse is in many ways exceptional - it is exceptionally fast, exceptionally efficient, but also, exceptionally confusing. It's approach to handling data goes against principles we use in other databases: its primary index doesn't index each row and doesn't guarantee uniqueness; a secondary index doesn't point to specific rows; JOIN to be avoided and transactions do not exist. But if used correctly, ClickHouse is a superb solution for online analytical processing. The goal of this talk is to help you get the most of ClickHouse and avoid pitfalls. We'll talk about analytics, columnar databases and sparse indexing. We'll look at the most puzzling concepts of ClickHouse, so that by the end of the talk you find them not only logical, but maybe even fascinating.

Mark Needham, Karin Wolok

In this session we will learn how to build our own real-time analytics application on top of a streaming data source using Apache Kafka, Apache Pinot, and Streamlit. Kafka is the de facto standard for real-time event streaming, Pinot is an OLAP database designed for ultra low latency analytics, and Streamlit is a Python based tool that makes it super easy to build data based apps. After introducing each of these tools, we’ll stream data into Kafka using its Python client, ingest that data into a Pinot real-time table, and write some basic queries using Pinot’s Python SDK. Once we’ve done that, we’ll bring everything together with an auto refreshing Streamlit dashboard so that we can see changes to the data as they happen. There will be lots of graphs and other visualisations!

Karen Jex

Containerised databases were once restricted to small, development environments, but this is no longer the case. It is become more and more common to see large-scale enterprise PostgreSQL clusters on Kubernetes. Why would you want to do this, and how you can go about deploying this type of architecture whilst ensuring you have the levels of high availability and disaster recovery you need and expect?

Heikki Nousiainen

Aiven is on a mission to help developers adopt and leverage Open Source data technologies in the public cloud. In this talk, we'll take a look at Aiven history and why was it started. We take a look at our engineering principles and our implementation to operate these technologies in a secure and reliable manner at scale. We share our lessons learned, and some of our insights and predictions on the importance of Open Source to developers and business alike.

Elisa Usai

Do you already work with DevOps tools and Open Source technologies? Are you curious about databases in containers? During this session, I will drive you through the design and setup of MySQL in Kubernetes. You will discover different solutions to run this stack: MySQL single instance vs MySQL InnoDB Cluster, using MySQL Operator for Kubernetes or Helm Charts, on-premise but also through Oracle Container Engine for Kubernetes on OCI, and much more. Let's have a look together!

Steve Flanders

OpenTelemetry is an open-source and vendor-agnostic project set on democratizing instrumentation and data collection of telemetry signals. Telemetry data is critical in solving both availability and performance issues, especially in distributed, polyglot architectures. Unfortunately, telemetry data has often been proprietary, at least when it comes to ease of use or combining different signals. OpenTelemetry looks to solve this problem by offering flexibility and choice combined with an open-standards approach. The project is extensible and offers support for a variety of open-source and commercial back-ends. In this session, you will learn about the components of OpenTelemetry and how you can get started.

Alexey Milovidov

ClickHouse is known for being fast. Fast at ingesting data. Fast at querying data. But how do we achieve this? Is a slower release a major regression? Or just something to be concerned about? How does the ClickHouse engineering team think about performance? In this talk, Alexey will share a bit of the 'internals' of ClickHouse. How we engineer for performance, why we ask for external benchmarks, performance testing in CI pipeline, etc. To build a fast database, you need a vision, a team, and process. We will discuss all 3 of these.

Frank Lyaruu

We all know and love our micro service architectures. As every service has ownership of its own code base and data, we can iterate on each service separately. However, sometimes we need to combine data from different services. In a monolith, it could be a single SQL join. In most micro service systems, this is not easy to do efficiently. This aspect tends to be overlooked when discussing micro services: Try as we might to decouple systems into sensible domains, sooner or later there will be a business requirement that does goes right across those boundaries and we will need to find a way to reunite this data. We will look at Debezium, a change capture solution by Red Hat, and Apache Flink, a stream processing framework, and of course a little bit of Kafka.

Daniel González Lopes

Grafana k6 is a developer-centric and open-source load testing tool built for making performance testing a productive and enjoyable experience. k6 makes it easy to catch performance regression and problems earlier, allowing you to build resilient systems and robust applications. During this talk, you will learn (in a very practical way) everything you need to start testing all the pieces that compose your complex software systems, from microservices APIs to all kinds of Databases, Queues, and other critical components.

Christopher Hoult

As they grow, companies continue to produce and ingest more and more data in the form of events, instrumentation and logs, but all too often just collect it in a central data store for analysis after the fact - if at all. Frequently this requires long ETL processes, combining disparate data sets for context missing at run-time, and slow ad-hoc queries. In this talk, I will give an introduction to Benthos - which promises "fancy stream processing made operationally mundane" - and present some effective, configuration-centric techniques to not only improve your real-time processing efforts, but to derive more business value from your data flows.

Itamar Syn-Hershko

I have been working with Elasticsearch since its dawn, started over 10 years ago with version 0.12, and saw Elasticsearch becoming the de-facto standard technology for search, monitoring and log analytics. Today, OpenSearch (fka Elasticsearch) is widely used as a serving layer for websites and applications; for workloads of search, faceting, simple analytics, and more. Like every technology, OpenSearch has its own shortcomings you have to avoid to implement it successfully. As new modern technologies emerge, they may be more suited for some use-cases then good old Elasticsearch / OpenSearch. This session is about two of those technologies I consider the most prominent – ClickHouse and Vespa. In this talk we will walk through the common use-cases where Elasticsearch is used, give a quick technical introduction to Vespa and ClickHouse, and discuss the use-cases they excel in, and when you should consider. And of course - their shortcomings as well.

Dave Stokes

Prepending EXPLAIN to your query will show you how the query optimizer wants to do the needed work to return the requested data from your database. However the output from EXPLAIN is not exactly easy to decipher. This session covers the basics of EXPLAIN use with PostgreSQL and MySQL so that you can understand what the out really means. You will learn the heuristics behind the choices made by the optimizer, some of which may not be optimal (such as MySQL's default of using the smaller of two available indexes), and some basic optimizations to speed up queries. And you will learn how to test and compare different optimization strategies for your queries.