Back to All Events

Meetup: Building Modern Data Pipelines



Join us on Monday, Feb 26 as we welcome back the NYC Real-Time Stream Processing User Group for a talk with Karthik Ramasamy of Streamlio.



Data pipelines are hard to build and maintain. This is due to complexity of big data open source ecosystem that has numerous software each specializing in solving one piece of the puzzle. In this talk, we will focus on three key open source software Apache Pulsar, Apache Heron and Apache BookKeeper and how are integrated to make it easy to build data pipelines.



For today’s enterprises, ensuring that data pipelines are available to every corner of the organization is key to building next generation data-driven applications. In this talk, Karthik Ramasamy of Streamlio will present on how to combine three best of breed open-source projects to have a solid data infrastructure that are is easy to develop against and simple to operate at scale in production.


Karthik will provide an overview of the merits of the three open source systems and the benefits they bring when integrated:

Apache Pulsar: unified queuing and streaming

Apache Heron: stream processing

Apache BookKeeper: distributed stream storage


About Karthik:

Karthik Ramasamy is the co-founder of Streamlio that focuses on building next generation real time processing engines. Before Streamlio, he was the engineering manager and technical lead for real-time analytics at Twitter where he co-created Twitter Heron. He has two decades of experience working in parallel databases, big data infrastructure, and networking. Karthik is the author of several publications, patents, and "Network Routing: Algorithms, Protocols and Architectures". He has a Ph.D. in computer science from the University of Wisconsin, Madison with a focus on big data and databases.



6:45 PM: Doors open

7:15 PM: Welcome introduction

7:30 PM: Presentation begins

8:45 PM: Program ends


Your Host

Eugene Dvorkin, Technical Architect Infrastructure Engineer, Lifion

    Earlier Event: February 21
    Meetup: Apache TinkerPop’s Gremlin
    Later Event: March 6
    Meetup: UX Support Group