PyCon 2019 | Thinking Inside the Box: How Python Helped Us Adapt to An Existing Data Ingestion Pipeline

PyCon 2019 | Thinking Inside the Box: How Python Helped Us Adapt to An Existing Data Ingestion Pipeline

Speaker: Eddie Schuman

We will cover how we used Python to adapt to a large institutional processing setup. We used Python to create the definitions, configuration files, and supplementary metadata for each of the weather radars we worked with. We used a variety of custom tools to interface with existing systems and processes that would have been infeasible to work with otherwise.

We took advantage of one of Python’s greatest strengths: its flexibility. We used it to perform the bulk of our data processing with NumPy, created custom utility functions to encourage code reuse, and created custom scripts for interfacing with the institutional data processing framework we worked within.

Slides can be found at: https://speakerdeck.com/pycon2019 and https://github.com/PyCon/2019-slides