Data Analysis from Kite-Based Sensors: Challenges of Importing and Processing Measurements from Multiple Devices
Abstract
Kites have a long history of being used for meteorological and atmospheric measurements. However for most of the 20th century, scientific measurements in the lower atmosphere have been conducted by ground stations, airplanes, drones, and balloons. Recently a community of scientists and engineers has promoted kites as effective vehicles for profiling atmospheric conditions in the lower atmosphere for community science, STEM education, and environmental research. This poster presents challenges posed by using relatively low-cost and lightweight commercial sensors to gather data and to process the data to make useful scientific conclusions. This presentation will focus on data collected from several sources: Kestrel 5500 weather meters, BlackSwift Technologies multi-hole probe, Anemoment Trisonica Mini ultrasonic 3D wind sensor, and custom-built sensors using Arduino-based or Raspberry Pi microprocessors. All these sensors provide data in text files - typically comma-separated-value (CSV) files but with different configurations for rows and columns of data and different conventions for naming variables. The process for importing data from all the sensors follows a series of typical Data Analysis steps, some of which need to be customized for the given sensor and file structures:
importing, cleaning up raw data trimming the dataset to only contain data from the field test identification and modification of appropriate time fields baselining specific data fields (i.e. to calculate the altitude above ground from barometric altitude) trimming the data again to focus on most relevant time frame output of processed data in format appropriate for subsequent processing plotting of relevant data in appropriate formats further analysis of data fields Even though all of these steps are straightforward to execute in spreadsheet software such as Excel, this process is cumbersome to complete individually on each data file, is prone to mistakes, and leads to output styles and nomenclature that varies between each field test. In our experience, these challenges also lead to long delays in processing data from field tests. This presentation will provide an overview of a python-based data analysis program that has been created to perform the Data Analysis steps listed above. There are several points of interaction where the analyst needs to provide input necessary for trimming and setting time fields, but the structured approach drastically reduces the time for processing the data. With data importing and analysis tools customized for the specific sensors, much more time can be spent on the subsequent challenges of comparing data across multiple devices to make conclusions about environmental observations from field test data.- Publication:
-
AGU Fall Meeting Abstracts
- Pub Date:
- December 2022
- Bibcode:
- 2022AGUFM.A11J..05C