Back to Volume
Paper: Instantaneous Archives
Volume: 521, Astronomical Data Analysis Software and Systems XXVI
Page: 20
Authors: Landry, W.; Monkewitz, S.
Abstract: IRSA, the NASA/IPAC Infrared Science Archive, is one of the largest and busiest astronomy archives in the world. In the past, our main emphasis was on making new data and new capabilities available. With the widespread implementation of Virtual Observatory protocols, there are a number of useful tools that can quickly and easily perform insightful, sophisticated queries from archives around the world (Taylor et al. 2019; Astropy Collaboration et al. 2013). The queries, if not handled quickly, can easily overwhelm the site and interfere with other users. In addition, reducing latency below the point of human perception enables more interactive and exploratory science. This paper discusses how we are re-architecting our query pipeline to eliminate process, file system, and database connection overheads. Taken together, these improvements delivered radical, order of magnitude improvements in latency and throughput.
Back to Volume