![]() They are processed to the USGS’s highest standard and then converted into a cloud enabled format that will stream and just work in analysis platforms, like a GIS. “These data are hosted by Amazon in their Open Data Registry for anyone to use for free. “With data releases like this, USGS is taking the lead to develop and release analysis-ready planetary science data,” Laura said. It took just under four hours of computing time to process the entire data set of >155,000 images, roughly 114 TB of data. Data were streamed from the NASA Planetary Data System cloud holdings, with more than 4000 images processed simultaneously. This feat of modern computing served as a demonstration of the cutting-edge HTCondor High Throughput Computing environment paired with the USGS Cloud Hosting Solutions Amazon Web Services environment. The team worked to test and refine the processing pipeline to ensure that it ran successfully and produced reliable, scientifically useful images and associated metadata. Releasing the entire HiRISE data catalog was the culmination of significant effort from a multi-disciplinary team of planetary scientists and cloud computing experts from across the USGS. This data release means that the HiRISE data set can now be seamlessly leveraged by machine learning scientists,” Laura said. “These data are ripe for discovery and use by machines and humans. Massive, fully processed data sets like this have been identified by the NASA Planetary Data Ecosystem Independent Review Board as necessary to support Artificial Intelligence and Machine Learning efforts in planetary science. USGS Astrogeology has led an effort to process over 155,000 HiRISE images and convert them to a stream-able form that can be easily accessed free of charge, making it possible to perform analysis in the cloud, access some of a data product without downloading the whole thing, view the images on a cell phone or tablet, or search and download them in batches in ready-to-use form. The level of detail in each image also means that the entire data set is rich with information that no scientist has had the time to study. The huge data size of HiRISE images is a blessing and a curse: to work with multiple images at once, scientists previously needed to download full images in their area of study one at a time, process them so that they can be viewed at different zoom levels, and add them to their map. Each individual HiRISE image covers only a small patch of the planet’s surface – strips just 6 km (3.7 mi) wide by up to 60 km (37 mi) long – but with a resolution of 25 cm (9.8 in) per pixel, the images can be upward of 1 GB apiece and are bursting with detail. USGS Astrogeology is also pioneering a new way to access and work with images from HiRISE. There are thousands of other potential CTX image pairs that Astrogeology is working to process, which will eventually result in even greater topographic coverage at 20 meters per pixel. They represent pairs that were collected alongside higher resolution image pairs from the High Resolution Imaging Science Experiment (HiRISE) camera. The 4,800 DTMs released so far are just the tip of the iceberg. When these data are highly accessible, anyone can contribute to scientific discovery.” We felt it was important to generate and release these products so that others could freely access the data. “Getting consistent, well aligned results is not easy. “These data are important because they democratize the availability of high-quality Mars topographic data,” Laura said. On a personal computer, this would have taken between two and 35 years of continuous processing! This took an enormous amount of computer processing, so Astrogeology used the USGS Denali super computer housed at the Eros Data Center in Sioux Falls, SD to process the data over the course of a few weeks. To do this, the USGS Astrogeology team first roughly aligned the individual DTMs to global low-resolution topography, then fed all of those approximately aligned DTMs back into the computer, working in batches of several hundred DTMs at a time. The NASA Ames Stereo Pipeline can be used to process individual pairs but aligning those pairs to each other and to global topography so that they can be seamlessly combined is no small feat. To produce a DTM, two overlapping images of the same area go through sophisticated computer processing to create a 3D view of the overlapping area, just as our brains process information from both eyes for depth perception. The Context Camera provides images with a resolution of ~6 meters per pixel covering swathes 30 km (18.6 mi) wide and up to 160 km (100 mi) long. The topographic data come from the Context Camera on the Mars Reconnaissance Orbiter. Visit Media to see details.įlyover of Valles Marineris, the "Grand Canyon" of Mars, highlighting two analysis-ready datasets provided by USGS.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |