super-cooper

I had a post a few days ago about data, and how it made my happy, and how that was kinda pathetic. And that got me to thinking – What’s the largest raster I’ve processed?…..Hhhmmmm. Well, here it is:

I needed to make shaded relief maps for our site at work, like the one seen here. In order to do this, I needed a DEM that covered a very large area. Namely, all of Arkansas, most of northern Louisiana, eastern Oklahoma, western Mississippi, southern Missouri, along with chunks of Kansas, Texas, Tennessee, Kentucky, and Illinois.

Oh yeah, and to calculate the hillshade, the DEM has to be seamless. I knew this would take a while. First, I downloaded all of the DEM data from the USGS Raster Extractor. Hell, it’s free, so get all that you want. I ended up with 56 tiles of unprojected DEMs (the image above). Each zipped chunk was about 100MB, and all of the data unzipped came to about 4.87GB. I tried to mosaic these, but found out there was a problem; that being that ArcInfo can only address a file 2.147GB in size. So even if I did mosaic these now, I wouldn’t be able to open it. Hhhhmmmmm. So I realized I had to convert the rasters from floating point decimal (i.e., elevation values of say, 21.2223 meters) to integer (remove the decimal places); this makes the rasters much, much smaller in file size. And since I wasn’t going to use this big DEM for anything but to make maps, I don’t need the decimal places.

Next I mosaiced them into five chunks to make the final mosaic a little easier on my machine, plus, the maximum number of grids you can mosaic at once in ArcView is 50. These processes really didn’t take that long on my ole’ trusty steed though; she’s a Dell Precision Workstation with dual 2.4Ghz Xeons and 2GB of RAM. Finally, I mosaiced the five larger mosaics into one, and then projected it into NAD83, UTM Zone 15 North, which is what I pretty much work with all of my data in this area in.

Now the projecting took a while, several hours if I remember correctly. That’s still not bad though. A few years ago, if I would have attempted this on a machine with 1Ghz P3 and 1GB of RAM, it would have taken all night to process, that is if it didn’t crash after a few hours. The filesize of the final integer, mosaiced, projected, full-size DEM (seen at right) is about 750MB, which is much smaller than the 4.87GB I started off with.

So what’s the point of all of this? Nothing really, other than it’s fun to push your equipment (and your knowledge of it, or how to work around its’ limitaions) to the limits sometimes, just to see what it (and you) can accomplish.