Webclient Caching with PowerShell

In Advanced Event 7 of the 2011 Microsoft Scripting Games, the goal was to map names to Twitter screen names. The information was to be gathered from the SQLSaturday site, but I wanted to add the functionality to query the actual Twitter API for more information.

Much of the Twitter API functionality requires authentication through oAuth, but I didn’t want to add that level of complexity. Fortunately, there are some useful things that can be done without authentication. The users/show API accepts a Twitter screen name and returns a lot of information about that user in an xml or json format. This is what I wanted to use to add some functionality to my PowerShell script.

While this particular API can be accessed anonymously, it is also subject to more restrictive rate limiting. This means I was only going to be able to make 150 requests per hour. This can be a real bummer when trying to test a script. Since I was retrieving the same information over and over, it seemed like a good idea to cache the data locally.

First, I generated a path to the CSV file to be used for the local cache. I wanted to use a specific name for the file, so that it would be found on subsequent calls to the function.

With the path to the cache file determined, its existence is checked and the file is imported if found.

With the cache loaded into the twitterCache variable, I check to see if it contains the screen name I’m looking for.

If the screen name wasn’t found in the cache, the Twitter API is queried and the returned information is added to twitterCache.

The updated cache is then written back to the CSV file.

I wrapped this functionality into the Invoke-TwitterQuery function, so that the rest of my script wouldn’t care if the data was cached or not. It just calls the function and doesn’t have to worry about how the information is being retrieved. It also means I could make adjustments to Invoke-TwitterQuery, like storing information in a database instead of a CSV file, and the rest of the script wouldn’t need to be changed.

All of my entries for the 2011 Scripting Games can be found at PoshCode.

Complete Script: