Category Archives: Software

Exploring GeoMaker and Yahoo! Placemaker

This week I was motivated to play with some of Yahoo’s Geo tools after hearing a talk by Christian Heilmann regarding the same.

One tool in particular is his Geomaker application which “creates microformats and maps from geographical information embedded in texts”. GeoMaker feeds your provided text into Yahoo’s Placemaker service which will identify any geo locations referenced. Geomaker then massages this data into a Yahoo map widget which can be placed on any web page such as a blog entry.

Geomaker accepts either a block of test, a web page, or an RSS feed. The RSS feed option is a great way to quickly visualize to locations being referenced. For example, the following is a map of recent items from the Yahoo! World News RSS feed:


To stress test Placemaker and see how accurate it is, I dropped in the lyrics for Johnny Cash’s rendition of “I’ve Been Everywhere“. Results? It identified 70 place references and missed 22. The detector seems to be less accurate when given place names like this in a list. It does better when places are used in context, like the news articles above.

Here is the resulting map for “I’ve Been Everywhere” (minus the outlying countries Argentina and Costa Rica):

Lorem Ipsum Text via YQL

Still finding more reasons to play with and learn YQL, and following up from my experiment with random numbers, today I implemented YQL support for lipsum.com, a “Lorem Ipsum” text generator.

What is Lorem Ipsum text? The lipsum.com site explains:

Lorem Ipsum is simply dummy text of the printing and typesetting industry. Lorem Ipsum has been the industry’s standard dummy text ever since the 1500s, when an unknown printer took a galley of type and scrambled it to make a type specimen book. It has survived not only five centuries, but also the leap into electronic typesetting, remaining essentially unchanged. It was popularised in the 1960s with the release of Letraset sheets containing Lorem Ipsum passages, and more recently with desktop publishing software like Aldus PageMaker including versions of Lorem Ipsum.

I made a simple custom data table to access this service via YQL. To use these services, you’ll need to reference the custom table I created, then “select” some random text. The service can create text in a specified amount of either paragraphs, words, bytes, or lists. You can also control whether or not you want each string to start at “lorem ipsum…”, or to start randomly.

For example, to request 5 paragraphs (try it in YQL Console):

use "http://scooterlabs.com/yql/lorem.ipsum.xml";
select * from lorem.ipsum
where amount="5" and what="paras";

To request 20 lists (try it in YQL Console):

use "http://scooterlabs.com/yql/lorem.ipsum.xml";
select * from lorem.ipsum
where amount="20" and what="lists";

Random Numbers via YQL

I recently discovered a very cool random number generator site appropriately called Random.org. After poking around a bit at the various generators available on the site, I discovered they also have a simple “http service” and sample client apps which use it.

As a quick exercise, I made custom data tables to access these random number services via YQL, Yahoo’s “select * from internet” service. To use these services in YQL, you’ll need to include the custom tables I created, then “select” some random numbers.

For example, to request a few random numbers (try it in YQL Console):

use "http://scooterlabs.com/yql/random.integers.xml";
select * from random.integers
where num="10" and min="1" and max="1000";

To request a random sequence, where every number in the range is used once (try it in YQL Console):

use "http://scooterlabs.com/yql/random.sequences.xml";
select * from random.sequences
where min="10" and max="19";

To generate some random strings (try it in YQL Console):

use "http://scooterlabs.com/yql/random.strings.xml";
select * from random.strings
where num="10" and len="20";

For a full description of each service, and more optional parameters, refer to the Random.org HTTP interface description.

Update 2010-02-25: I heard back from the owner of the Random.org site who said he has been blocking all Yahoo! Pipes traffic. So, this YQL query may stop working at some point.

RSS to CSV Converter

Recently @tonyadam was asking about for a way to import Twitter search results into Excel via RSS or CSV. I couldn’t find a suitable method on my slightly out of date Excel 2003, but sent Tony a link explaining how Excel 2007 could get him closer to what he wants. Turns out he was on the Mac anyways, so we needed an alternative way.

Importing into Excel usual involves using the CSV format. So all we needed was an RSS to CSV format converter, but surprisingly couldn’t find anything relevant on the web. In general there isn’t much interest in XML to CSV conversion because it would only make sense for the simplest of XML files — basically, one table with rows & columns. Anything more complicated won’t fit within the confines of CSV.

To solve this problem, I created an RSS to CSV converter script, using a bit of YQL for the feed fetching and parsing.

We can test it using Tony’s original request — Twitter search results — as follows:

  1. Login to twitter
  2. Search for something (my sample: nascar)
  3. On the right-hand side, copy the “RSS feed for this query link”
  4. Go to RSS to CSV converter
  5. Paste in Twitter RSS link, changing “.atom” to “.rss” (my sample: http://search.twitter.com/search.rss?q=nascar)
  6. Click Submit
  7. Save CSV file when prompted, or import into Excel

Update 2011-09-11: Twitter’s UI changes have removed the old links for search result RSS feeds, so the above steps won’t quite work. See this article from Learning LibTech for an alternative way to find the feeds directly. Also, the source code for this script is now on Github.

One problem I discovered (again) is that Excel is not happy with Unicode characters in CSV format. There are some alternatives described, so I may enhance this script to properly handle UTF-8 if someone has a use-case for it.

Yahoo! 360° Closing Today

The Yahoo! 360° blogging/profile/social networking product launched in April 2005. After about 2 years, talk of closing down the service began, but was delayed until there was a similar product to which existing users could switch. Now, 2 years after that announcement, the closure is finally here. Today is the last day 360° will be online and available for users to download or migrate their old blog content over to the new Yahoo! Profiles.

I did not work on Yahoo! 360­°, nor was I a serious user of it. I’m sure a lot of existing users are frustrated either because Profiles doesn’t do what they want, or the simple fact that it’s just different. I will say that if a product like this is no longer a high priority internally and does not have a clear future, it’s really best to close it down (in an orderly fashion) and redirect those internal resources elsewhere. Even without any new feature development, the ops impact alone for keeping a service like this online is pretty significant.

Yahoo! 360 Closure Notice Screenshot

For those that are looking for help with 360° migration or other issues, please see the online help.