File Under: JavaScript, Programming

Mash up Data Sources to Find iPhones

When the iPhone 3G was released, it was a difficult item to find. To help communicate about inventory, Apple created an availability tool which listed all the stores in a particular state and whether each had stock of the three models.

We at Webmonkey thought that was great, but that extreme Apple fanboys would want to know more. The truly committed might even drive across state lines to get their hands on just the right color iPhone. We knew we needed to be able to check inventory not just by state, but by proximity.

This article details how we made the iPhone locator from an Apple feed, a Google Map and a little Web 2.0 baling wire.

Download the source files and follow along below.

Contents

  1. What You Need to Know
  2. Apple’s JSON Feed
  3. Apple Store Locations
    1. Find the List of Locations
    2. Use Microformats to Extract Address
    3. Geocode Each Address
  4. Incorporate Google Ajax Map With Search
  5. Calculate Proximity
  6. Check Stock
  7. Go Forth And Mash

What You Need to Know

This tutorial puts together a whole bunch of different web technologies. We can’t go in depth into each of them, but we’ll do our best to point to other resources to fill in the gaps.

That qualified out of the way, you will get more out of this tutorial if you have:

  • Knowledge of JavaScript and HTML
  • Familiarity with JSON and/or JavaScript objects
  • Willingness to dig through someone’s code


Apple’s JSON Feed

The crux of this mashup is the data available via Apple’s JSON feed. An example copy of the original feed is available in the source as 3g_us_inv.json.

JSON is a format for describing data that is easily converted to most programming languages, especially JavaScript. In fact, the J in JSON stands for JavaScript. The code from a JSON file can look like gobbledygook, but it’s highly organized gobbledygook.

Here’s a shortened version of Apple’s feed:

{"date":"Wed, 16 Jul 2008 00:00:00 -0700","locations":{

"AL":[{"name":"The Summit","storeid":"R225","city":"Birmingham","url":"/retail/thesummit/","available":{"black8":false,"black16":true,"white16":false}},{"name":"Bridge Street","storeid":"R266","city":"Huntsville","url":"/retail/bridgestreet/","available":{"black8":false,"black16":false,"white16":false}}],

"AZ":[{"name":"Chandler Fashion Center","storeid":"R026","city":"Chandler","url":"/retail/chandler/","available":{"black8":false,"black16":true,"white16":false}},{"name":"Biltmore","storeid":"R031","city":"Phoenix","url":"/retail/biltmore/","available":{"black8":false,"black16":true,"white16":true}},{"name":"La Encantada","storeid":"R086","city":"Tucson","url":"/retail/laencantada/","available":{"black8":false,"black16":false,"white16":false}},{"name":"SanTan Village","storeid":"R267","city":"Gilbert","url":"/retail/santanvillage/","available":{"black8":false,"black16":false,"white16":false}}],

...

...

}}

In Get Started With JSON we dive deep into how to structure JSON files, but here’s a quick run-down of the data in Apple’s feed:

  • The date the feed was published
  • A locations object which contains an array of states, each containing an array holding an object for every store in that state.
  • Within each individual state object, we have the name of the store, its ID, the URL to the store off the Apple homepage, and the availability data for the three models of iPhone.

Since JavaScript works so well with JSON, it just takes one line to convert JSON text into a JavaScript object. Assuming the entire text feed is stored in a variable called jsontext, here’s the magic line of code:

jsonobj = eval("(" + jsontext + ")");

Now you can access individual pieces of data within the feed. For example, here’s how you retrieve the name of the first Arizona store:

jsonobj["locations"]["AZ"][0]["name"]

The result, as those who are able to read through the JSON code above will see, should be Chandler Fashion Center. Success! We are able to get at the availability data in Apple’s feed. Now we’re going to have to put this JSON feed on the back burner while we figure out a few more pieces to the puzzle.


Apple Store Locations

One important piece of information missing from Apple’s feed is the location of each Apple Store. We know the city, but not the address, and certainly not the geo-coordinates (latitude and longitude) that will be necessary to compute the closest store.

Find the List of Locations

Luckily, we already have a list of locations. Even better, we have the link to each location’s homepage, where the address is listed. From the JSON text, we found each URL, visited the site, and extracted the address.

This section will not go into detail of how we coded scripts to harvest the Apple Store geo-coordinates, but we will look at the technologies used.

Use Microformats to Extract Address

Apple made it easier to retrieve addresses on their store websites by using a Microformat called hCard. Microformats allow content producers to add additional tags to their HTML to signify special content. In the case of hCard, the tags explain the individual pieces of contact information.

On the website of my local Portland Apple Store, for example, here’s what the code of the address section looks like:

<div class="hcard">

	<strong>Address:</strong>

	<p>

		<span class="fn org">Apple Store, Pioneer Place</span>

		<span class="adr">



			<span class="street-address">700 Southwest 5th Avenue, Suite #1035</span>

			<span class="locality">Portland</span>, <span class="region">OR</span> <span class="postal-code">97204</span>



		</span>

		<span class="tel">(503) 222-3002</span>

	</p>

</div>

Geocode Each Address

After retrieving an address for each store, we had to convert it to geo-coordinates. To do this we used Google’s HTTP Geocoder. The CSV output option made it very easy to only get the pieces of data we wanted: the latitude and longitude points.

Unfortunately, several addresses were unable to be geocoded. Since many Apple Stores are in malls, they often have suite numbers. These caused problems almost every time they were present. In some cases, like my local store, the latitude would be 20 degrees off! We couldn’t have that throwing off searches for nearby iPhones.

We decided to give up a little granularity in exchange for reliability. Instead of full store addresses, we used Zip codes.

Like in the previous section, we won’t go into detail about geocoding all 188 Apple Stores. You can find out more about geocoding in our Google Geocoder tutorial. You can also find the individual store coordinates in the file stores-geocoded.txt in the source files.

As a final step with geocoding, I needed to add the latitude and longitude coordinates to the JSON file. I inserted them after the URL, but before the availability information. For example, here is the Chandler Fashion Center portion of the JSON file, this time with geocode information:

{"name":"Chandler Fashion Center","storeid":"R026","city":"Chandler","url":"/retail/chandler/", "lat": 33.266332, "long": -111.943009,"available":{"black8":false,"black16":false,"white16":false}}


Incorporate Google Ajax Map With Search

Now that our data sources are in order, it’s time for the fun stuff. We started with Google’s simple Ajax geocoding example. It has a text box for address input, a feature we wanted for our tool, too.

In Google’s version, when the user input is geocoded, it is then placed on the map. We wanted to step into the middle of that process. Instead of mapping the point, we wanted to compare the point to all the coordinates of Apple Stores.

These two lines in the searchStores function start the process:

currentPoint = point;

getStores();

The first line stores the user’s point globally. The second line is a function that grabs the JSON text and sends it off for perusal.

We chose to grab the text with Ajax using the Prototype JavaScript framework. We could have hard coded the JSON in JavaScript, but using an external file makes it easier to update.


Calculate Proximity

Finally, we made it to the part where we find the closest Apple Store. When the findClosest function is called, we already have the user’s location, as well as the JSON text for every store, including its coordinates.

To determine proximity, we decided to iterate through each store and calculate its distance from the user’s point. At each step, if we found a store closer than the previous closest it gets stored as the current winner. When we’ve considered every store, the current winner is the closest of all stores.

Calculating the distance between two pairs of coordinates requires a little math. It’s essentially based on the Pythagorean theorem, which makes it more complicated than it is. We want to find the hypotenuse of a right triangle. The hypotenuse represents the distance between the two points, as the crow flies.

Here is a function to calculate the proximity of two coordinate pairs:

function calculateDistance(pt1, pt2)

{

  var xdist = 69.1 * (pt2.x - pt1.x);

  var ydist = 53.0 * (pt2.y - pt1.y);

  var dist = Math.sqrt(xdist*xdist+ydist*ydist);



  return dist;

}


Check Stock

You may be feeling pretty good at this point. We have located the closest Apple Store! Unfortunately, we have not checked stock at any stores, which was the point of this tool.

Before declaring a current winner for closest store, we must check that the store has iPhones. Here we employed an if statement and only declare a current winner if the statement is true.

Assuming thisstore contains the JSON object for the current store we are checking, here is the if statement:

if (thisstore.available["black16"] || thisstore.available["black8"] || thisstore.available["white16"])

This says that if a black 16 GB iPhone is available, or a black 8 GB is available, or a white 16 GB is available, then the store has stock. In other words, if the store has at least one model in stock, count it.

Of course, in our tool, we offer the option to search for a specific model of iPhone. That required a little bit of logic play. Essentially, if the user wants to search for only one model, that is all we check. Otherwise, we use the above statement to check for any model.


Go Forth And Mash

We don’t expect your will create mashups with iPhone inventory. By the time you’re reading this, the iPhone 3G is probably obsolete. Through this process, we hope you have gained some ideas about how to piece together the many different web technologies to create something altogether new.

If you have questions, check out the discussion tab. If you have advice to add, please consider editing this article.