Secrets of the City

I thought these types of mashups typically were done using Google Maps.

Mmmm, gots to get me some “tater leeks.”

No offense to the poster, but I hope this strange little fad dies off soon. It was only a matter of time until someone did Minneapolis. Now it’s done. nootropic has filled the Minneapolis gap in this meme. Yay.

How about something useful, like a mashup of google maps with bus routes and schedules? That would be rad.

‘Fanatical Somali Metro’ = Mall of America.

Oh my.

How about something useful, like a mashup of google maps with bus routes and schedules? That would be rad.
»» Submitted by mike s at 11:06 AM on February 27

google debuted just that last year… unfortunately limited to just portland, OR right now…

Fanatical Somali Metro: a reference to the line’s nickname, the ‘Somali Trolley’?

How about something useful, like a mashup of google maps with bus routes and schedules? That would be rad.

»» Submitted by mike s at 11:06 AM on February 27

I’ve been wanting to do this myself, actually. I even emailed metrotransit to ask about their trip planner and how it handles the data, but unfortunately they can’t release the proprietary information so it would require someone to manually type in all of the schedules by hand. No thank you! If they ever manage to put the schedules out in HTML or another format that isn’t PDF I might have a go at it. Making all the gpoints for every route is still tedious, but a lot more do-able than having to type the schedules as well (especially when a lot of them change every few months).

I was invited to be in a focus group for Metro Transit on March 16th concerning their online trip planning. Thanks to gerg for pointing to Google Transit which eventually led me to here. I’ve had some initial ideas and feedback for MTC but I figure the more informed I am the better I’ll be able to influence them (although I’m guessing the people running the focus group may not appreciate that – they probably just want feedback and answers to their questions).

Ultimately IMHO, if they’re able to open up their information more and work with some common frameworks, the public and other businesses will do half of the work for them. I also still trust Google enough to strongly recommend they do whatever they can to be a part of Google Transit. It’ll eventually be a lot easier for business websites to be able to integrate dynamic transit information for people wanting to travel to their location. Developers could easily make custom applications built for mobile phones and mobile devices that are able to use transit information optimized for the platform or device… especially nifty with location-aware devices (i.e. type in where you want to go, and it tells you were to walk or what bus/train to take from your current location and how much time you have left before your ride arrives… all without needing to read maps). I’m salivating.

trigonalmayhem, I noticed that you can download the schedules in text-searchable PDF format. For example, the bus routes I would take to work in the morning include:

14R 6:54 6:57   7:01 7:07 7:17 7:26 7:34 7:39 7:49  7:53
14R 7:03 7:06 7:13 7:19 7:29 7:38 7:46 7:51 8:01  8:05
14N 7:18 7:21   7:25 7:31 7:41 7:50 7:58 8:03 8:13 8:16 8:19
14R 7:27 7:30 7:37 7:43 7:53 8:02 8:10 8:15 8:25  8:29
14R 7:43 7:46   7:50 7:56 8:06 8:15 8:23 8:28 8:38  8:42

I also tried using (1) Adobe’s online converter and (2) Acrobat Pro 6.0 for the conversion. Even on my slow-ass computer, the former is much faster than the latter, but the output is weird. I think the fastest way to get the data would be to manually copy it out of the PDFs.

Personally, I think it would be great for local businesses to be able to have links to a googlemaps page that will show what routes go nearby and allow users to plan their own schedules. I use buses all the time, but MTC’s TripPlanner is just brutal for going to places when I don’t know what route to take.

mike s,

funny you mention that because it occured to me between when I posted that and now that I might be able to mine the pdfs for the data. It won’t be fun, but if I concentrate on the core bus routes and leave out all the 100+ numbered ones it might be easy enough to do for a proof-of-concept, at least.

Integration with local businesses would be a great thing too, but then you’d be crossing the line into a commercial use from a public service use, sort of. There might be issues then with metrotransit or some of the firms that own some of the specific sets of data (no idea if they frown upon that or not).

Ultimately I would hope that google integrates transit into their other local services and expands the ability of local businesses to customize their listings on it rather than using the default and often incorrent yellow pages type listing service they have now.

Oh, and to s4xton:

I exchanged emails with someone at metrotransit about just that sort of thing and he mentioned that they didn’t see the demand for mobile phone applications, especially on the tight budget they’re forced to live with. A big part of why metrotransit doesn’t offer a lot of these great services is not that they’re lacking in ideas or inspiration (they have both from what I could tell), but they’re lacking in funding. They had to save up part of their budget for several years just to upgrade their backend servers for the trip planner.

Another thing I was told is that a lot of the different information sources are owned in some way or another by the firms that they contract out to for the services. That might make it difficult to get information on the framework for the user base to make these sorts of things available, unfortunately.

Maybe if they weren’t paying out the ass to run the train they could do some of this stuff. They do have a big map of the metro that shows all the routes. It’s a bit cumbersome, but that’s what I use a lot.

I don’t think we’re paying out the ass for the train, Kevin. It’s the bus routes to the suburbs that don’t pay. And God knows the legislature would rather just let Minneapolis and St. Paul freeze in the dark so everyone in Happy Valley can have a four lane road right to their own cul de sac.

I find it sad how underrepresented the actual urban areas of the metro area are in the met council. With the governor and state legislate unwilling to provide enough funding of course the suburban-minded met council is going to cut urban bus service before suburuban lines. They should really spin metrotransit back off into an independent body concerned only with mass transit (and give it a decent budget too).

Oh and the dial-a-ride paratransit services are always huge losers for transit services, but they’re required to run them by law (I think). I was amazed at how much of the budget the paratransit service for suburban towns ate up back in Worcester, MA where I went to school. It was like half the total transit budget.

But back to the suburban bus service, I imagine that urban routes pay more of their own way than suruban routes, which means urban bus riders are subsidising suburban bus riders. Doesn’t that remind you of how inner-city taxes back in the day paid to build utilities and roads that sapped them of their vitality? It’s never made sense to me why a downtown should subsidise it’s own demise, but that’s urban planning and politics for you!

Jeremy’s Iron?

Those are all very nice anti-suburban statements. Kudos. I’m sure most suburbanites would likely give the two cities all the bus routes in exchange for a decent highway system, which if we had most of this would be mute.

I’m watching FOX 9 right now, apparently some people are pissed at MnDOT for cutting down trees as part of the new Hwy 212 project. It appears idiotitry has no bounds.

Public transit is something that is not and should not be expected to operate in the black. Yes, costs should be managed responsibly and innovative ways to maximize efficiency should be encouraged, but it’s essentially a subsidized part of the infrastructure that’s intrinsic to the function of a healthy municipality. Cops, sewer and street repair don’t generate a profit, either.

I’m sure most suburbanites would likely give the two cities all the bus routes in exchange for a decent highway system

Tell that to the 4/5 busloads of people on the 77A Express up from Apple Valley everyday at the buttcrack of dawn. I think they might not care for the cut.

By the way, what would a “decent highway system” look like here? 12, 16 lanes? Onramps on both sides of the road? Double-decker toll roads? If you go to Chicago/Phoenix/Atlanta/NYC/Anywhere, you’ll see that those things don’t work either. The more “decent highway” you get, the more fools you’ll get jamming it up again.

I need to un-italicize.

I’m watching FOX 9 right now, apparently some people are pissed at MnDOT for cutting down trees as part of the new Hwy 212 project.

Is that in Canadia? I don’t think I could find Highway 212 with a map, a sextant, and a canoo full of drunken voyageurs.

Take Hwy 62 west – go left. Hwy. 212.

 South Dakota?

What’s with the italics? I didn’t select italics.

Mike, is there some sort of joke I’m not getting? Do you really not know where Hwy 212 is?

The new road through Chaska, Chanhassen and points southeast will be designated 312 I believe.

… which if we had most of this would be mute.

I know you didn’t mean “moot,” Kevin, because a “decent highway system” is not the crux of the issue.

… nor the solution.

And italics & off.

off?

off??

No, we’re stuck with italics.

212 is the yellow brick road to fabulous Von Maur!

I’ve never heard of any of these places.

Italics is bugging me.

Well, that didn’t work.

I tried.

Back from the dead…. your italics mess is fixed.

Whoo-hoo!

So, this will probably be WAAAY more information than most need but not like the topic isn’t on life support already.

Anyway, I wrote an application that does what I think you guys are looking for maybe a year or so ago now as an experiment in ruby. The route information you’re after from the MTC is already made publically available as GIS data, updated quarterly and is available via ftp here (might be down because they are probably updating the new March route changes, it was hit or miss when it was up when I was developing, I have the files from December 2005):

The app is a bus route planner that takes as input 2 spatial GIS coordinates, one temporal coordinate, and returns you a string containing the shortest bus route plan consisting of bus identfiers, spatial location of the initial bus stop, and stop times that will best serve your needs. Might be fun to verify the application algorithm and the datasets accuracy.

I can dust off the prototype and put it online somewhere if you want. The app is queryable via SMS (mobile phone) or http (web).

The bulk of the transit data you’re looking for comes in four (loosely standard) GIS shapefiles: Bus Service, Bus Routes, Bus Route Segments, and Bus Stops.

Here’s a map of the current transit service in the seven county metro using the data..

here’s all the bus stops in the data for the downtown area plotted along with functional class roads.

It would be trivial to add the ability to use United States Geological Survey Places and Names so that you could use strings such as Mall of America, HHH terminal, etc to plan your route instead of geeky GIS coordinates.

I kind of grew bored with it, could get interested in it again if the MTC would make available the GPS data from their buses. I assume they have GPS aboard their buses by now or maybe even cooler!?! a sensor network through which the buses move. Would be a problem of interest to me to provide a real-time plan based on the actual traffic conditions. I’ve been wanting to wrestle with the reference implementation of DOMINO for awhile and never had a real data set to work with. I think someone at google was playing with a similar concpet for awhile by plotting manhattan passenger van companies locations.

I could write an adapter I guess to transform the data into the info Google needs to put it into Google’s system if you like their interface without too much problem if the MTC doesn’t want to do anything. Have to see what restrictions they have put on their data. I probably still have contacts in Google labs.. let me know how your meeting goes s4xton. There is seriously NO good reason wtf portland of all places is supported by google and not minneapolis.

To give you a flavor of the information available here is what attributes the Uber Twin Cities Bus Planner(tm) application slices and dices from the bus route segment shapes:

LINE ATTRIBUTES:

SEGID: A unique identifying number for each segment.

LINE_ID: A number identifying the route number associated with the bus and its schedules. Prefaced with an M for Metro Transit operated, and P for Private Provider operated.

DIRECTION: 0, or 1. 0 is for northbound or eastbound, 1 for southbound or westbound.

FROM_NODE: Timepoint, aka Node, where segment begins.

TO_NODE: Timepoint, aka Node, where segment ends.

MILES: Length of the segment in miles.

WKDY_TRIPS: The total number of trips made by this segment during a weekday.

WKDY_FTRIP: The time of the first trip that segment makes on a weekday, in decimal form. This time is the average between the stop times of the two timepoints on either end of the segment.

WKDY_LTRIP: The time of the last trip that segment makes on a weekday, in decimal form. This time is the average between the stop times of the two timepoints on either end of the segment.

WKDY_RNTME: The average daily scheduled running time by route segment during a weekday.

WKDY_MPH: The average daily miles per hour by route segment during a weekday.

AM_Trips: The total number of trips made by this segment from 7:15 AM to 8:15 AM.

Mid_Trips: The total number of trips made by this segment from 9:00 AM to 2:59 PM.

PM_Trips: The total number of trips made by this segment from 4:15 PM to 5:15 PM.

Ev_Trips: The total number of trips made by this segment from 6:30 PM to 9:59 PM.

SAT_TRIPS: The total number of trips made by this segment during a Saturday.

SAT_FTRIP: The time of the first trip that segment makes on a Saturday, in decimal form.

SAT_LTRIP: The time of the last trip that segment makes on a Saturday, in decimal form.

SAT_RNTME: The average daily scheduled running time by route segment during a Saturday.

SAT_MPH: The average daily miles per hour by route segment during a Saturday.

SUN_TRIPS: The total number of trips made by this segment during a Sunday.

SUN_FTRIP: The time of the first trip that segment makes on a Sunday, in decimal form.

SUN_LTRIP: The time of the last trip that segment makes on a Sunday, in decimal form.

SUN_RNTME: The average daily scheduled running time by route segment during a Sunday.

SUN_MPH: The average daily miles per hour by route segment during a Sunday.

LINE_ID_N: A numeric field identifying the route number associated with the route and its schedules.

hmm lost my map links

Shows up correctly in preview…

Well here are the links to the images anyway…. mnspeak doesn’t appear able to handle img links nested in href’s…

bus stops and bus service

Not2Sure,
You are seriously my hero. I had no idea the routes were released as GIS data! I’ve never seen bus map layers before, only train lines in other cities. And the addition of the actual stop locations is marvelous. It seems like with the attribute data a rough estimation of at least bus frequency could be made without knowing the true schedule of the route!

It looks like you’ve already done a lot of good work getting the data between gis and a more usable format. This could really prove to be something awesome. I know there’s a few places out there that turn lat/lon into Gpoints data for integration so I doubt it would be much of an issue to shuttle data from GIS->gmaps with a little tweaking.

Oh, and I know this post is terribly disorganised, but I also heard in my correspondence that they’re planning to implement a ‘real bus arrival time’ option for the trip planner and possibly phone schedule service, which means they certainly do track the gps units on the buses. I doubt they’ll make the backend data available, though ):

But seriously, great work! I might have to dust off the old ArcGIS and download those layers when I get home from work.

You’re very welcome. The app was (is) pretty functional. It replicated the full schedule of a route for a week to compare with a paper schedule from MTC as a test case to verify and validate the GIS data. Didn’t really validate much of it, given the source of the data was the same.

The motivation for doing the work was to provide the ability to query via SMS capable phones and pagers. I wanted to find a way for recent immigrants / non-native English speakers to get good routes quickly without having to call MTC and stumble through an English conversation.