A quick update to let people know that I made an online map of Cleveland’s Building ages

I’ve wanted to make one for a couple years now but didn’t have access to the data. I finally do now and uploaded the raw data on github . Once you click on that link, click raw, and save the file as a CSV.

The Case for Open Data:

I’ve wanted to make this map for years… Cleveland and Cuyahoga County’s data access is less than stellar and falling behind other cities. Open Data policies foster the culture where information like building construction dates is shared and readily accessible and updated on the internet for government employees, non-profits, private businesses, and anyone else.

In Cleveland, there’s growing awareness open data’s value to communities Open Cleveland , Code For America Brigade, is among several groups and individuals including Cleveland Public Library , Center on Urban Poverty and Community Development at Case Western Reserve University and One Community to promote open data policies and demonstrate its value.

If you’re interested to participate in this, you’re welcome to become a part of Open Cleveland and attend one of our meetups and events.

Some technical/cartography details:

It’s only the 2nd choropleth map (I usually make base maps or transportation maps) that I’ve ever made and learned more about that area of cartography. I tried out a couple different data classification methods (jenks, Equal Interval, and others) that are built-in to cartodb, the mapping software library that I used. None of them really felt ‘right’ to me so I made my own breaks after examining the source data and looking for trends.

I only used 6 breaks. Is it too many or too few? I am not experienced in this area of cartography and found advice for sticking with 5-7 (on the belief too many would confuse the reader). I would have liked to see how a scheme where each increase in year change the color ever so slightly, perhaps increasing the Hue value by 1 or 2 points for each year. I couldn’t figure how to do this (and I don’t know the proper cartographical term either) in cartodb, so I opted to look at cartodb’s data classifications…

Why cartodb:

I actually originally tried mapbox’s mapbox-studio as I was more experienced with their softwares and thought it would be a good tool for the job (and I was really itching to use mapbox-studio again - Yes, I know the latter point is not a good reason to base your decision on what tool(s) to use to make a map).. But, mapbox-studio was unable to create the vector layer for me; it give me an error that I couldn’t have X (I think it was 50k) amount of points at a particular zoom level. So, I went to cartodb. It handled the 40mb geojson file that I uploaded without a problem.

Why points:

There’s been a few building age maps similar to this one (web maps, detailed to the block level) for NYC, Austin, PDX, Chicago. All of them had used the actual building footprint. Unfortunately, the source data only gives point centroid of the land parcel for the geography (lat/lon). I briefly thought about trying to match them up with the publicly available building footprints were last updated in 2007 (!) provided by the County (they’re currently updating them now) but they are woefully incomplete in coverage. I also thought about matching them up with the building footprints that the Cleveland Metroparks made (they are the forefront of open source GIS in Northeast Ohio) from ~2012. That data set is relatively complete in terms of coverage. However, I’m doing this in my limited free time and didn’t want to go down that rabbit hole, so I decided to just go with the points.

Notice errors or omissions in the source data? I have too and I need to figure out who to contact and send them the corrections. This data was missing buildings built in 2013 and 2014.

Before I went to visit Carroll County in North Central Ohio for a weekend in July, its state in OSM was really poor. It consisted mostly of TIGER-imported ways that hadn’t been whose geometry hadn’t been changed since the original TIGER Import in 2008. In many cases, the roads’ geometry didn’t make up with reality. Some roads were 30 meters off of where they really are; small townships of 10-20 streets on the map were like jigsaw puzzle pieces that needed to be rotated; random roads were in the middle of grassy fields. One exception to it was Carrollton, the largest city in the county, which was improved very well OSM user Evan Edwards.

After that weekend, I corrected some road geometries and added a dozen or so POIs. and wondered how long it will take me to fix all of the roads in the county and what would it reveal about editing TIGER data.

Answer: a lot longer than I thought! (~30 hours over a couple months). Near the end, it honestly became a chore. But I want to see how it would like visually, I didn’t want to do that work for nothing…

How I did it (very briefly): I was able to more quickly improve the ways by using where I’d copy the geometry from a TIGER2013 file; verify that the TIGER13 matches the aerial imagery better than the existing way.

This a great strategy and I’d advise anyone who is interested in improving TIGER-imported ways in the future.

Watch a quick example of this workflow in the GIF.

If you’re interested in a detailed workflow, skip to the end…

Visualization:

Here’s a first draft of the map;

• http://skorasaurus.github.io/carroll.html

Now, I’m trying to visualize the improvements that I made by displaying the distance between 2 linestrings (before I edited with OSM data from spring 2014 - which a postgis table named old_line, and OSM data from Nov. 2014 which is a postgis table named new_line))

How I’m planning to do this.

A] select each point in the every linestring from old_line as oldpointdump

SELECT ST_DumpPoints(way) FROM old_line as oldpointdump

B] FOR each point in oldpointdump, find the closet linestring in new_line; measure this distance C] create a new column for this distance (as distfromtiger)

F] then in every line string in old_table between 2 points (let’s call these points J and K), add a column and assign that column a value that is (.5 * (distfromtiger for J) + (distfromtiger of K))

G] style the linestring in mapbox/cartodb using the value that I derived in F

==============================

A much more thorough workflow follows:

I used ogr2osm to convert 2013 TIGER Shapefile (then tiger2014 once they became available) into an OSM file and then loaded that file in josm and deleted all of the unnecessary tags that were created in it, except the fullname field; to ensure that the name was matching up with the correct road, and saved on my computer for my workflow.

Then, I went to editing. I downloaded some OSM data from JOSM, opened the TIGER osm file, and checked out a particular area with bing imagery. For a particular street, go over the entire street to see if the 2013 road is in fact closer to matching the aerial bing imagery than the existing road is. I’d also load up the history for the road to see if any nodes were changed. (Cntrl+H). Many roads were changed to proper highway type (tertiary) but didn’t have any of the geometry modified since the original tiger import. To more easily find roads that were not yet modified I would do: user:bot-mode OR user:DaveHansenTiger AND type:way AND -modified

If the 2013 is more closely aligned than the existing road, delete the name tag in the TIGER 2013 way you’re about to select; copy (cntrl+c the highway as it is in a separate layer with the separate layer selected in josm); switch to your osm data layer, and then paste into your data layer of OSM data. Then select the two ways, the new way and the way you’re replacing; and hit cntrl+shift+g. CNTRL+shift+g is the ‘replace geometry’ shortcut that keeps the history of the original way and relations that the way belongs to.

I looked over each way that I imported. In some cases the imported way was about 5-10 meters from the center of the road in the bing imagery but still better than before and within the general margin of error that we have in OSM.

After you finish all of the ways in your area; hit the JOSM validation, first fix all of the errors marked highway duplicate nodes; then hit the josm validation button again; and go to the ways ended near other nodes, right click on each one in the list which brings me to the location of the node, hit ‘N’ for each. At this point, I’d go back to the ways I modified, and simplify them (cntrl+y) because TIGER with deletes any excess nodes without deleting the nature of the geometry. I set my Simplify way settings (done in advanced preferences in josm, and finding the ) to 0.8 . JOSM’s simplification uses the Ramer–Douglas–Peucker algorithm.

* Get it from [github](https://github.com/scotte/jekyll-clean). * See the [live demo](https://scotte.github.io/jekyll-clean). <<<<<<< HEAD * See it [in action on my own blog](https://scotte.github.io). ======= * See it [in action on my own blog](https://scotte.org). >>>>>>> upstream/gh-pages Welcome to the sample post for the Jekyll Clean theme. A simple and clean Jekyll theme using [bootstrap](http://getbootstrap.com) (not to be confused with jekyll-bootstrap) that's easy to modify and very modular in component and element reuse. It uses Disqus for comments and includes Google Analytics support. Both of these features are disabled by default and can be enabled via \_config.yml. You can also rip this code out of the templates if you like (footer.html and post.html). The beauty of Jekyll - keep things clean... Jekyll Clean! The theme works well on mobile phones, using a collapsable nav bar and hiding the sidebar. The links pane in the sidebar is available on mobile through the nav menu, and you can do the same thing for any other sections added to the sidebar. Don't forget to occassionally merge against my upstream repository so you can get the latest changes. Pull requests are encouraged and accepted! Installation ============ If you don't have a blog already on github, start by cloning this repository. Best to do that directly on github and then clone that down to your computer. If you already do have a blog, You can certainly apply this theme to your existing blog in place, but then you won't be able to merge as the theme changes. If you re-apply your blog history on top of this theme's **gh-pages** branch, it's then easy to update to the latest version of the theme. You also don't want to have to deal with resolving old conflicts from your existing history, so you may wish to to push your existing master off to a new branch so you have the old history and start a new branch with this as the start, merging in your \_posts and other assets (after git rm'ing the current \_posts. Not ideal, but you have to make a choice - either apply it manually or base your blog off this theme's branch. Either way it will work, and both have their own pros and cons. You can setup an upstream tracking repository like so: ``` $ git remote add upstream git@github.com:scotte/jekyll-clean.git ``` And now when you wish to merge your own branch onto the latest version of the theme, simply do: ``` $ git fetch upstream $ git merge upstream/gh-pages ``` Of course you will have to resolve conflicts for \_config.yml, \_includes/links-list.html, and \_posts, and so on, but in practice this is pretty simple. This is how I maintain my own blog which is based on this theme. The old history is sitting in an **old-master** branch that I can refer to when I need to. <<<<<<< HEAD ======= Running Locally =============== Here's the exact set of packages I need to install on Debian to run jekyll locally with this theme for testing. ``` $ sudo aptitude install ruby ruby-dev rubygems nodejs $ sudo gem install jekyll jekyll-paginate ``` And then it's just a simple matter of running jekyll locally: ``` $ jekyll serve --baseurl='' ``` Now browse to http://127.0.0.1:4000 Using gh-pages ============== Running a jekyll site is a bit outside the scope of this doc, but sometimes it can be a bit confusing how to configure jekyll for project pages versus user pages, for example. To start with, read through [the documentation here](https://help.github.com/articles/user-organization-and-project-pages/). This will provide a good overview on how it all works. The git branch and baseurl (in _config.yml) will change depending on the sort of site deployed. When you clone this repository, it's set up for project pages, so the deployed branch is "gh-pages" and baseurl is configured to 'jekyll-clean', because that's the name of this project. If you plan to deploy this as user pages, the deployed branch is "master" and baseurl is configured to '' (i.e. empty). Comment Systems =============== Jekyll clean supports both [isso](https://posativ.org/isso) and [disqus](https://disqus.com) comment systems. After enabling **comments**, either **isso** or **disquss** must be configured. Don't try configuring both! Isso Comments ============= Isso requires running a local server, so is not suitable for hosting in github pages, for example. Isso is open source and keeps all your data local, unlike Disqus (who knows exactly what they are doing with your data). In _config.yml you'll need to set **isso** to the fully-qualified URL if your isso server (this is the value for **data-isso** passed to the isso JS). Make sure **comments** is true. Disqus Comments =============== Getting Disqus to work can be a bit more work than it seems like it should be. Make sure your Disqus account is correctly configured with the right domain of your blog and you know your Disqus shortname. In _config.yml you'll need to set **disqus** to your Disqus shortname and make sure **comments** is true. Finally, in posts, make sure you have **comments: true** in the YAML front matter. More information on using Disqus with Jekyll is [documented here](https://help.disqus.com/customer/portal/articles/472138-jekyll-installation-instructions). Code Syntax Highlighting ======================== To use code syntax highlighting, use the following syntax: ``` ```python import random # Roll the die roll = random.randint(1, 20) print('You rolled a %d.' % roll) ``` #REMOVE ``` (Remove #REMOVE from the end of the last line). Which will look like this in the rendered jekyll output using the default css/syntax.css provided with this theme (which is the **colorful** theme from [https://github.com/iwootten/jekyll-syntax](https://github.com/iwootten/jekyll-syntax)): ```python import random # Roll the die roll = random.randint(1, 20) print('You rolled a %d.' % roll) ``` You can, of course, use any theme you wish, see the jekyll and pygments documentation for more details. >>>>>>> upstream/gh-pages License ======= The content of this theme is distributed and licensed under a  [Creative Commons Attribution 4.0 License](https://creativecommons.org/licenses/by/4.0/legalcode) This license lets others distribute, remix, tweak, and build upon your work, even commercially, as long as they credit you for the original creation. This is the most accommodating of licenses offered. Recommended for maximum dissemination and use of licensed materials. In other words: you can do anything you want with this theme on any site, just please provide a link to [the original theme on github](https://github.com/scotte/jekyll-clean) so I get credit for the original design. Beyond that, have at it! This theme includes the following files which are the properties of their respective owners: * js/bootstrap.min.js - [bootstrap](http://getbootstrap.com) * css/bootstrap.min.css - [bootstrap](http://getbootstrap.com) * js/jquery.min.js - [jquery](https://jquery.com) * images/cc_by_88x31.png - [creative commons](https://creativecommons.org) <<<<<<< HEAD ======= * css/colorful.css - [iwootten/jekyll-syntax](https://github.com/iwootten/jekyll-syntax) >>>>>>> upstream/gh-pages