{"id":420,"date":"2014-01-06T21:31:57","date_gmt":"2014-01-07T02:31:57","guid":{"rendered":"http:\/\/homepages.uc.edu\/~yaozo\/wordpress\/?p=420"},"modified":"2014-01-06T21:31:57","modified_gmt":"2014-01-07T02:31:57","slug":"using-contour-lines-to-map-density-of-nypds-stop-and-frisk-data","status":"publish","type":"post","link":"https:\/\/zhuoyao.net\/index.php\/2014\/01\/06\/using-contour-lines-to-map-density-of-nypds-stop-and-frisk-data\/","title":{"rendered":"Using Contour Lines to Map Density of NYPD\u2019s Stop and Frisk Data"},"content":{"rendered":"<h2>Using Contour Lines to Map Density of NYPD\u2019s Stop and Frisk Data<\/h2>\n<div>\n<div>July 13 2012\u00a0by\u00a0<a href=\"https:\/\/www.mapbox.com\/about\/team\/#chris-herwig\">\u00a0Chris Herwig<\/a><\/div>\n<\/div>\n<div>\n<p>Using elevation contour lines is one of several ways to visualize density from vector point data. I recently used this approach to map \u201cstop and frisks\u201d &#8211; the practice of a police officer stopping an individual for reasonable suspicion of criminal activity &#8211; in New York City. This practice is often criticized as happening more often in neighborhoods with higher numbers of minority residents, so showing the density of such stops is critical to the story behind this data. If you would like to know more about Stop and Frisk, check out the research conducted by the\u00a0<a href=\"http:\/\/ccrjustice.org\/stop-and-frisk-does-not-reduce-crime\">Center for Constitutional Rights<\/a>and the\u00a0<a href=\"http:\/\/www.nyclu.org\/stopandfrisk\">New York Civil Liberties Union<\/a>.<\/p>\n<p>In this post I\u2019ll discuss how I visualized the point density of stop and frisk occurrences using the<a href=\"http:\/\/www.nyc.gov\/html\/nypd\/html\/analysis_and_planning\/stop_question_and_frisk_report.shtml\">NYPD\u2019s Stop and Frisk databases<\/a>. To get an idea of some of the others challenges faced and approaches in mapping this data, check out the\u00a0<a href=\"http:\/\/wiki.datawithoutborders.cc\/index.php?title=Project:Current_events:NYC_DD:NYCLU\">Data Without Borders Data Dive NYC wiki page<\/a>\u00a0or my\u00a0<a href=\"https:\/\/github.com\/hrwgc\/precinct\">precinct<\/a>\u00a0github repo.<\/p>\n<p><a name=\"From.points.to.heat.map.to.contour.lines\"><\/a><\/p>\n<h1>From points to heat map to contour lines<\/h1>\n<p>From the larger Stop and Frisk database, which contains more than 4.2 million records, I selected only stops that occurred in 2008 where the person stopped was either \u201cBlack\u201d, \u201cLatino,\u201d or \u201cBlack Latino\u201d, and which contained coordinates for the stop.<\/p>\n<pre><code> ogr2ogr -f SQLite \/path\/to\/outfile\/sf-race-geo.sqlite PG:\"dbname='' host='localhost' port='5432'\" -sql \"SELECT distinct ogc_fid, lat_lon, year, race, pct FROM public.\"new_all\" where year = '2008' AND xcoord &lt;&gt; '' AND (race = 'B' OR race = 'Q' OR race = 'P')\"\n<\/code><\/pre>\n<p>This resulted in 432,515 records, which are\u00a0<a href=\"https:\/\/github.com\/hrwgc\/precinct\/blob\/gh-pages\/data\/shp\/contours\/example\/sf-race-geo.sqlite\">available as an sqlite file<\/a>.<\/p>\n<p>For some uses, you can visualize density using only the point data and manipulating the opacity and size of markers. I used this method to create\u00a0<a href=\"http:\/\/a.tiles.mapbox.com\/v3\/herwig.map-4qy8mic8.html#12.00\/40.6879\/-73.8673\">this map, which shows all of the stops from 2008<\/a>.<\/p>\n<p><iframe loading=\"lazy\" src=\"https:\/\/a.tiles.mapbox.com\/v3\/herwig.map-9lv4zsb6.html?secure=1#12\/40.8\/-73.912\" height=\"400\" width=\"640\" frameborder=\"0\"><\/iframe>However, in this case I wanted to really draw attention to the neighborhoods where stop and frisks happened most frequently. To better achieve this, I created elevation contours based off of vector points to show the density of the stops. Here\u2019s how I did that.<\/p>\n<p><a name=\"Step.1\"><\/a><\/p>\n<h2>Step 1<\/h2>\n<ul>\n<li>Open\u00a0<a href=\"http:\/\/qgis.org\/\">QGIS<\/a><\/li>\n<li>Load\u00a0<a href=\"https:\/\/github.com\/hrwgc\/precinct\/blob\/gh-pages\/data\/shp\/contours\/example\/sf-race-geo.sqlite\">sqlite file<\/a>\u00a0as a vector layer<\/li>\n<\/ul>\n<p><img decoding=\"async\" alt=\"Loaded SQLite file\" src=\"https:\/\/farm8.staticflickr.com\/7279\/7563182582_38d412cf2e_z.jpg\" \/><\/p>\n<p><a name=\"Step.2\"><\/a><\/p>\n<h2>Step 2<\/h2>\n<ul>\n<li>Using QGIS&#8217; heat map plugin located under the Raster Menu, create a raster heat map from the input point vector\u00a0blstops2008<\/li>\n<li>I used a buffer radius of\u00a010\u00a0and a decay ratio of\u00a0.5.<\/li>\n<\/ul>\n<p><img decoding=\"async\" alt=\"Heatmap plugin with 10 buffer, .5 decay\" src=\"https:\/\/farm8.staticflickr.com\/7134\/7563182314_64d10d8215.jpg\" \/><\/p>\n<p><a name=\"Step.3\"><\/a><\/p>\n<h2>Step 3<\/h2>\n<ul>\n<li>Load the heat map raster layer in QGIS. It will be a .tif file if you chose GeoTIFF in the step above.<\/li>\n<li>Verify the heat map worked by opening up the layer properties for your .tif file in QGIS.<\/li>\n<li>Set color map to\u00a0Pseudocolor<\/li>\n<\/ul>\n<p><img decoding=\"async\" alt=\"Select &quot;Pseudocolor&quot; under the color map option\" src=\"https:\/\/farm9.staticflickr.com\/8025\/7563181466_fa71a2f9c4_z.jpg\" \/><\/p>\n<p>This is what mine looked like. Success!<\/p>\n<p><img decoding=\"async\" alt=\"Heat map showing high frequency of points\" src=\"https:\/\/farm9.staticflickr.com\/8161\/7563181280_b8a54906d8_z.jpg\" \/><\/p>\n<p><a name=\"Step.4\"><\/a><\/p>\n<h2>Step 4<\/h2>\n<p>Next we convert the raster heat map into vector contours using the\u00a0gdal_contour\u00a0option available in QGIS under the raster \/ extraction menu.<\/p>\n<p>I chose to use an interval value of\u00a0150\u00a0and selected to include an\u00a0ELEV\u00a0attribute column. This is the column we\u2019ll be basing our Carto styling in TileMill.<\/p>\n<p><img decoding=\"async\" alt=\"Interval 150, Attribute name = ELEV\" src=\"https:\/\/farm9.staticflickr.com\/8428\/7563182398_3009354561_z.jpg\" \/><\/p>\n<p>You can now run the contour from within QGIS, or you can copy the script at the bottom of the prompt and execute it on the command line.<\/p>\n<pre><code> gdal_contour -a ELEV -i 150.0 \/path\/to\/input\/blstops2008.tif \/path\/to\/output\/bl-contours-2008.shp\n<\/code><\/pre>\n<p><img decoding=\"async\" alt=\"select categorized, &quot;ELEV&quot; column\" src=\"https:\/\/farm8.staticflickr.com\/7114\/7563181390_74b7536364_z.jpg\" \/><\/p>\n<p>Preview the contours in QGIS, selecting\u00a0categorized\u00a0styling based on the\u00a0ELEV\u00a0column.<\/p>\n<p><img decoding=\"async\" alt=\"styling based on categorized elev column\" src=\"https:\/\/farm9.staticflickr.com\/8012\/7563181614_492f207b50_z.jpg\" \/><\/p>\n<p><a name=\"Step.5\"><\/a><\/p>\n<h2>Step 5<\/h2>\n<p>Now we have everything we need to style the vector contours in\u00a0<a href=\"http:\/\/mapbox.com\/tilemill\">TileMill<\/a>.<\/p>\n<p><img decoding=\"async\" alt=\"Import\" src=\"https:\/\/farm9.staticflickr.com\/8007\/7563182226_1ea006e0fc_z.jpg\" \/><\/p>\n<ul>\n<li>Import the shapefile into TileMill.<\/li>\n<li>Style the layer based on the\u00a0ELEV\u00a0column. I chose to do this using the\u00a0spin(color, spin distance)\u00a0Carto feature to save myself some key strokes. Here\u2019s the\u00a0<a href=\"https:\/\/gist.github.com\/1b508cc2a77ec06bc9da\">full stylesheet.<\/a><\/li>\n<\/ul>\n<p><img decoding=\"async\" alt=\"Style and preview with black background\" src=\"https:\/\/farm8.staticflickr.com\/7272\/7563181820_490c4e0cff_z.jpg\" \/><\/p>\n<p>I added a black background for previewing the map but removed it before exporting. Now the contours are a transparent\u00a0layer\u00a0that we can composite with other layers.<\/p>\n<p><a name=\"Step.6\"><\/a><\/p>\n<h2>Step 6<\/h2>\n<p>Export to MBTiles and upload to MapBox. If you don\u2019t have an account yet,\u00a0<a href=\"http:\/\/mapbox.com\/plans\/\">sign up for one<\/a>.<\/p>\n<p><a href=\"http:\/\/a.tiles.mapbox.com\/v3\/herwig.stop-and-frisk-bl-contours-2008.html\">Contour Lines Layer<\/a><\/p>\n<p><a name=\"Step.7\"><\/a><\/p>\n<h2>Step 7<\/h2>\n<p>Use the map layer as a layer itself, or as part of a mash up. Here are the contour lines \u2013 which show concentrations of stops of Black, Latino, and Black Latino persons in 2008 \u2013 on top of all of the points from all stops in 2008. To add geographical context, I used a\u00a0<a href=\"http:\/\/mapbox.com\/blog\/custom-styles-mapbox-streets\/\">MapBox Streets<\/a>\u00a0base layer.<\/p>\n<p><img decoding=\"async\" alt=\"MapBox Streets\" src=\"https:\/\/farm9.staticflickr.com\/8141\/7563181078_d442faeafc_z.jpg\" \/><\/p>\n<p><a name=\"Final.density.map.using.contour.lines\"><\/a><\/p>\n<h1>Final density map using contour lines<\/h1>\n<p>Here\u2019s our final map.<\/p>\n<p><iframe loading=\"lazy\" src=\"https:\/\/a.tiles.mapbox.com\/v3\/herwig.map-olatzdh8,herwig.all-stops-2008.html?secure=1#13\/40.804\/-73.951\" height=\"400\" width=\"640\" frameborder=\"0\"><\/iframe><a href=\"http:\/\/a.tiles.mapbox.com\/v3\/herwig.map-olatzdh8,herwig.all-stops-2008.html#13.00\/40.6993\/-73.9550\">Black, Latino, and Black Latino Stops on top of All Stops, 2008<\/a><\/p>\n<p><a name=\"Data.mentioned.in.this.blog.post\"><\/a><\/p>\n<h3>Data mentioned in this blog post<\/h3>\n<ol>\n<li><a href=\"http:\/\/www.nyclu.org\/stopandfrisk\">Stop and Frisk Databases<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/hrwgc\/precinct\/blob\/gh-pages\/data\/shp\/contours\/example\/sf-race-geo.sqlite\">SQLite file containing the points from 2008<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/hrwgc\/precinct\/blob\/gh-pages\/data\/shp\/contours\/example\/blstops2008.tif\">Step 2 result: Heatmap raster file<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/hrwgc\/precinct\/blob\/gh-pages\/data\/shp\/contours\/example\/bl-contours-2008.shp\">Step 4 result: Contours Shapefile<\/a><\/li>\n<li><a href=\"https:\/\/gist.github.com\/1b508cc2a77ec06bc9da\">Carto stylesheet for contours<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/hrwgc\/precinct\/blob\/gh-pages\/data\/shp\/contours\/example\/sf-contours-example\/\">TileMill project<\/a><\/li>\n<\/ol>\n<\/div>\n","protected":false},"excerpt":{"rendered":"<p>Using Contour Lines to Map Density of NYPD\u2019s Stop and Frisk Data July 13 2012\u00a0by\u00a0\u00a0Chris Herwig Using elevation contour lines is one of several ways&hellip; <\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[2],"tags":[],"class_list":["post-420","post","type-post","status-publish","format-standard","hentry","category-arcgis"],"_links":{"self":[{"href":"https:\/\/zhuoyao.net\/index.php\/wp-json\/wp\/v2\/posts\/420","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/zhuoyao.net\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/zhuoyao.net\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/zhuoyao.net\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/zhuoyao.net\/index.php\/wp-json\/wp\/v2\/comments?post=420"}],"version-history":[{"count":0,"href":"https:\/\/zhuoyao.net\/index.php\/wp-json\/wp\/v2\/posts\/420\/revisions"}],"wp:attachment":[{"href":"https:\/\/zhuoyao.net\/index.php\/wp-json\/wp\/v2\/media?parent=420"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/zhuoyao.net\/index.php\/wp-json\/wp\/v2\/categories?post=420"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/zhuoyao.net\/index.php\/wp-json\/wp\/v2\/tags?post=420"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}