profile
viewpoint
Matt Gardner allthesignals @NYCPlanning United States https://planninglabs.nyc/ Developer for NYC Planning Labs

allthesignals/avocado 1

"M(att)VT" Tiling, HTTP SQL API, and ETL services, Docker(composer)ized and modular

allthesignals/BostonGreenMediaHarvest 1

I'm harvesting open photographs from social media to support the Boston Green Space project

allthesignals/cdb-sync-manager 1

GUI for managing synchronizations between Carto and Postgres

allthesignals/active_model_serializers 0

ActiveModel::Serializer implementation and Rails hooks

allthesignals/airbnb 0

Data collection for Airbnb business

allthesignals/airbnboston 0

An analysis of Airbnb listings in Boston. **WIP**

allthesignals/am-i-rent-stabilized 0

A web app to inform NYC residents about rent stabilization

allthesignals/argument 0

Decorators for Component and Object arguments in Ember

allthesignals/boston-green-ember-cartodb 0

Attempt to recreate Boston Green as an Ember app built on CartoDB

startedkdy1/cargo-profile

started time in 3 hours

pull request commentNYCPlanning/db-facilities

Rename to dcp_colp and update field names

`(CASE

		-- Admin of Gov

		WHEN usetype LIKE '%AGREEMENT%'

			OR usetype LIKE '%DISPOSITION%'

			OR usetype LIKE '%COMMITMENT%'

			OR excatdesc LIKE '%PRIVATE%'

			THEN 'Properties Leased or Licensed to Non-public Entities'

		WHEN usetype LIKE '%SECURITY%' THEN 'Miscellaneous Use'

		WHEN usetype LIKE '%PARKING%'

			AND usetype NOT LIKE '%MUNICIPAL%'

			THEN 'City Agency Parking'

		WHEN usetype LIKE '%STORAGE%' THEN 'Storage'

		WHEN usetype LIKE '%CUSTODIAL%' THEN 'Custodial'

		WHEN usetype LIKE '%GARAGE%' THEN 'Maintenance and Garages'

		WHEN usetype LIKE '%OFFICE%' THEN 'City Government Offices'

		WHEN usetype LIKE '%MAINTENANCE%' THEN 'Maintenance and Garages'

		WHEN usetype LIKE '%NO USE%' THEN 'Miscellaneous Use'

-- WHEN usetype LIKE '%MISCELLANEOUS USE%' THEN 'Miscellaneous Use'

		WHEN usetype LIKE '%OTHER HEALTH%' AND parcelname LIKE '%ANIMAL%' THEN 'Miscellaneous Use'

		WHEN agency LIKE '%DCA%' and usetype LIKE '%OTHER%' THEN 'Miscellaneous Use'

		WHEN usetype LIKE '%UNDEVELOPED%' THEN 'Miscellaneous Use'

		WHEN (usetype LIKE '%TRAINING%' 

			OR usetype LIKE '%TESTING%')

			AND usetype NOT LIKE '%LABORATORY%'

			THEN 'Training and Testing'



		-- Trans and Infra

		WHEN usetype LIKE '%MUNICIPAL PARKING%' THEN 'Parking Lots and Garages'

		WHEN usetype LIKE '%MARKET%' THEN 'Wholesale Markets'

		WHEN usetype LIKE '%MATERIAL PROCESSING%' THEN 'Material Supplies'

		WHEN usetype LIKE '%ASPHALT%' THEN 'Material Supplies'

		WHEN usetype LIKE '%AIRPORT%' THEN 'Airports and Heliports'

		WHEN usetype LIKE '%ROAD/HIGHWAY%'

			OR usetype LIKE '%TRANSIT WAY%'

			OR usetype LIKE '%OTHER TRANSPORTATION%'

			THEN 'Other Transportation'

		WHEN agency LIKE '%DEP%'

			AND (usetype LIKE '%WATER SUPPLY%'

			OR usetype LIKE '%RESERVOIR%'

			OR usetype LIKE '%AQUEDUCT%')

			THEN 'Water Supply'

		WHEN agency LIKE '%DEP%'

			AND usetype NOT LIKE '%NATURE AREA%'

			AND usetype NOT LIKE '%NATURAL AREA%'

			AND usetype NOT LIKE '%OPEN SPACE%'

			THEN 'Wastewater and Pollution Control'

		WHEN usetype LIKE '%WASTEWATER%' THEN 'Wastewater and Pollution Control'

		WHEN usetype LIKE '%LANDFILL%' 

			OR usetype LIKE '%SOLID WASTE INCINERATOR%'

			THEN 'Solid Waste Processing'

		WHEN usetype LIKE '%SOLID WASTE TRANSFER%'

			OR (agency LIKE '%SANIT%' AND usetype LIKE '%SANITATION SECTION%')

			THEN 'Solid Waste Transfer and Carting'

		WHEN usetype LIKE '%ANTENNA%' OR usetype LIKE '%TELE/COMP%' THEN 'Telecommunications'

		WHEN usetype LIKE '%PIER - MARITIME%'

			OR usetype LIKE '%FERRY%' 

			OR usetype LIKE '%WATERFRONT TRANSPORTATION%'

			OR usetype LIKE '%MARINA%'

			THEN 'Ports and Ferry Landings'

		WHEN usetype LIKE '%RAIL%'

			OR (usetype LIKE '%TRANSIT%'

				AND usetype NOT LIKE '%TRANSITIONAL%')

			THEN 'Rail Yards and Maintenance'

		WHEN usetype LIKE '%BUS%' THEN 'Bus Depots and Terminals'



		-- Health and Human

		WHEN agency LIKE '%HHC%' THEN 'Hospitals and Clinics'

		WHEN usetype LIKE '%HOSPITAL%' THEN 'Hospitals and Clinics'

		WHEN usetype LIKE '%AMBULATORY HEALTH%' THEN 'Hospitals and Clinics'

		WHEN agency LIKE '%OCME%' THEN 'Other Health Care'

		WHEN agency LIKE '%ACS%' AND usetype LIKE '%HOUSING%' THEN 'Shelters and Transitional Housing'

		WHEN agency LIKE '%AGING%' THEN 'Senior Services'

		WHEN (agency LIKE '%DHS%' OR agency LIKE '%HRA%')

			AND (usetype LIKE '%RESIDENTIAL%'

			OR usetype LIKE '%TRANSITIONAL HOUSING%')

			THEN 'Shelters and Transitional Housing'

		WHEN agency LIKE '%DHS%' AND usetype NOT LIKE '%OPEN SPACE%' THEN 'Non-residential Housing and Homeless Services'

		WHEN (agency LIKE '%NYCHA%' 

			OR agency LIKE '%HPD%')

			AND usetype LIKE '%RESIDENTIAL%'

			THEN 'Public or Affordable Housing'

		WHEN usetype LIKE '%COMMUNITY CENTER%' OR (agency LIKE '%HRA%' AND parcelname LIKE '%CENTER%') 

			THEN 'Community Centers and Community School Programs'



		-- Parks, Cultural

		WHEN usetype LIKE '%LIBRARY%' THEN 'Public Libraries'

		WHEN usetype LIKE '%MUSEUM%' THEN 'Museums'

		WHEN usetype LIKE '%CULTURAL%' THEN 'Other Cultural Institutions'

		WHEN usetype LIKE '%ZOO%' THEN 'Other Cultural Institutions'

		WHEN usetype LIKE '%CEMETERY%' THEN 'Cemeteries'

		WHEN agency LIKE '%CULT%' AND usetype LIKE '%MUSEUM%' THEN 'Museums'

		WHEN agency LIKE '%CULT%' THEN 'Other Cultural Institutions'

		WHEN usetype LIKE '%NATURAL AREA%'

			OR (usetype LIKE '%OPEN SPACE%'

				AND agency LIKE '%DEP%')

			THEN 'Preserves and Conservation Areas'

		WHEN usetype LIKE '%BOTANICAL GARDENS%' THEN 'Other Cultural Institutions'

		WHEN usetype LIKE '%GARDEN%' THEN 'Gardens'

		WHEN agency LIKE '%PARKS%'

			AND usetype LIKE '%OPEN SPACE%'

			THEN 'Streetscapes, Plazas, and Malls'

		WHEN usetype = 'MALL/TRIANGLE/HIGHWAY STRIP/PARK STRIP'

			THEN 'Streetscapes, Plazas, and Malls'

		WHEN usetype LIKE '%PARK%' THEN 'Parks'

		WHEN usetype LIKE '%PLAZA%'

			OR usetype LIKE '%SITTING AREA%' 

			THEN 'Streetscapes, Plazas, and Malls'

		WHEN usetype LIKE '%PLAYGROUND%'

			OR usetype LIKE '%SPORTS%'

			OR usetype LIKE '%TENNIS COURT%'

			OR usetype LIKE '%PLAY AREA%'

			OR usetype LIKE '%RECREATION%'

			OR usetype LIKE '%BEACH%'

			OR usetype LIKE '%PLAYING FIELD%'

			OR usetype LIKE '%GOLF COURSE%'

			OR usetype LIKE '%POOL%'

			OR usetype LIKE '%STADIUM%'

			THEN 'Recreation and Waterfront Sites'

		WHEN usetype LIKE '%THEATER%' AND agency LIKE '%DSBS%'

			THEN 'Other Cultural Institutions'



		-- Public Safety, Justice etc

		WHEN agency LIKE '%ACS%' AND usetype LIKE '%DETENTION%' THEN 'Detention and Correctional'

		WHEN agency LIKE '%CORR%' AND usetype LIKE '%COURT%' THEN 'Courthouses and Judicial'
		
		WHEN agency LIKE '%COURT%' AND usetype LIKE '%COURT%' THEN 'Courthouses and Judicial'

		WHEN agency LIKE '%OCA%' AND usetype LIKE '%COURT%' THEN 'Courthouses and Judicial'

		WHEN agency LIKE '%CORR%' THEN 'Detention and Correctional'

		WHEN usetype LIKE '%AMBULANCE%' THEN 'Other Emergency Services'

		WHEN usetype LIKE '%EMERGENCY MEDICAL%' THEN 'Other Emergency Services'

		WHEN usetype LIKE '%FIREHOUSE%' THEN 'Fire Services'

		WHEN usetype LIKE '%POLICE STATION%' THEN 'Police Services'

		WHEN usetype LIKE '%PUBLIC SAFETY%' THEN 'Other Public Safety'

		WHEN agency LIKE '%OCME%' THEN 'Forensics'



		-- Education, Children, Youth

		WHEN usetype LIKE '%UNIVERSITY%' THEN 'Colleges or Universities'

		WHEN usetype LIKE '%EARLY CHILDHOOD%' THEN 'Day Care'

		WHEN usetype LIKE '%DAY CARE%' THEN 'Day Care'

		WHEN agency LIKE '%ACS%' AND usetype LIKE '%RESIDENTIAL%' THEN 'Foster Care Services and Residential Care'

		WHEN agency LIKE '%ACS%' THEN 'Day Care'

		WHEN agency LIKE '%EDUC%' and usetype LIKE '%PLAY AREA%' THEN 'Public K-12 Schools'

		WHEN usetype LIKE '%HIGH SCHOOL%' THEN 'Public K-12 Schools'

		WHEN agency LIKE '%CUNY%' AND usetype NOT LIKE '%OPEN SPACE%' THEN 'Colleges or Universities'

		WHEN agency LIKE '%EDUC%' AND usetype LIKE '%SCHOOL%' THEN 'Public K-12 Schools'

		WHEN usetype LIKE '%EDUCATIONAL SKILLS%' THEN 'Public K-12 Schools'

		ELSE 'Miscellaneous Use'
	END) as facsubgrp`

This will get everything categorized but some facility subgroups are missing from the subset of COLP data, so I'm looking back to see what's missing, but go with this for now

mgraber

comment created time in 16 hours

startedschell/old-gods

started time in 17 hours

startedcogciprocate/ocl

started time in 17 hours

push eventNYCPlanning/db-community-profiles

mgraber

commit sha 71dcf606a56ede2dc4bbf3ae64dec038bd402295

Add LEP vars to combined table

view details

Molly Graber

commit sha 434acff37fedd3399bb2ac98130b0ee49fa2dd3c

Merge pull request #50 from NYCPlanning/uncomment-lep Add LEP vars to combined table

view details

push time in 19 hours

PR opened NYCPlanning/db-community-profiles

Add LEP vars to combined table

Since #49 got merged

+6 -6

0 comment

1 changed file

pr created time in 19 hours

create barnchNYCPlanning/db-community-profiles

branch : uncomment-lep

created branch time in 19 hours

issue openedNYCPlanning/docker-geosupport

Automated check for new releases did not catch 20d

So far the automated workflow has only built a new geosupport docker container when triggered by a dispatch, not by the scheduled run.

For example, several days returned nycplanning/docker-geosupport:20.3.5 already exist well after the November 15 release of 20d.

created time in 19 hours

startedruffle-rs/ruffle

started time in 20 hours

push eventNYCPlanning/db-community-profiles

mgraber

commit sha 52cc72a4860f5dc6037a326acfc0aa7b1193630b

Pull latest geo & facdb, set env first

view details

push time in 20 hours

push eventNYCPlanning/docker-geosupport

Molly Graber

commit sha a49d6614e74ccdca7e8d7c692a68bf62788eaf32

Change dockerfile to new path convention

view details

push time in 20 hours

push eventNYCPlanning/db-community-profiles

mgraber

commit sha f94d5eb6b663f316ab1fe721b11782ad1577cb9b

Create versions csv

view details

mgraber

commit sha 79fd6c530aa0ed78310ff03d3da5818443bc0bbd

Export versions at each step

view details

mgraber

commit sha 316686a32a1f87f47198844fba370397fe0cc018

Merge branch '33-download-views' of github.com:NYCPlanning/db-community-profiles into 33-download-views

view details

push time in 20 hours

issue openedNYCPlanning/db-geolookup

Track input data versions

  • Population division lookup
  • Floodplain
  • Park access

created time in 21 hours

issue closedNYCPlanning/db-community-profiles

ACS: Aggregated CT level indicators to CD level instead of taking PUMA level data

Update the logic that CD geographies are like we create NTA values, and not created PUMA values

closed time in 21 hours

AmandaDoyle

issue closedNYCPlanning/db-community-profiles

ACS/Decennial Floodplain Fields

#11

ACS-derived variables

Field Name ACS equivalent Description
fp_100_cost_burden fp_100_cost_burden_value / B25091_001E Percentage of owner-occupied household units considered cost-burdened within the 2015 Preliminary Flood Insurance Rate Maps (PFIRM) 1% annual chance floodplain. Owner-occupied households are characterized as cost-burdened if 30% or more of household income is spent on housing costs.
fp_500_cost_burden fp_500_cost_burden_value / B25091_001E Percentage of owner-occupied units considered cost-burdened, within the  2050s 1% annual chance floodplain*. Owner-occupied households are characterized as cost-burdened if 30% or more of household income is spent on housing costs.
fp_100_cost_burden_value B25091_019E + B25091_020E + B25091_021E + B25091_022E + B25091_008E + B25091_009E + B25091_010E + B25091_011E Count of households that are cost-burdened within the 2015 Preliminary Flood Insurance Rate Maps (PFIRM) 1% annual chance floodplain.
fp_500_cost_burden_value B25091_019E + B25091_020E + B25091_021E + B25091_022E + B25091_008E + B25091_009E + B25091_010E + B25091_011E Count of households that are cost-burdened within the  2050s 1% annual chance floodplain*.
fp_100_mhhi B19013_001E Median household income within the 2015 Preliminary Flood Insurance Rate Maps (PFIRM) 1% annual chance floodplain
fp_500_mhhi B19013_001E Median household income within the 2050s 1% annual chance floodplain*
fp_100_mortg_value B25091_002E Count of owner-occupied households with mortgages within the 2015 Preliminary Flood Insurance Rate Maps (PFIRM) 1% annual chance floodplain.
fp_500_mortg_value B25091_002E Count of owner-occupied households with mortgages within the  2050s 1% annual chance floodplain*.
fp_100_ownerocc B25003_002E / B25002_001E Proportion of households that are owner-occupied within the 2015 Preliminary Flood Insurance Rate Maps (PFIRM) 1% annual chance floodplain.
fp_500_ownerocc B25003_002E / B25002_001E Proportion of households that are owner-occupied
fp_100_ownerocc_value B25091_001E Count of owner-occupied households within the 2015 Preliminary Flood Insurance Rate Maps (PFIRM) 1% annual chance floodplain.
fp_500_ownerocc_value B25091_001E Count of owner-occupied households within the  2050s 1% annual chance floodplain*.
fp_100_permortg B25091_002E / B25091_001E Percentage of owner-occupied units with a mortgage within the 2015 Preliminary Flood Insurance Rate Maps (PFIRM) 1% annual chance floodplain. Mortgage refers to all forms of debt where the property is pledged as security for repayment of the debt. Mortgage status is an important determinant in the floodplain as any property with a federally-insured mortgage must carry a federal flood insurance policy.
fp_500_permortg B25091_002E / B25091_001E Percentage of owner-occupied household units with a mortgage within the 2050s 1% annual chance floodplain*. Mortgage refers to all forms of debt where the property is pledged as security for repayment of the debt. Mortgage status is an important determinant in the floodplain as any property with a federally-insured mortgage must carry a federal flood insurance policy.
fp_100_rent_burden fp_100_rent_burden_value / B25070_001E Percentage of renter-occupied household units considered cost-burdened within the 2015 Preliminary Flood Insurance Rate Maps (PFIRM) 1% annual chance floodplain. Renter-occupied households are characterized as cost-burdened if 30% or more of household income is spent on housing costs.
fp_500_rent_burden fp_500_rent_burden_value / B25070_001E Percentage of renter-occupied units considered cost-burdened, within the 2050s 1% annual chance floodplain*. Renter-occupied households are characterized as cost-burdened if 30% or more of household income is spent on housing costs.
fp_100_rent_burden_value B25070_007E + B25070_008E + B25070_009E + B25070_010E Count of households that are rent-burdened within the 2015 Preliminary Flood Insurance Rate Maps (PFIRM) 1% annual chance floodplain.
fp_500_rent_burden_value B25070_007E + B25070_008E + B25070_009E + B25070_010E Count of households that are rent-burdened within the  2050s 1% annual chance floodplain*.

Methodology

Block groups were selected for inclusion if the centroids of any one of their constituent blocks intersected with floodplain geographies. Estimates for selected socio-economic variables were obtained from American Community Survey (ACS) for these block groups. ACS estimates were summed to produce public use microdata area level profiles.

Example API call to extract census block group level ACS data by county https://api.census.gov/data/2018/acs/acs5?get=B00001_001E&for=block%20group:*&in=state:01%20county:025

Input tables

Provided by PopDiv

Variable Table Year 100 500
Total Occupied Housing Units B25003 ACS 2014-2018
Owner Occupied B25091 ACS 2014-2018 fp_100_ownerocc fp_500_ownerocc
Owner Occupied with Mortgage B25091 ACS 2014-2018 fp_100_permortg fp_500_permortg
Estimate: Housing Owner Cost Burdened (30% or more of income on rent) B25091 ACS 2014-2018 fp_100_cost_burden fp_500_cost_burden
Renter Occupied B25070 ACS 2014-2018
Estimate: Rent Burdened Households (30% or more of income on rent) B25070 ACS 2014-2018 fp_100_rent_burden fp_500_rent_burden
Median Household Income B19013 ACS 2014-2018 fp_100_mhhi fp_500_mhhi

Note: to calculate estimates on Rent/Owner Cost Burdened Housholds, add 30.0 to 34.9 percent, 35.0 to 39.9 percent, 40.0 to 49.9 percent, 50.0 pecent or more & calculate associated MOE

Geography of census blocks, as related to the two floodplains

-- In RECIPES
CREATE TEMP TABLE AS cd_bctcb2010_centroids (
    SELECT 
    cd,
    (CASE
            WHEN LEFT(a.bctcb2010, 1) = '1' THEN '36061'||RIGHT(a.bctcb2010, 10)
            WHEN LEFT(a.bctcb2010, 1) = '2' THEN '36005'||RIGHT(a.bctcb2010, 10)
            WHEN LEFT(a.bctcb2010, 1) = '3' THEN '36047'||RIGHT(a.bctcb2010, 10)
            WHEN LEFT(a.bctcb2010, 1) = '4' THEN '36081'||RIGHT(a.bctcb2010, 10)
            WHEN LEFT(a.bctcb2010, 1) = '5' THEN '36085'||RIGHT(a.bctcb2010, 10)
    END) as block_fips, 
    (CASE
            WHEN LEFT(a.bctcb2010, 1) = '1' THEN '36061'||SUBSTRING(a.bctcb2010, 2, 7)
            WHEN LEFT(a.bctcb2010, 1) = '2' THEN '36005'||SUBSTRING(a.bctcb2010, 2, 7)
            WHEN LEFT(a.bctcb2010, 1) = '3' THEN '36047'||SUBSTRING(a.bctcb2010, 2, 7)
            WHEN LEFT(a.bctcb2010, 1) = '4' THEN '36081'||SUBSTRING(a.bctcb2010, 2, 7)
            WHEN LEFT(a.bctcb2010, 1) = '5' THEN '36085'||SUBSTRING(a.bctcb2010, 2, 7)
    END) as block_group_fips,
    ST_Centroid(b.wkb_geometry) as centroid_geom
    FROM dcp_bctcb2010_cd_puma a
    JOIN dcp_censusblocks.latest b
    ON a.bctcb2010 = b.bctcb2010
);
CREATE INDEX "bctcb_2010_geom_idx" ON cd_bctcb2010_centroids USING GIST (centroid_geom gist_geometry_ops_2d);

CREATE TEMP TABLE cd_blocks_floodplain AS (
        SELECT
              a.*,
              bool_or(CASE 
     	               WHEN NOT a.centroid_geom && b.wkb_geometry THEN FALSE
     	                ELSE ST_Intersects(a.centroid_geom,  b.wkb_geometry) 
              END) as in_100,
	     bool_or(CASE
		     WHEN NOT a.centroid_geom && c.wkb_geometry THEN FALSE
		     ELSE ST_Intersects(a.centroid_geom,  c.wkb_geometry)
	     END) as in_500 
FROM cd_bctcb2010_centroids a, fema_pfirms2015_100yr.latest b, fema_firms_500yr.latest c
GROUP BY a.cd, a.block_fips, a.block_group_fips, a.centroid_geom);


Decennial-derived variables

Field Name Census equivalent Description
fp_100_pop P001001 Population of the CD that lives within the 2015 Preliminary Flood Insurance Rate Maps (PFIRM) 1% annual chance floodplain
fp_500_pop P001001 Population (2010) of the CD that lives within the 2050s 1% annual chance floodplain*

Methodology

Floodplain geographies were defined in terms of 2010 census block geographies employing the centroid containment method, which selected 2010 census blocks if their centroids intersected the floodplain areas. If a block’s centroid lay external to the floodplain, the census block did not qualify for inclusion. For selected blocks residing in floodplain areas, selected demographic data were obtained and then aggregated to produce demographic profiles of floodplain areas that lay within community districts.

Example API call to extract census block level decennial data by county https://api.census.gov/data/2010/dec/sf1?get=P001001,NAME&for=block:*&in=state:01%20county:009

closed time in 21 hours

mgraber

issue closedNYCPlanning/db-community-profiles

Switch parks methodology to block centroids

If a census block centroid intersects with the park access boundary, count the entire population of that block as having access to parks.

DPR would like a comparison of park access rates using the two methods.

closed time in 21 hours

mgraber

push eventNYCPlanning/db-community-profiles

Molly Graber

commit sha 935e7b16edbc2ad0cd0d7fb713da81723eed7e68

Split pip commands

view details

push time in a day

push eventNYCPlanning/db-community-profiles

Molly Graber

commit sha 71e6e1682524ba612a0d5c354206fb21378a97d9

Install setuptools

view details

push time in a day

push eventNYCPlanning/db-factfinder

Molly Graber

commit sha 93359d612d5b8e961a3bf78a77d19c46ce2253fa

Remove setuptools

view details

push time in a day

push eventNYCPlanning/db-factfinder

Molly Graber

commit sha fa992b6cb2aacf118ad12f43dc96d1d85f7637b6

Add setuptools to requirements

view details

push time in a day

push eventNYCPlanning/db-community-profiles

Molly Graber

commit sha 46752aa4387a4abafae439bc83ec69d0c8d8d8cb

Change name in old workflow

view details

push time in a day

push eventNYCPlanning/db-community-profiles

Molly Graber

commit sha 735e40bc2381356a7c4dfedd51261151bac0df99

Change dispatch tag in old workflow

view details

push time in a day

push eventNYCPlanning/db-community-profiles

mgraber

commit sha a5ab4c413dacba6a30a7cccc4e7533a55dbdbcfd

Include 500yr floodplain unit and bldg vars

view details

Molly Graber

commit sha 68c7ec681204570dfbd934f5c9c3c307414484e8

Merge pull request #37 from NYCPlanning/fp-500-bldg Add missing PLUTO-derived floodplain fields

view details

push time in a day

PR merged NYCPlanning/db-community-profiles

Reviewers
Add missing PLUTO-derived floodplain fields
  • [x] fp_500_bldg
  • [x] fp_500_resunits

Census fields will be a different PR

+22 -2

0 comment

2 changed files

mgraber

pr closed time in a day

push eventNYCPlanning/db-community-profiles

mgraber

commit sha c4dc2f5c145fc39fc5753fdb72ba176933597544

Use fact finder for parks aggregation

view details

mgraber

commit sha f06fef55f5cce08f126036942c2183cbb098c229

Use new pff workflow to calculate fp acs vars

view details

mgraber

commit sha e83c892fbe88229da50b497c163a15e6df023222

Fix imports

view details

mgraber

commit sha 0537acb2bad574cb3ac391e9e36ac06dbfd4e80e

Pull acs and decennial dates from version.env

view details

mgraber

commit sha 421a6895532e91ef162f7ef2a57fe7ae25e8a07f

Add build.yml to run through actions

view details

mgraber

commit sha e6e3dd966afc60c9259ca0ca38ff38e87e03ea44

Use dispatch for versions

view details

Molly Graber

commit sha 4c8792b7e256fa43573b3307e3aa9267d185784c

Merge pull request #46 from NYCPlanning/run-through-actions Add build.yml to run through actions

view details

mgraber

commit sha c919d609451697d197055e3f48dcdccc998bf56a

Fix year parsing

view details

Molly Graber

commit sha 0dfa9df4f3a8eeca274a19e7acb45d0927081daa

Merge pull request #47 from NYCPlanning/run-through-actions Fix year parsing

view details

mgraber

commit sha c2e28de4da90a6a8dd19234b4eb7e974a5f4770c

Add in_floodplain_demo.sql

view details

mgraber

commit sha bd1b792102d415c62c81e562088fde18cbb997dc

Include floodplain demo fields in combine.sql

view details

mgraber

commit sha deee66f0fccbe65723aec6f33adb928eb0c7a7ca

Add out_acs.py

view details

mgraber

commit sha d2ce3d1437fa5b142984ddb2814e94045810867d

Modify in_acs to reflect factfinder method

view details

mgraber

commit sha dc009003e30787d2792aa0e6b62f40ecbf6c6a0a

Comment out lep rate vars

view details

mgraber

commit sha c316cb4bedf102e34baffc5f604dcefb9cc29686

Switch to python version of out_acs

view details

mgraber

commit sha 5528946e10ed4e204cce929ea79223ca1d3dff01

Add LEP variables

view details

Molly Graber

commit sha 9a3f739d8308d2a31d9d56ecc8d0c4dd942d74d8

Merge pull request #49 from NYCPlanning/add-lgoenlep1 Add LEP variables

view details

mgraber

commit sha ea0a7332c927449013a792e42d9a35df44b3630e

Add missing THEN statement

view details

Molly Graber

commit sha 1da820aa234713c86b11c356a97742fc589ec24e

Merge pull request #45 from NYCPlanning/refactor-parks Refactor parks, ACS, and floodplain workflows to use factfinder

view details

push time in a day

PR merged NYCPlanning/db-community-profiles

Reviewers
Refactor parks, ACS, and floodplain workflows to use factfinder

Merge https://github.com/NYCPlanning/db-factfinder/pull/52 and #49 first.

PR includes:

  • Transitions workflow from build.sh runner to github actions
    • build.yml is dispatch-triggered. If we settle on this method, be sure to change trigger in the existing yml or delete the existing yml so only one gets triggered.
    • Dispatch contains versions, rather than using version.env. This might not be the best permanent fix. Example dispatch:
curl --location --request POST 'https://api.github.com/repos/NYCPlanning/db-community-profiles/dispatches?Accept=application/vnd.github.v3+json&Content-Type=application/json' \
--header 'Authorization: Bearer {insert token here}' \
--header 'Content-Type: text/plain' \
--data-raw ‘{"event_type" : "build",\
		 "client_payload": {"V_PLUTO":"20v4",\
			"V_ACS":"Y2014-2018",\
                        "V_DECENNIAL":"2010",\
                        "V_FACDB":"2020/06/24",\
                        "V_CRIME":"2019",\
                        "V_SANITATION":"2020/08",\
                        "V_GEO":"20b",\
                        "V_PARKS":"2019/01/29",\
                        "V_POVERTY":"Y2014-2018"}’
  • Uses factfinder package to calculate park access, floodplain, and ACS variables. This will likely need additional testing after merge. Testing github actions pre-merge is challenging. An example of just the python portion of this process, including example csv output, is available here.
  • Retains old workflow for parks calculations (with output location changed to PARKS_PROPORTIONAL) because DPR wants a comparison between the two.
  • Note that the ACS implementation does not include lep variables. Merge https://github.com/NYCPlanning/db-factfinder/pull/52 and #49 first for this implementation.

#15 #43 #44

+687 -304

1 comment

15 changed files

mgraber

pr closed time in a day

push eventNYCPlanning/db-community-profiles

mgraber

commit sha ea0a7332c927449013a792e42d9a35df44b3630e

Add missing THEN statement

view details

push time in a day

push eventNYCPlanning/db-factfinder

mgraber

commit sha 89eee04579264506a3e53dcaac749ec32f61b3a2

Add comm prof version of lgoenlep1

view details

Molly Graber

commit sha 76ad785b9e077558acc207e85e1b07f8e3c28049

Merge pull request #52 from NYCPlanning/comm-prof-lep Add lgoenlep1 for community profiles

view details

push time in a day

PR merged NYCPlanning/db-factfinder

Add lgoenlep1 for community profiles

#43

Future language data for PFF will come from the collapsed 5-year table C16001. This table is similar to the previous source, but collapsed languages into larger categories. Population has yet to scope out the variables they want for each language category. Because the variable lgoenlep1 is the sum of low-english proficiency across all non-English languages, we can create this variable for Community Profiles prior to mapping out all language variables for PFF.

This PR includes lgoenlep1 and its associated base variable with domain community_profiles to use until we fully build out the replacement language variables for PFF.

+31 -0

0 comment

1 changed file

mgraber

pr closed time in a day

more