This repository is a historical record of the City Record Online Workgroup (CROW), 2014–2017. The project succeeded:
- CROL launched in 2014 under Local Law 7 of 2014 (Intro 363-2014) — the City Record is published online daily at a856-cityrecord.nyc.gov.
- The data is open: the City Record Online dataset on NYC Open Data carries 1M+ notices (2003–present; full multi-section coverage from 2013), updated daily.
- Programmatic access: BetaNYC maintains nyc-record-mcp, an MCP server over the City Record API.
- The 2008–2014 text-file archive in
Current DCAS Implementation/is preserved (with checksummed manifest) in a BetaNYC archive repository, in preparation for public release. - The sister schema effort lives at CROL-Schema. (
CROL-PDFandCROL-Parsingwere earlier names of this repo.)
Code under Planned Implementation/ is Python 2-era and unmaintained — kept for historical reference, not for use. Several community links below (talk.beta.nyc, Google Drive folders) are from 2014–2015 and may only exist in the Wayback Machine.
This is the main repository containing efforts pertaining to the parsing efforts of CROW. For Notice Schema development, see https://github.com/CityOfNewYork/CROL-Schema.
Disclaimer. In case of conflicting document versions, please refer to documents mentioned in GitHub as the latest version.
- Gold standard - a human parsed file that showed the "correct" extraction of the different object.
- Preserved in this repo (2026 export):
Planned Implementation/Gold Standard.xlsx— CSV: gold-standard sheet (479 rows, human-parsed Oct–Dec 2014 procurement notices)
- Preserved in this repo (2026 export):
- The Main Schema - a reference file that shows what all the output fields should be and where (the source) they can be derived from.
- Preserved in this repo (2026 export):
Planned Implementation/Main Schema (Output).xlsx— CSV versions: main sheet, schema guide
- Preserved in this repo (2026 export):
###Open Standard Links
- Discussion List (discussion forum retired; Wayback captures)
- Ongoing Tasks
- Resources
- Tools and Wiki
- Download PDFs
- Press Release & News Articles
As the City embarks on implementing Intro 363-2014 and unlocking its daily actions, we are working together with the Department of Citywide Services to publish the City Record as open, clean and structured data. At the same time, we are unlocking decades of historical information and making it accessible to all, at no charge.
Our goal is to optimize the utility of City Record content by making accessible and structuring the data; addresses, dates, persons, subjects, agencies, contract types and more are parsed and made available as individual objects. This way, residents, organizations and small and large businesses alike will be able to access, interact and stay informed, whether through notifications, visualizations or other easy-to-use community tools.
- City of New York
- BetaNYC
- Commune
- Citizens Union
- Dev Bootcamp
- Ontodia
- Socrata
- Sunlight Foundation
- Came together to form a CROW parsing and scraping volunteer team
- Set up collaboration framework with DCAS
- Scraped PDFs from 2008 - 2014
- Proposed public notice schema
- Added “addresses” and “time & dates” fields to the City’s input workflow
For a list of current tasks, please see Issues.
-
Develop a set of collaboratively produced open source library parsers to populate the Public Notice Data Standard schema using the DCAS pipeline
-
Work with DCAS to implement the pipeline into the City’s workflow by August 1, and use that as their way of publishing the City Record data
-
Publish a Public Notice Data Standard and documentation on an interactive website
- Scrape the archival PDFs
- Apply and modify the parsers to be able to parse and structure the data in the PDFs