My Neighborhood Lookup
A Drupal component that displays various pieces of information about the entered address.
The my neighborhood application is a Drupal component that can be added to any page on boston.gov. Currently, it is available here: https://www.boston.gov/my-neighborhood
Technical Architecture and Synchronization
The My Neighborhood Lookup (MNL) application on Boston.gov relies on a weekly data synchronization from the City's Civis Analytics platform to a Drupal endpoint.
Data Import Workflow
Data is pushed from Civis to Drupal in chunks of approximately 50,000 records.
Endpoint:
https://www.boston.gov/rest/mnl/import?api_key=<secretkey>.Duration: A healthy full synchronization typically takes 30–45 minutes.
Performance Baseline: Under normal conditions, a single chunk of 50,000 records should process in approximately 3–4 minutes.
Completion Signaling
The Drupal system expects a specific signal to finalize the import and update its internal status flags.
The last record in the final data chunk must be a single line containing the text
complete!.If this signal is missing, the system defaults to checking if the queue is empty.
Note: Relying on the empty queue (rather than the
complete!signal) can cause state issues if the time between sending chunks exceeds the time Drupal takes to process them.
Troubleshooting and Maintenance
Known Issues: "Stuck" Imports
A common failure mode is the MNL import process becoming stuck "in process". While in this state, the system will reject requests for new imports.
How to Identify a Stuck Process:
Check the MNL monitoring page; the status will show "in process".
Check the import status flag via Drush:
drush @bostond8.prod cget bos_mnl.settings mnl_import_import_status.0 = Idle.
1 = Active/Importing.
How to Reset the Process:
If the queue is confirmed empty but the status remains "1", manually reset the flag:
drush @bostond8.prod cset -y bos_mnl.settings mnl_import_import_status 0.
Server Performance (Acquia)
Server unresponsiveness on the Acquia hosting environment can cause "silent timeouts".
Symptom: Chunk processing time climbs to 7+ minutes.
Resolution: In such cases, the production server may require a restart by Acquia to restore expected performance levels.
Monitoring Schedule
The MNL export is scheduled to run once a week on Sundays. Because it is part of an orchestrated build process, the exact timing is variable but generally occurs late Sunday night or early Monday morning.
Archived Information (No Longer Active)
Drupal Import Scripts
There are two Drupal endpoints associated with this application.one for receiving the updates
One for receiving the updated records on a nightly basis (updates):
https://www.boston.gov/rest/mnl/updateOne for receiving the full load of records:
https://www.boston.gov/rest/mnl/import
This page has information on the status of Drupal import scripts that run nightly and once a month.
Data Sources
The data used in this application come from a variety of GIS data sources. This spreadsheet lists them all in addition to the workflow that brings each one into Civis.
These datasources are combined with the SAM address dataset in Civis. This workflow is the one that combines all the datasets.
Every night, the my neighborhood workflow runs and sends the any records that have updated or changed to boston.gov. Once a month on the 1st, the workflow sends the entire load of records to Drupal.
NOTES: There are two cards that show data which are hard coded and don't sync with Civis. They are the mayor's name in the "YOUR MAYOR" card found in "mnl_config.js" file and the "YOUR AT-LARGE CITY COUNCILORS" card found in "Representation.js" file.
In addition to the hard coded items, there too is a data dependency on ReCollect. We built an endpoint in Drupal "rest/recollect" found in Drupal module "bos_mnl" to query ReCollect API with the user's inputted address and returns the next trash and recycling date for that specific address.
Historical Documentation
Last updated
Was this helpful?