Legislator matching been OCPF and legislature IDs#2168
Open
J-C-L wants to merge 6 commits into
Open
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
013e12c to
c85bd92
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The legislature and OCPF use different ID systems, so in order to use campaign finance data from OCPF, we need to map OCPF filer IDs to the corresponding member ID for the legislator.
This creates a firebase function to get the filers list from OCPF, do the mapping against our current court members, and store the results in Firestore. Successful matches are stored in config/OcpfMemberMapping. Members that couldn't be matched (either no match found or multiple possible matches) are stored in OcpfMemberMappingFlags, with details written to the logs.
This process will have to be run once for each legislative election cycle, and we will have to manually address the unmatched members.
Checklist
firestore.indexes.json(Please do not only create indexes through the Firebase Web UI, even though the error messages may reccommend it - indexes created this way may be obliterated by subsequent deploys)Steps to test/reproduce
Describing Local steps (Will replace with dev steps once I've tested there):
Make sure Firestore generalCourts/194/Members is populated (If not, run curl "http://localhost:5001/demo-dtp/us-central1/triggerPubsubFunction?scheduled=startMemberBatches" twice with 30 second delay
Run curl -X POST http://localhost:5001/demo-dtp/us-central1/matchOcpfMembers
Confirm Firestore config/OcpfMemberMapping exists and the number of entries matches terminal output from {"results":{"matched":182...}...}
Confirm entry exists for SND0, with OCPF ID: 15031, name : "Sal N. DiDomenico"
Confirm config/OcpfMemberMappingFlags now exists with "unmatched" and "ambiguous" fields, and each field has the number of entries shown in terminal output: {"results":{"matched":182,"unmatched":8,"ambiguous":5}...}
Look at the Firebase logs in the console and filter for matchOcpfMembers. Scan results to see the details logged for the "unmatched" and "ambiguous" members mentioned above.
Confirm that once a manual fix is made, it will not be overwritten on a subsequent run:
new field for MemberID (M_C1) , with type “map” with 2 nested fields inside: { cpfId: 15470, name: "Mike Connolly" }