-
Notifications
You must be signed in to change notification settings - Fork 4
Prevent publication of duplicate theses with holds #1511
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -6,6 +6,7 @@ class RegistrarImportJob < ActiveJob::Base | |
| def perform(registrar) | ||
| results = { read: 0, processed: 0, new_users: 0, new_theses: 0, updated_theses: 0, new_degrees: [], new_depts: [], | ||
| new_degree_periods: [], errors: [] } | ||
| new_theses = [] # Track newly created theses to detect duplicate theses with holds | ||
|
|
||
| CSV.new(registrar.graduation_list.download, headers: true).each.with_index(1) do |row, i| | ||
| Rails.logger.info("Parsing row #{i}") | ||
|
|
@@ -39,7 +40,12 @@ def perform(registrar) | |
| # Set whodunnit for thesis transaction | ||
| PaperTrail.request.whodunnit = 'registrar' | ||
| thesis = Thesis.create_or_update_from_csv(user, degree, department, grad_date, row) | ||
| thesis.new_thesis? ? results[:new_theses] += 1 : results[:updated_theses] += 1 | ||
| if thesis.new_thesis? | ||
| results[:new_theses] += 1 | ||
| new_theses << thesis | ||
| else | ||
| results[:updated_theses] += 1 | ||
| end | ||
| logger.info("Thesis is #{thesis.inspect}") | ||
| rescue RuntimeError | ||
| e = "Multiple theses found for author #{user.name} for term #{grad_date}, requires Processor attention. CSV row ##{i}: #{row.inspect}" | ||
|
|
@@ -55,11 +61,36 @@ def perform(registrar) | |
| author.set_graduated_from_csv(row) | ||
| results[:processed] += 1 | ||
| end | ||
|
|
||
| multiple_hold_users = collect_users_with_multiple_hold_theses(new_theses) | ||
|
|
||
| Rails.logger.info(results.to_s) | ||
| ReportMailer.registrar_import_email(registrar, results).deliver_later | ||
| ReportMailer.registrar_import_email(registrar, results, multiple_hold_users).deliver_later | ||
| results | ||
| end | ||
|
|
||
| def collect_users_with_multiple_hold_theses(new_theses) | ||
| multiple_hold_users = [] | ||
|
|
||
| new_theses.each do |thesis| | ||
| other_theses_with_holds = thesis.other_theses_with_holds.includes(:users).to_a | ||
| thesis.users.each do |user| | ||
| user_with_other_theses_with_holds = other_theses_with_holds.select do |other_thesis| | ||
| other_thesis.users.any? { |u| u.id == user.id } | ||
| end | ||
| next if user_with_other_theses_with_holds.empty? | ||
|
|
||
| multiple_hold_users << { | ||
| user: user, | ||
| new_thesis: thesis, | ||
| other_theses_with_holds: user_with_other_theses_with_holds | ||
| } | ||
| end | ||
| end | ||
|
Copilot marked this conversation as resolved.
|
||
|
|
||
| multiple_hold_users | ||
| end | ||
|
Comment on lines
+72
to
+92
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think this is a tad overstated. We execute one DB query per new thesis to fetch related theses and eager-load users on those theses. |
||
|
|
||
| # The thesis model sets the day of the month to 1 if only supplied a month | ||
| # and a year during thesis creation, which means in practice we have to | ||
| # assume that the day is always 1 (because this will be true for any theses | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,32 +1,33 @@ | ||
| class ReportMailer < ApplicationMailer | ||
| def registrar_import_email(registrar, results) | ||
| def registrar_import_email(registrar, results, multiple_hold_users = []) | ||
| return unless ENV.fetch('DISABLE_ALL_EMAIL', 'true') == 'false' # allows PR builds to disable emails | ||
|
|
||
| @registrar = registrar | ||
| @results = results | ||
| mail(from: "MIT Libraries <#{ENV['ETD_APP_EMAIL']}>", | ||
| to: ENV['THESIS_ADMIN_EMAIL'], | ||
| cc: ENV['MAINTAINER_EMAIL'], | ||
| @multiple_hold_users = multiple_hold_users | ||
| mail(from: "MIT Libraries <#{ENV.fetch('ETD_APP_EMAIL', nil)}>", | ||
| to: ENV.fetch('THESIS_ADMIN_EMAIL', nil), | ||
| cc: ENV.fetch('MAINTAINER_EMAIL', nil), | ||
|
Comment on lines
+8
to
+10
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This conflicts directly with rubocop guidance. |
||
| subject: 'Registrar data import summary') | ||
| end | ||
|
|
||
| def publication_results_email(results) | ||
| return unless ENV.fetch('DISABLE_ALL_EMAIL', 'true') == 'false' # allows PR builds to disable emails | ||
|
|
||
| @results = results | ||
| mail(from: "MIT Libraries <#{ENV['ETD_APP_EMAIL']}>", | ||
| to: ENV['THESIS_ADMIN_EMAIL'], | ||
| cc: ENV['MAINTAINER_EMAIL'], | ||
| mail(from: "MIT Libraries <#{ENV.fetch('ETD_APP_EMAIL', nil)}>", | ||
| to: ENV.fetch('THESIS_ADMIN_EMAIL', nil), | ||
| cc: ENV.fetch('MAINTAINER_EMAIL', nil), | ||
|
Comment on lines
+18
to
+20
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This conflicts directly with rubocop guidance. |
||
| subject: 'DSpace publication results summary') | ||
| end | ||
|
|
||
| def preservation_results_email(results) | ||
| return unless ENV.fetch('DISABLE_ALL_EMAIL', 'true') == 'false' # allows PR builds to disable emails | ||
|
|
||
| @results = results | ||
| mail(from: "MIT Libraries <#{ENV['ETD_APP_EMAIL']}>", | ||
| to: ENV['THESIS_ADMIN_EMAIL'], | ||
| cc: ENV['MAINTAINER_EMAIL'], | ||
| mail(from: "MIT Libraries <#{ENV.fetch('ETD_APP_EMAIL', nil)}>", | ||
| to: ENV.fetch('THESIS_ADMIN_EMAIL', nil), | ||
| cc: ENV.fetch('MAINTAINER_EMAIL', nil), | ||
|
Comment on lines
+28
to
+30
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This conflicts directly with rubocop guidance. |
||
| subject: 'Archivematica preservation submission results summary') | ||
| end | ||
| end | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Instead of keeping this larger list and then winnowing it down at the end to the (hopefully much smaller or empty) list I feel like we could update
collect_users_with_multiple_hold_thesesto act on a single thesis and call it here and refactor to only be trackingmultiple_hold_usersarray. I think this is both more memory efficient and easier to follow unless we need the full list for something I'm not seeing yetThere was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Unless I misunderstand the flow, I think we need the full list to refer back to for later loop iterations.
Let's say we have a thesis with two coauthors. Author 1 is earlier in the CSV and has no other held theses; Author 2 is later and does have another held thesis. If the hold check only runs when the thesis is first created, we would miss the hold warning for Author 2. It's an unlikely use case, but one that we ought to consider.
I agree that we can refactor such that it doesn't store the thesis objects in memory while still running a hold check on every row, but that would likely add complexity. How concerned are we about efficiency?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah, I missed that important use case. Let's leave it as is and if we notice performance issues we can revisit. We probably won't :)
My concern noted the performance concern, but was really rooted in trying to make this easier to follow... but accuracy is more important than simplicity for this for sure.