diff --git a/docs/genai/02_onboarding/01_intro.mdx b/docs/genai/02_onboarding/01_intro.mdx index b8e6e982a9..d34d8738e0 100644 --- a/docs/genai/02_onboarding/01_intro.mdx +++ b/docs/genai/02_onboarding/01_intro.mdx @@ -2,7 +2,7 @@ [research-workspace-request]: https://forms.gle/6DVA5dX4uPPrdX4Q9 [member-onboarding-form]: https://forms.gle/wd3zcntnY94Q4LWq5 -This section deals with the eligibility for getting access to Pythia. To learn more about affiliate access (students after graduation, collaborators from other institutions, please refer to [this section](../../hpc/01_getting_started/02_getting_and_renewing_an_account.mdx)) +This section deals with the eligibility for getting access to Pythia. To learn more about affiliate access (students after graduation, collaborators from other institutions, please refer to [this section](../../hpc/01_getting_started/02_HPC_Accounts/01_getting_and_renewing_an_account.mdx)) :::tip[workspaces] Access to Pythia is facilitated via workspaces. Beyond facilitating access, they also allow team members to collaboratively dvelop prompts. diff --git a/docs/hpc/01_getting_started/01_intro.mdx b/docs/hpc/01_getting_started/01_intro.mdx index 8cdfed6041..87dc30ec2f 100644 --- a/docs/hpc/01_getting_started/01_intro.mdx +++ b/docs/hpc/01_getting_started/01_intro.mdx @@ -2,6 +2,10 @@ Welcome to the Torch HPC documentation! If you do not have an HPC account, please proceed to the next section that explains how you may be able to get one. +:::note +Please be aware that a requirement of the Torch cluster is that you also have an associated Slurm Account with resources allocated to it. You can learn more about this in the section on [Slurm Accounts](./03_Slurm_Accounts/01_intro_slurm_accounts.mdx) +::: + If you are an active user, you can proceed to one of the categories on the left. ## Announcements diff --git a/docs/hpc/01_getting_started/02_getting_and_renewing_an_account.mdx b/docs/hpc/01_getting_started/02_HPC_Accounts/01_getting_and_renewing_an_account.mdx similarity index 95% rename from docs/hpc/01_getting_started/02_getting_and_renewing_an_account.mdx rename to docs/hpc/01_getting_started/02_HPC_Accounts/01_getting_and_renewing_an_account.mdx index c3be3cf395..ff2cc7bc7b 100644 --- a/docs/hpc/01_getting_started/02_getting_and_renewing_an_account.mdx +++ b/docs/hpc/01_getting_started/02_HPC_Accounts/01_getting_and_renewing_an_account.mdx @@ -23,7 +23,7 @@ This section deals with the eligibility for getting HPC accounts, the process to :::warning -An active HPC account gives you access to Torch, but an active allocation within the HPC projects management portal gives you access to a SLURM account which is needed to run jobs on Torch. More information on using the HPC project management portal can be found [here](./07_hpc_project_management_portal.mdx) +An active HPC account gives you access to Torch, but an active allocation within the HPC projects management portal gives you access to a SLURM account which is needed to run jobs on Torch. More information on using the HPC project management portal can be found [here](../03_Slurm_Accounts/02_hpc_project_management_portal.mdx) ::: @@ -36,11 +36,11 @@ NYU HPC clusters and related resources are available to full-time NYU faculty an ## Getting a New Account on the NYU HPC Clusters -To request an NYU HPC account please log in to [NYU Identity Management service][nyu ims link] and follow the link to "Request HPC account" by clicking on the hamburger icon on the top left of the page and selecting "Manage Access" . We have a walkthrough of how to [request an account through IIQ](03_walkthrough_request_hpc_account.mdx). If you are a student, alumni or an external collaborator you will need an NYU faculty sponsor. +To request an NYU HPC account please log in to [NYU Identity Management service][nyu ims link] and follow the link to "Request HPC account" by clicking on the hamburger icon on the top left of the page and selecting "Manage Access" . We have a walkthrough of how to [request an account through IIQ](02_walkthrough_request_hpc_account.mdx). If you are a student, alumni or an external collaborator you will need an NYU faculty sponsor. ## Renewing HPC Account -Each year, non-faculty users must renew their HPC account by filling in the account renewal form from the [NYU Identity Management service][nyu ims link]. See [Renewing your HPC account](05_walkthrough_renew_hpc_account.md) for a walkthrough of the process. +Each year, non-faculty users must renew their HPC account by filling in the account renewal form from the [NYU Identity Management service][nyu ims link]. See [Renewing your HPC account](04_walkthrough_renew_hpc_account.md) for a walkthrough of the process. ## Information for Faculty Who Sponsor HPC Users @@ -72,7 +72,7 @@ HPC faculty sponsors are expected to: - Respond promptly to account-related requests from HPC staff -Each year, your sponsored users must renew their account. You will need to approve the renewal by logging into the [NYU Identity Management service][nyu ims link]. We have a [walkthrough of the approval process here](04_walkthrough_approve_hpc_account_request.md) +Each year, your sponsored users must renew their account. You will need to approve the renewal by logging into the [NYU Identity Management service][nyu ims link]. We have a [walkthrough of the approval process here](03_walkthrough_approve_hpc_account_request.md) ## Bulk HPC Accounts for Courses @@ -97,7 +97,7 @@ Accounts created for courses last until the end of the semester, rather than a f If you are collaborating with NYU researchers you will need to obtain **affiliate** status before applying for an NYU HPC account. A full-time NYU faculty member must sponsor a non-NYU collaborator for affiliate status. -Please see [instructions for affiliate management][affiliate and account management link] (NYU NetID login is required to follow the link). [Please read instructions about sponsoring external collaborators here](06_hpc_accounts_external_collaborators.md). +Please see [instructions for affiliate management][affiliate and account management link] (NYU NetID login is required to follow the link). [Please read instructions about sponsoring external collaborators here](05_hpc_accounts_external_collaborators.md). ## Access to Cluster After Graduation diff --git a/docs/hpc/01_getting_started/03_walkthrough_request_hpc_account.mdx b/docs/hpc/01_getting_started/02_HPC_Accounts/02_walkthrough_request_hpc_account.mdx similarity index 78% rename from docs/hpc/01_getting_started/03_walkthrough_request_hpc_account.mdx rename to docs/hpc/01_getting_started/02_HPC_Accounts/02_walkthrough_request_hpc_account.mdx index 3f3f15f366..8b5f14c364 100644 --- a/docs/hpc/01_getting_started/03_walkthrough_request_hpc_account.mdx +++ b/docs/hpc/01_getting_started/02_HPC_Accounts/02_walkthrough_request_hpc_account.mdx @@ -4,7 +4,7 @@ import TabItem from '@theme/TabItem'; # How to request an HPC account :::tip -Make sure you don't already have an HPC account. You can check this by attempting to log in to the cluster, according to the instructions at [Connecting to the HPC Cluster](../02_connecting_to_hpc/01_connecting_to_hpc.mdx). +Make sure you don't already have an HPC account. You can check this by attempting to log in to the cluster, according to the instructions at [Connecting to the HPC Cluster](../../02_connecting_to_hpc/01_connecting_to_hpc.mdx). ::: Login to the URL given below, using your NetID/password, to create or manage HPC Account Requests: @@ -16,30 +16,30 @@ You need to be on the NYU VPN to perform this task! Upon logging in, an end user’s landing page will look like this: - ![Request account landing page](./static/identity_landing.png) + ![Request account landing page](../static/identity_landing.png) Note that if the menu does not appear, select the "burger" menu on the top left hand corner: - ![Request burger](./static/request_burger.png) + ![Request burger](../static/request_burger.png) Navigate to Manage Accounts > Request HPC Account: - ![Request account left hand button](./static/request_account.png) + ![Request account left hand button](../static/request_account.png) You will be prompted with a form. Please fill it out and Submit. - ![Request account form](./static/request_form.png) + ![Request account form](../static/request_form.png) When an approver/sponsor/faculty logs in, it will look like this: - ![Faculty landing](./static/faculty_request_landing.png) + ![Faculty landing](../static/faculty_request_landing.png) with “QuickLinks” to both “Bulk HPC Account Request” and “Request HPC Account” and will show any recent or pending approvals or forms. If the **Request HPC Account** QuickLink is clicked, the following form appears: - ![Faculty request form](./static/faculty_request_form.png) + ![Faculty request form](../static/faculty_request_form.png) The user’s name will be pre populated, and the forms required fields must be completed (sponsor, reason for request, consent to terms of use). After clicking “Submit” the chosen sponsor will be notified of the request and provisioning will only occur after approval. If the **Bulk HPC Account Request** QuickLink is clicked, the following form appears: - ![Bulk account request form](./static/bulk_acct_req_form.png) + ![Bulk account request form](../static/bulk_acct_req_form.png) The requestor’s name will be prepopulated, and the forms required fields must be completed (sponsor, list of netids, course identifier, reason for request, consent to terms of use). These requests are auto-approved since they are usually submitted by the sponsor themselves or a member of the HPC Admin team. @@ -47,6 +47,6 @@ You need to be on the NYU VPN to perform this task! :::tip -- An HPC account gives you access to Torch, but an active allocation within the HPC projects management portal gives you access to a SLURM account which is needed to run jobs on Torch. More information on using the HPC project management portal can be found [here](./07_hpc_project_management_portal.mdx) +- An HPC account gives you access to Torch, but an active allocation within the HPC projects management portal gives you access to a SLURM account which is needed to run jobs on Torch. More information on using the HPC project management portal can be found [here](../03_Slurm_Accounts/02_hpc_project_management_portal.mdx) ::: diff --git a/docs/hpc/01_getting_started/04_walkthrough_approve_hpc_account_request.md b/docs/hpc/01_getting_started/02_HPC_Accounts/03_walkthrough_approve_hpc_account_request.md similarity index 83% rename from docs/hpc/01_getting_started/04_walkthrough_approve_hpc_account_request.md rename to docs/hpc/01_getting_started/02_HPC_Accounts/03_walkthrough_approve_hpc_account_request.md index c59bd50082..f52d52e49f 100644 --- a/docs/hpc/01_getting_started/04_walkthrough_approve_hpc_account_request.md +++ b/docs/hpc/01_getting_started/02_HPC_Accounts/03_walkthrough_approve_hpc_account_request.md @@ -7,19 +7,19 @@ You need to be on the NYU VPN to perform this task! ::: You can also [log into IIQ at any time](https://iiq.nyu.edu/identityiq), and if you have a request awaiting your approval, it will appear in your "Actions Items" box, as per the following screenshot: -!["Actions Items" box](./static/work_item.png) +!["Actions Items" box](../static/work_item.png) Another way to get to pending approvals is to click on the line item in the “Latest Approvals” section which will lead directly to the approval page. For new HPC Account Requests, the page will look like this: -![“Latest Approvals” section](./static/latest_approvals.png) +![“Latest Approvals” section](../static/latest_approvals.png) Here, the Approve or Deny button should be clicked, then confirmed, in order to complete the request. For HPC Account Renewals, the page will look like this: -![Approver renewal](./static/approve_renewal.png) +![Approver renewal](../static/approve_renewal.png) Here, all systems should be selected by clicking the check box in the menu bar, and choosing “Select Everything”. -![Approve select everything](./static/approve_select_everything.png) +![Approve select everything](../static/approve_select_everything.png) Then, the “Select Bulk Action” menu is used to Approve or Reject all items selected. Please note that the line items may span multiple pages and all items must be acted upon in order to complete the request. Clicking “Complete” will complete the request. diff --git a/docs/hpc/01_getting_started/05_walkthrough_renew_hpc_account.md b/docs/hpc/01_getting_started/02_HPC_Accounts/04_walkthrough_renew_hpc_account.md similarity index 88% rename from docs/hpc/01_getting_started/05_walkthrough_renew_hpc_account.md rename to docs/hpc/01_getting_started/02_HPC_Accounts/04_walkthrough_renew_hpc_account.md index bcbccedd41..5f193a53a8 100644 --- a/docs/hpc/01_getting_started/05_walkthrough_renew_hpc_account.md +++ b/docs/hpc/01_getting_started/02_HPC_Accounts/04_walkthrough_renew_hpc_account.md @@ -10,22 +10,22 @@ https://identity.it.nyu.edu/ Upon logging in, an end user’s landing page will look like this -![Login screen](./static/renew_01.PNG) +![Login screen](../static/renew_01.PNG) If the menu does not appear, select the "burger" menu on the top left hand corner: -![View menu](./static/renew_02.PNG) +![View menu](../static/renew_02.PNG) The burger menu will show an "Update/Renew HPC Account" option - select this. -!["Update/Renew HPC Account" option](./static/renew_03.png) +!["Update/Renew HPC Account" option](../static/renew_03.png) Next complete the form as instructed. Please note that all accounts require the sponsorship of a full-time NYU faculty member. -![Select sponsor](./static/renew_04.png) +![Select sponsor](../static/renew_04.png) The user’s name will be pre-populated, and the form's required fields must be completed (sponsor, reason for request, consent to terms of use). After clicking “Submit” the chosen sponsor will be notified of the request and provisioning will only occur after approval. :::note If your HPC Account is due for renewal, you will get an update on your dashboard that will suggest that you fill out a form given in the "Latest form" widget for renewing your account: -![Banner messages](./static/renew_05.png) +![Banner messages](../static/renew_05.png) ::: If you are not a full-time NYU faculty member, you will need an NYU faculty member to sponsor your application. This is probably your thesis supervisor, or NYU collaborator. diff --git a/docs/hpc/01_getting_started/06_hpc_accounts_external_collaborators.md b/docs/hpc/01_getting_started/02_HPC_Accounts/05_hpc_accounts_external_collaborators.md similarity index 93% rename from docs/hpc/01_getting_started/06_hpc_accounts_external_collaborators.md rename to docs/hpc/01_getting_started/02_HPC_Accounts/05_hpc_accounts_external_collaborators.md index c9aad3d114..f1a9a7999e 100644 --- a/docs/hpc/01_getting_started/06_hpc_accounts_external_collaborators.md +++ b/docs/hpc/01_getting_started/02_HPC_Accounts/05_hpc_accounts_external_collaborators.md @@ -13,10 +13,10 @@ External (non-NYU) collaborators can access, with proper sponsorship, the NYU HP :::note Once the sponsoring faculty approves the account request, the HPC account is created within one hour. ::: -- Once the HPC account is created, the external collaborator can access HPC resources as described in our documentation page [Connecting to the HPC Cluster](../02_connecting_to_hpc/01_connecting_to_hpc.mdx). +- Once the HPC account is created, the external collaborator can access HPC resources as described in our documentation page [Connecting to the HPC Cluster](../../02_connecting_to_hpc/01_connecting_to_hpc.mdx). :::note -As with all sponsored accounts, HPC accounts for external collaborators are valid for a period of 12 months, at which point a renewal process is required to continue access to the NYU HPC environment. You can find information about renewing your account on our documentation page [Getting and Renewing an HPC Account](./02_getting_and_renewing_an_account.mdx). +As with all sponsored accounts, HPC accounts for external collaborators are valid for a period of 12 months, at which point a renewal process is required to continue access to the NYU HPC environment. You can find information about renewing your account on our documentation page [Getting and Renewing an HPC Account](../02_HPC_Accounts/01_getting_and_renewing_an_account.mdx). ::: [nyu vpn link]: https://www.nyu.edu/life/information-technology/infrastructure/network-services/vpn.html diff --git a/docs/hpc/01_getting_started/02_HPC_Accounts/_category_.json b/docs/hpc/01_getting_started/02_HPC_Accounts/_category_.json new file mode 100644 index 0000000000..24e1523513 --- /dev/null +++ b/docs/hpc/01_getting_started/02_HPC_Accounts/_category_.json @@ -0,0 +1,3 @@ +{ + "label": "HPC Accounts" +} diff --git a/docs/hpc/01_getting_started/03_Slurm_Accounts/01_intro_slurm_accounts.mdx b/docs/hpc/01_getting_started/03_Slurm_Accounts/01_intro_slurm_accounts.mdx new file mode 100644 index 0000000000..e0d7901a6e --- /dev/null +++ b/docs/hpc/01_getting_started/03_Slurm_Accounts/01_intro_slurm_accounts.mdx @@ -0,0 +1,3 @@ +# Slurm Accounts + +Users are required to have at least one active Slurm acount to submit jobs on the cluster. You will specify this with the `--account=` flag. Accounts are associated with an active allocation within a project that PIs manage using the [HPC Project Portal](02_hpc_project_management_portal.mdx). \ No newline at end of file diff --git a/docs/hpc/01_getting_started/07_hpc_project_management_portal.mdx b/docs/hpc/01_getting_started/03_Slurm_Accounts/02_hpc_project_management_portal.mdx similarity index 77% rename from docs/hpc/01_getting_started/07_hpc_project_management_portal.mdx rename to docs/hpc/01_getting_started/03_Slurm_Accounts/02_hpc_project_management_portal.mdx index b5711aa8e3..802b0907bd 100644 --- a/docs/hpc/01_getting_started/07_hpc_project_management_portal.mdx +++ b/docs/hpc/01_getting_started/03_Slurm_Accounts/02_hpc_project_management_portal.mdx @@ -6,7 +6,7 @@ import TabItem from '@theme/TabItem'; :::warning[Accessing the portal] - You need to be connected to [NYU VPN](https://www.nyu.edu/life/information-technology/infrastructure/network-services/vpn.html). -- You need to have an active [HPC account](./02_getting_and_renewing_an_account.mdx). +- You need to have an active [HPC account](../02_HPC_Accounts/02_walkthrough_request_hpc_account.mdx). ::: :::tip[HTTP 431 Errors] @@ -35,11 +35,9 @@ The HPC project and allocation management portal for Torch can be accessed by na How do I: -- [Create a project?](08_creating_hpc_projects.mdx) -- [Request an allocation?](09_requesting_an_allocation.mdx) -- [View resources that are available to me?](09_requesting_an_allocation.mdx) -- [Add publications and grants for your project](10_adding_publications_grants.mdx) -- [Approve an allocation request?](11_approving_an_allocation_request.mdx) - +- [Create a project?](03_creating_hpc_projects.mdx) +- [Request an allocation?](04_requesting_an_allocation.mdx) +- [Approve an allocation request?](05_approving_an_allocation_request.mdx) +- [View resources that are available to me?](04_requesting_an_allocation.mdx) An active allocation in the HPC projects portal is needed to perform any work on Torch. The next sections describe the project and allocation request process in detail. diff --git a/docs/hpc/01_getting_started/08_creating_hpc_projects.mdx b/docs/hpc/01_getting_started/03_Slurm_Accounts/03_creating_hpc_projects.mdx similarity index 88% rename from docs/hpc/01_getting_started/08_creating_hpc_projects.mdx rename to docs/hpc/01_getting_started/03_Slurm_Accounts/03_creating_hpc_projects.mdx index 04531f535f..f2a43fb4e7 100644 --- a/docs/hpc/01_getting_started/08_creating_hpc_projects.mdx +++ b/docs/hpc/01_getting_started/03_Slurm_Accounts/03_creating_hpc_projects.mdx @@ -5,15 +5,15 @@ You need to be connected to [NYU VPN](https://www.nyu.edu/life/information-techn ::: Login to HPC projects portal at [projects.hpc.nyu.edu](https://projects.hpc.nyu.edu) on the NYU VPN. Navigate to the "Project" item in the navigation bar on the top of the page and select "Projects" from the drop down menu.
-![Project Dropdown](./static/project_dropdown.png) +![Project Dropdown](../static/project_dropdown.png)
If you are a PI, you will be able to see a "+ Add a Project" button that will let you create a new project.
-![Project Add Button](./static/project_add_button.png) +![Project Add Button](../static/project_add_button.png) Click on it to head to the project creation page where you can add the project title, description and the school it will be associated with. The school a project is associated with determines the resources it will have access to. The school approver for the associated school has the capacity to approve allocation requests for resources associated with the school.
-![Project Create](./static/project_create.png) +![Project Create](../static/project_create.png) :::warning[No `SLURM` account yet] `SLURM` accounts are created for "Allocations" not projects, so creating a project does not create `SLURM` account. Please head to the next section to learn about how you can submit an allocation request. diff --git a/docs/hpc/01_getting_started/09_requesting_an_allocation.mdx b/docs/hpc/01_getting_started/03_Slurm_Accounts/04_requesting_an_allocation.mdx similarity index 86% rename from docs/hpc/01_getting_started/09_requesting_an_allocation.mdx rename to docs/hpc/01_getting_started/03_Slurm_Accounts/04_requesting_an_allocation.mdx index 51041aaf93..cd6da4ffc4 100644 --- a/docs/hpc/01_getting_started/09_requesting_an_allocation.mdx +++ b/docs/hpc/01_getting_started/03_Slurm_Accounts/04_requesting_an_allocation.mdx @@ -8,17 +8,17 @@ You need to be connected to [NYU VPN](https://www.nyu.edu/life/information-techn ::: Login to HPC projects portal at [projects.hpc.nyu.edu](https://projects.hpc.nyu.edu) on the NYU VPN. Navigate to the "Project" item in the navigation bar on the top of the page and select "Projects" from the drop down menu.
-![Project Dropdown](./static/project_dropdown.png) +![Project Dropdown](../static/project_dropdown.png) From the list of project, click on the project you'd like to submit an allocation request for and you'll see the current details: -!["PI_One Project" section](./static/PI_one_project.png) +!["PI_One Project" section](../static/PI_one_project.png) ## Current allocations Scroll down and you'll reach the Allocations section. This section lists all the allocations associated with this project. All allocations have SLURM accounts associated with them. Allocations that are "Active" will allow you to submit jobs using the SLURM account associated with it. ## Requesting new allocations If you scroll down, you'll see "+Request Resource Allocation" button. -!["PI request allocation" section](./static/PI_request_allocation.png) +!["PI request allocation" section](../static/PI_request_allocation.png) After clicking "+Request Resource Allocation", you'll see a list of resources you can request. All projects can request access to Torch, which provides access to the cluster. However, depending on the school the project is affiliated with, high priority resources may be available. @@ -45,14 +45,14 @@ After clicking "+Request Resource Allocation", you'll see a list of resources yo ::: Please select a resource and fill in justification to complete the allocation request process. -![Allocation request justify submit](./static/allocation_request_submit.png) +![Allocation request justify submit](../static/allocation_request_submit.png) Now your allocation request is created! You'll see an allocation request with "New" status. -!["Allocation request created" section](./static/allocation_request_created.png) +!["Allocation request created" section](../static/allocation_request_created.png) Once your school approver approves the request, the status will change to "Active" (and you will receive an email notifying you of the change). An active allocation now allows you to submit jobs on Torch. ## `SLURM` Account Association Within the "Allocations" section in the project, you'll see the `slurm_account_name` listed as shown below: -!["SLURM account association"](./static/allocation_slurm_account_name.png) +!["SLURM account association"](../static/allocation_slurm_account_name.png) This account name needs to be specified via the `--account` flag to `srun` and `sbatch`. diff --git a/docs/hpc/01_getting_started/11_approving_an_allocation_request.mdx b/docs/hpc/01_getting_started/03_Slurm_Accounts/05_approving_an_allocation_request.mdx similarity index 75% rename from docs/hpc/01_getting_started/11_approving_an_allocation_request.mdx rename to docs/hpc/01_getting_started/03_Slurm_Accounts/05_approving_an_allocation_request.mdx index ca70c1d721..6915ed99ac 100644 --- a/docs/hpc/01_getting_started/11_approving_an_allocation_request.mdx +++ b/docs/hpc/01_getting_started/03_Slurm_Accounts/05_approving_an_allocation_request.mdx @@ -6,22 +6,22 @@ You need to be connected to [NYU VPN](https://www.nyu.edu/life/information-techn If you are an approver for your school, first login to the HPC projects portal page and you'll see "staff" in the navigation bar. Click the "staff" navigation bar and click "Allocation Requests" to see a list of requests waiting for your approval. -!["Staff Allocation Requests" section](./static/approver_staff.png) +!["Staff Allocation Requests" section](../static/approver_staff.png) Now, you'll see a list of allocation requests. Click "Details" button for the request that you would like to approve. You should click "Details" instead of just "Approve" button if you'd like to set more details. -!["Allocation Requests Details" section](./static/allocation_requests.png) +!["Allocation Requests Details" section](../static/allocation_requests.png) After clicking "Details", you'll see Allocation information part to fill in with your approval. Here, you could set start date, end date, description, etc. If you're done with adding the information, you could click "Update" button to update the allocation request with the information you put. -!["Approving Allocation Requests via Details" section](./static/approving_allocation.png) +!["Approving Allocation Requests via Details" section](../static/approving_allocation.png) After clicking "Update", the allocation request is approved, and the allocation is activated now! -!["Approved Allocation Requests via Details" section](./static/allocation_approved.png) +!["Approved Allocation Requests via Details" section](../static/allocation_approved.png) The `SLURM` account name is also available on the allocation attributes if you need it. -!["Approved Allocation Requests via Details" section](./static/slurm_account_name.png) +!["Approved Allocation Requests via Details" section](../static/slurm_account_name.png) If you have any difficulties or questions, please contact us at hpc@nyu.edu. diff --git a/docs/hpc/01_getting_started/10_adding_publications_grants.mdx b/docs/hpc/01_getting_started/03_Slurm_Accounts/06_adding_publications_grants.mdx similarity index 90% rename from docs/hpc/01_getting_started/10_adding_publications_grants.mdx rename to docs/hpc/01_getting_started/03_Slurm_Accounts/06_adding_publications_grants.mdx index 20164bb0b9..6c0e2006e9 100644 --- a/docs/hpc/01_getting_started/10_adding_publications_grants.mdx +++ b/docs/hpc/01_getting_started/03_Slurm_Accounts/06_adding_publications_grants.mdx @@ -13,11 +13,11 @@ Publications can be added to a project by the PI or manager. Publications can b ### Adding Publications Using DOI Search You can find the 'Add Publication' button on the project page: -![Add Publication button](./static/addpub.png) +![Add Publication button](../static/addpub.png) Enter the DOI number(s) in the search box and click the Search button. Multiple DOIs should be entered on separate lines. -![Add Publication with DOI](./static/doisearch.png) +![Add Publication with DOI](../static/doisearch.png) To add the publication found, click the checkbox and then the "Add Selected Publications to Project" button. @@ -25,7 +25,7 @@ To add the publication found, click the checkbox and then the "Add Selected Publ After clicking the 'Add Publication' button on the Project Detail page, click the 'Enter publication manually' The PI/manager can then enter basic details about the publication: title, author, year, and journal name. These fields are all required. Once complete, click the 'Add Publication' button to add to the project. -![Add Publication Manually](./static/manualpub.png) +![Add Publication Manually](../static/manualpub.png) ## Enter Grants @@ -36,7 +36,7 @@ Grant data is useful to Center Directors for making the case for enabling scienc You can find the 'Add Grant' button on the project page: -![Add Grant Button](./static/addgrant.png) +![Add Grant Button](../static/addgrant.png) PIs or managers must supply the fields marked with an asterisk: -![Add Grant Page](./static/manualgrant.png) +![Add Grant Page](../static/manualgrant.png) diff --git a/docs/hpc/01_getting_started/03_Slurm_Accounts/_category_.json b/docs/hpc/01_getting_started/03_Slurm_Accounts/_category_.json new file mode 100644 index 0000000000..09337b52aa --- /dev/null +++ b/docs/hpc/01_getting_started/03_Slurm_Accounts/_category_.json @@ -0,0 +1,3 @@ +{ + "label": "Slurm Accounts" +} diff --git a/docs/hpc/03_storage/04_research_project_space.mdx b/docs/hpc/03_storage/04_research_project_space.mdx index 38a4cd6977..d32c39dbb5 100644 --- a/docs/hpc/03_storage/04_research_project_space.mdx +++ b/docs/hpc/03_storage/04_research_project_space.mdx @@ -44,7 +44,7 @@ Determine the total annual cost of the request and the contact info of the Schoo ### Step 3: Verify that the project PI has a valid HPC user account -The PI administers the top level RTS directory and grants access to other users. Thus ***the PI must have a valid HPC user account*** at the time of request. Please note that the HPC user account of NYU faculty never expires and thus does not need to be renewed every year. If the PI does not have an HPC account, please [request one here](../01_getting_started/02_getting_and_renewing_an_account.mdx). +The PI administers the top level RTS directory and grants access to other users. Thus ***the PI must have a valid HPC user account*** at the time of request. Please note that the HPC user account of NYU faculty never expires and thus does not need to be renewed every year. If the PI does not have an HPC account, please [request one here](../01_getting_started/02_HPC_Accounts/01_getting_and_renewing_an_account.mdx). ### Step 4: The PI submits the request to the HPC team via email diff --git a/docs/hpc/03_storage/05_best_practices.mdx b/docs/hpc/03_storage/05_best_practices.mdx index e024cb8d86..c166153c6a 100644 --- a/docs/hpc/03_storage/05_best_practices.mdx +++ b/docs/hpc/03_storage/05_best_practices.mdx @@ -6,21 +6,8 @@ All users have quote limits set on HPC fie systems. There are several types of q _One of the common issues users report is running out of inodes in their home directory._ This usually occurs during software installation, for example installing conda environment under their home directory. Running out of quota causes a variety of issues such as running user jobs being interrupted or users being unable to finish the installation of packages under their home directory. ::: -Users can check their current utilization of quota using the myquota command. The myquota command provides a report of the current quota limits on mounted file systems, the user's quota utilization, as well as the percentage of quota utilization. +Users can check their current utilization of quota using the myquota command. The myquota command provides a report of the current quota limits on mounted file systems, the user's quota utilization, as well as the percentage of quota utilization. For details, please see [myquota](../06_tools_and_software/08_utils.mdx#myquota) -In the following example the user who executes the `myquota` command is out of inodes in their home directory. The user inode quota limit on the `/home` file system **30.0K inodes** and the user has **33000 inodes**, thus **110%** of the inode quota limit. -```sh -$ myquota -Quota Information for NetID -Hostname: torch-login-2 at 2025-12-09 17:18:24 - -Filesystem Environment Backed up? Allocation Current Usage -Space Variable /Flushed? Space / Files Space(%) / Files(%) - -/home $HOME YES/NO 0.05TB/0.03M 0.0TB(0.0%)/54(0%) -/scratch $SCRATCH NO/YES 5.0TB/5.0M 0.0TB(0.0%)/1(0%) -/archive $ARCHIVE YES/NO 2.0TB/0.02M 0.0TB(0.0%)/1(0%) -``` You can use the following command to print the list of files within each sub-folder for a given directory: ```sh $cd $HOME diff --git a/docs/hpc/05_submitting_jobs/01_slurm_submitting_jobs.md b/docs/hpc/05_submitting_jobs/01_slurm_submitting_jobs.md index 29a0cfc546..be48061b1f 100644 --- a/docs/hpc/05_submitting_jobs/01_slurm_submitting_jobs.md +++ b/docs/hpc/05_submitting_jobs/01_slurm_submitting_jobs.md @@ -5,7 +5,7 @@ If you are new to using HPC resources and would like to learn about the principl ::: :::warning Active allocation in the HPC projects portal -An active allocation in the HPC projects portal is needed to submit any jobs on Torch. For more information on how to get one, please refer to [this section](../01_getting_started/07_hpc_project_management_portal.mdx). All job submissions must include the `--account` parameter. On Torch, you can list the SLURM accounts you have access to by running the command `my_slurm_accounts`. +An active allocation in the HPC projects portal is needed to submit any jobs on Torch. For more information on how to get one, please refer to [this section](../../hpc/01_getting_started/03_Slurm_Accounts/02_hpc_project_management_portal.mdx). All job submissions must include the `--account` parameter. On Torch, you can list the SLURM accounts you have access to by running the command `my_slurm_accounts`. ::: ## Partitions diff --git a/docs/hpc/06_tools_and_software/08_utils.mdx b/docs/hpc/06_tools_and_software/08_utils.mdx new file mode 100644 index 0000000000..4a6b479f8d --- /dev/null +++ b/docs/hpc/06_tools_and_software/08_utils.mdx @@ -0,0 +1,64 @@ +# Torch Utility Applications + +Torch has several utility applications that can give you information related to your account and jobs on the cluster: + +## myquota + +Users can check their current utilization of quota using the myquota command. The myquota command provides a report of the current quota limits on mounted file systems, the user's quota utilization, as well as the percentage of quota utilization. + +In the following example the user who executes the myquota command is out of inodes in their home directory. The user inode quota limit on the `/home` file system **30.0K inodes** and the user has **33000 inodes**, thus **110%** of the inode quota limit. +```sh +$ myquota +Quota Information for NetID +Hostname: torch-login-2 at 2025-12-09 17:18:24 + +Filesystem Environment Backed up? Allocation Current Usage +Space Variable /Flushed? Space / Files Space(%) / Files(%) + +/home $HOME YES/NO 0.05TB/0.03M 0.0TB(0.0%)/33000(110%) +/scratch $SCRATCH NO/YES 5.0TB/5.0M 0.0TB(0.0%)/1(0%) +/archive $ARCHIVE YES/NO 2.0TB/0.02M 0.0TB(0.0%)/1(0%) +``` + +## my_slurm_accounts + +my_slurm_accounts returns a list of slurm accounts associated with your HPC account. For more information about slurm accounts please see [Slurm Accounts](../01_getting_started/03_Slurm_Accounts/01_intro_slurm_accounts.mdx) + +## nvidia-smi +nvidia-smi (NVIDIA System Management Interface) is a command-line utility, based on the NVIDIA Management Library (NVML), used to monitor and manage NVIDIA GPU devices + +It will provide detailed information like: +- GPU utilization +- Memory usage +- P-States: Performance states from P0 (max performance) to P12 (minimum idle) +- device details like power consumption and temperature + +:::tip +To monitor running jobs, use: +```bash +[NetID@gl001 ~]$ watch -n 1 nvidia-smi +``` +on the compute node +::: + +:::tip +You can get very detailed information about the GPU with: +```bash +[NetID@gl001 ~]$ nvidia-smi -q +``` +::: + +## seff + +The seff script can be used to display status information about a user’s historical or running jobs. + +It shows information about: +- CPU efficiency +- memory efficiency + + +{/* coming soon +## rpsquota + +rpsquota returns detailed information about how much of your Research Project Space quota you're using (if you have RPS space). To use it just type `rpsquota` in the base RPS directory or give the base RPS path as an input argument like `rpsquota /path/to/rps/space`. +*/}