Process overview and key aspects

All proposals submitted to the Main Call that request less than 50 h on the 12-m Array or less than 150 h on the 7-m Array in standalone mode will be peer reviewed using a distributed system, in which a designee from each proposal team participates as a reviewer in the review process. The key aspects of the process are as follows.

Ethical Conduct in the Review Process

1. All participants in the review process are required to follow the Code of Conduct and behave in an ethical manner. If a reviewer fails to uphold ethical standards or does not complete their reviews in good faith, the proposal(s) for which they are the designated reviewer may be disqualified.

Designating Reviewers

2. Each proposal must designate one reviewer, who can either be the Principal Investigator (PI) or one of the co-Investigators (co-Is).
3. Reviewers may review up to THREE Proposal Sets. Therefore, PIs submitting multiple proposals are encouraged to designate a co-I as the reviewer. While the Observing Tool (OT) does not enforce this limit, the Proposal Handling Team (PHT) will verify compliance after the proposal submission If a reviewer exceeds the limit, the PHT will contact them to select alternative reviewers from the list of co-Is. Failure to designate replacements by 29 April 2025, 15:00 UTC will result in therejection of proposals with the highest proposal code until the allowed limit of Proposal Sets to review is met.
4. The reviewer must be specified in the OT at the time of proposal submission, with changes permitted only under exceptional circumstances (e.g., medical emergencies or urgent family obligations). PIs may request a change by submitting a ticket to the "Proposal Review Support" department through the ALMA helpdesk. Once approved, the replacement reviewer will assume responsibility for the assigned proposal set, and the Stage 1 deadline will remain unchanged.
5. PIs without a PhD may serve as reviewers if they designate a mentor holding a PhD in astronomy or a related field. Mentors must be specified in the OT at the time of submission, are not required to be part of the proposal team and will have read-only access to assignments and reviews. Co-Is without a PhD are not eligible to serve as reviewers.

Proposal Assignments

6. Proposals will be assigned to reviewers based on the expertise provided in their user preferences, which must be updated by 29 April 2025, 15:00 UTC. If no expertise is registered, assignments will default to the scientific topics of the reviewer's submitted proposals. More details on how reviewer expertise is used can be found here.
7. Reviewers may submit a list of investigators with whom they have a conflict of interest, based on the conflict criteria. The list must be submitted by 29 April 2025, 15:00 UTC. Reviewers who submit this list will not be assigned proposals in which any investigator of their list is the PI, a co-PI, or a co-I. Since automated checks may not identify all conflicts, reviewers must declare any major conflicts of interest with their assigned proposals by 14 May 2025, 15:00 UTC, and any affected proposals will be reassigned.

Review Process

8. Stage 1:
   Reviewers must:
   1. Rank the proposals in their Proposal Set by scientific priority, from 1 (highest) to 10 (lowest).
   2. Write a review for each proposal.

 

Participation in Stage 1 is mandatory. Failure to submit reviews and rankings by the deadline (04June 2025, 15:00 UTC) will result in the disqualification of the proposal(s) for which the reviewer is responsible.

9. Stage 2:
   Reviewers will gain access to the anonymized Stage 1 reviews submitted by other reviewers for their assigned proposals. Based on this feedback, reviewers can revise their reviews and adjust their rankings.
10. Reviews and rankings will be sent anonymously to PIs without any editing by the PHT. If reviewers participate in Stage 2, their edited reviews and rankings will be Otherwise, Stage 1 reviews and rankings will be used.

Communication

11. The PHT will communicate with reviewers via the email associated with their ALMA user account. Reviewers should ensure their user preferences are up to date to avoid missing important
12. Reviewers can contact the PHT through the ALMA helpdesk by opening a ticket to the "Proposal Review Support" department.

Proposal review assignments

After the proposal deadline, the PHT will assign ten proposals to the designated reviewer of each submitted proposal. The PHT will attempt to assign proposals within the area of expertise of the reviewer. The expertise will be assessed based on the reviewer’s proposal history using machine learning algorithms and the expertise specified in their user preferences accessible from the ALMA Science Portal. If the reviewer does not specify their expertise, they will be assumed to be an expert in the scientific topic of the submitted proposal.

The PHT cannot guarantee that all proposal assignments will match the reviewer's expertise. Reviewers may receive proposals in different topics within the scientific category of their expertise, or even “similar” categories; e.g., a reviewer who is an expert in Category 1 (Cosmology and the high redshift universe) may receive proposals from Category 2 (Galaxies and galactic nuclei) and vice versa. The PHT will avoid assigning proposals far outside the reviewer’s expertise; e.g. the PHT will not assign a stellar evolution proposal to a cosmology expert unless the reviewer indicates their expertise is in both categories.

If a reviewer receives a proposal that does not match their expertise, they are nonetheless asked to review it. PIs are instructed to write their proposals for a non-expert. Reviewers will also have the opportunity to view comments from the other reviewers in Stage 2.

Declaring conflicts of interest

Proposals are assigned to reviewers to avoid major conflicts of interest. Because of the dual anonymous review process, reviewers can provide a list of investigators with whom they have a conflict of interest through their user preferences on the ALMA Science Portal. If no major conflicts of interest are specified in the user preferences, then an algorithm automatically identifies potential conflicts based on the PIs and co-Is of proposals in which the reviewer has participated. Detailed guidelines on the procedure and the criteria for defining a conflict of interest are available here.

Stage 1

The PHT will assign ten proposals per each submitted proposal; this set of ten proposals is called a “Proposal Set”. During Stage 1, reviewers will rank the proposals in their Proposal Set relative to each other in order of scientific priority (from 1 [strongest] to 10 [weakest]) and will write a review for each one of them. Reviewers access their Proposal Set through the Reviewer Tool. Usage of this tool is explained in the User's Manual. All reviewers should refer to that documentation for a detailed description of the tool's functionalities; here a broad outline of the main actions is provided.

At Stage1, all reviewers should carry out the following sequence of steps:

1. Access the Reviewer Tool.
2. Browse through the abstract of the proposals in their Proposal Set and declare in the Reviewer Tool any possible conflicts of interest that the assignment algorithm did not identify. Reviewers should note that the proposal PDF files are not available until conflicts of interest are submitted.
3. Submit the conflict decisions before starting to review the proposals. Reviewers should submit their decisions by 14 May 2025, 15:00 UTC, when the PHT will review and replace any conflicting proposals with other assignments when the conflict is confirmed. If a person is the designated reviewer on more than one proposal (i.e., they have more than one Proposal Set), they will need to identify conflicts of interest on all their Proposal Sets before being able to submit them. Even if no conflicts are identified, the reviewer should submit the decisions by the deadline. Conflict decisions submitted after the mentioned date will have lower chances of receiving replacement proposals that are aligned with the reviewer's expertise
4. Download the proposal package. For every Proposal Set, the corresponding proposal package will contain the PDF files of the proposals that were declared free of conflict.
5. Review the proposals to provide a rank from 1 (strongest) to 10 (weakest) and write a review summarizing the proposal's strengths and weaknesses.
6. Submit the ranks and reviews. Stage 1 is mandatory. If a reviewer does not submit the Stage 1 ranks and reviews by the Stage 1 deadline (04 June 2025, 15:00 UTC), then the proposal for which they were identified as the designated reviewer will be disqualified. If a person is acting as designated reviewer on more than one proposal, they will be able to submit the reviews for each Proposal Set independently.

Stage 2

In Stage 2, reviewers read anonymized reviews from the other nine reviewers assigned to the same proposal. This process helps reviewers to identify critical strengths or weaknesses they may have overlooked in their initial review. Based on these insights, reviewers can revise their initial rankings and reviews. If a reviewer begins Stage 2 but does not submit their revised rankings and reviews by the Stage 2 deadline, their Stage 1 submissions will be considered final. Note that the purpose of Stage 2 is not to facilitate discussion among reviewers, but to allow each reviewer to independently address any overlooked aspects of the proposal.

Stage 2 consists of the following steps:

1. Access the Reviewer Tool.
2. For each proposal, read the anonymized reviews from the other nine reviewers, following the instructions in Section 7 of “How to use the Reviewer Tool” guide.
3. Edit the Stage 1 reviews as needed, and/or re-rank the proposals in their Proposal Set.
4. Submit the updated reviews and ranks. The deadline to complete the Stage 2 review is 19 June 2025, 15:00 UTC. Stage 2 is optional, and no proposals will be disqualified if the designated reviewer does not submit Stage 2 ranks and reviews by the deadline.

Providing feedback to the PHT

The PHT continuously seeks to enhance the proposal review process. To gather valuable insights, two surveys have been introduced to collect feedback from the reviewers on different aspects of the review process.

Reviewer Expertise Survey

The purpose of this survey is to gather information about the reviewers’ expertise level on their proposal assignments. The collected data will help assess the effectiveness of the assignment process and identify areas where it can be improved.

This survey is available in the Reviewer Tool. Reviewers can access it after submitting the Stage 1 ranks and reviews. Reviewers with multiple Proposal Sets will complete a separate survey for each Proposal Set.

Feedback on Review Comments

The purpose of this survey is to evaluate the effectiveness of review comments in providing constructive feedback on the strengths and weaknesses of proposals. Reviewers will be presented with a random selection of 10 reviews for proposals in their Proposal Set and asked to rate the quality of each comment.

This survey will be available during Stage 2, outside the Reviewer Tool, via this <link>. Reviewers must use their ALMA credentials to access the survey. Participation is voluntary and open to all reviewers, regardless of whether they are currently participating or have already completed Stage 2.

Creating the ranked list of proposals

After the completion of Stage 2, the ranks of the individual reviewers are combined to create an overall ranked list of proposals. To mitigate the impact of potential outliers, the highest and lowest ranks for each proposal are removed. The remaining ranks are then averaged, and the resulting averages are normalized based on the number of proposals within each scientific category. Finally, the normalized average ranks for all proposals are sorted to produce the overall ranked list.

Frequently asked questions

Answers to frequently asked questions are available on the distributed peer review FAQ page.

---

Return to the main ALMA Proposal Review page