PRECISE-AI Frequently Asked Questions

TA2 and TA3 budgets can contain a  component to help support site engagement, including for DST deployment, however this activity is primarily the responsibility of TA1. 

Combining TA1, TA2, and TA3 is allowed and, in some circumstances, potentially advantageous for the program if it allows for efficiencies to be realized. Since adding a new TA would involve additional work, you can consider a larger budget if adding a TA.

January 15th is the confirmed submission date. Change in the submission date is possible but unlikely. Please check the PRECISE-AI program site for updates.

There is no specific threshold required, and we encourage you to use your best judgment. 

A Federal Wide Assurance is required for any entity engaged in HSR.  You can provide your FWA number, a copy of the assurance or  a screen shot of the OHRP website that shows that you have an active FWA.  

Volume 3 (Administrative & National Policy Requirements) has a section that asks whether the  proposed work involves Human Subject Research, and if so, asks the proposer to provide evidence of or a plan for review by an institutional review board (IRB).

Yes, you can follow the NIH limit of 5 pages for the full proposal. Please note that a limit of 5 pages per resume will be reflected in the final ISO.

Yes please provide narratives and detailed budgets for subaward activities. 

There is no formal limit on how many key personnel should be listed on your full proposal. This is in contrast to the solution summary, where a limit of five was required. Although there is no limit for the full proposal, we encourage you to use judgment.

Please note that the ISO states that "Proposals that address TA4 cannot address any of the other TAs." However, an organization can be on multiple proposals, even if one of the proposals addresses TA4.

The "Milestones and Deliverables" tab requires pricing according to Task and Subtask. The cells listed in Column H "Milestone/Deliverables" should only be filled out if the Milestones and/or deliverables are identical to the Task/Subtask listed.

As shown on page 26 of the ISO, "Proposers must submit a single proposal ... that addresses program Phase I and Phase II relevant to the proposed TA(s). To minimize risk and facilitate effective integration of TAs into a combined tool suite, we anticipate that progression to Phase III will involve a down-select and will require performers to re-propose as self-selected teams that collectively address TAs 1-3."

Unlike the Solution Summary, the proposals must be budgeted for 3 years, Phases I and II only.

Scalability and generalizability are very important for PRECISE-AI. Therefore, we are looking for performers to work with two distinct clinical use cases, validating EACH use case in three clinical sites in Phase 1. In Phase 2, proposers will need to expand to two ADDITIONAL use cases, and five clinical sites each. 

There is no specific template for the budget narrative/justification.

Assume a start date of July 1, 2025.

As of now, the final proposals are due on January 15th, 2025. Please refer to the program page and the final solicitation.

You can use the standard forms for both Volume III and Volume I.

While many meetings may be remote, some may be in person, as needed, at a location that helps achieve the purpose of the meeting. For budgeting travel, it's reasonable to assume roughly two, two-day trips per year to DC. 

You can be subcontractor on a full proposal that has an associated solution summary. However, full proposals from prime organizations that did not submit a solution summary will not be accepted.

As stated in the ISO, proposals will primarily leverage TA4-furnished cloud computing resources. Additionally, ARPA-H and TA4 will work together will work together to provide cloud resource early in the program.

As summarized in the "Program Timeline and Milestones" table in the ISO, TA1 performers need to operationalize their software for 2 use cases and 3 clinical sites in Phase 1, and increase the number of use cases to 4 in phase 2. For a team proposing to both TA1 and TA2, this can be satisfied by using your own use cases. If a different performer has the same use case, it would strengthen the program for you to share your surrogate ground truth with the other performer. Likewise, if the use cases from two performers are similar, it would strengthen the program for the performers to share, early in the program, their software for surrogate ground truth extraction.

Distinct outpatient clinics in the same hospital system would be considered as distinct sites for the purposes of meeting the site requirements.

Rates should be proposed by labor category. These categories should be proposed by the entities submitting; they are not being dictated by ARPA-H. Please propose only fully-burdened labor rates.

As described in the ISO, TA1 proposals will "Identify the clinical sites required in Phase I and provide evidence of their willingness to partner throughout the program. Proposers are encouraged to identify more clinical sites than required in Phase I."

For the full proposal, you are permitted to adjust your initial  choice of AI-DST from your solutions summary.

Phase 1 of TA1-TA4 is considered basic research. Phase 2 of TA1-TA4 is considered applied research. The appropriate exemptions will apply.

At present, there are no plans to change the deadline for full proposals.

Any technology that addresses the scope and desired outcomes of the program is welcomed. However please note that efforts that aim to achieve goals outside of the program scope are strongly discouraged.

The deadline for Solution Summaries has already passed, and new Solution Summaries are no longer accepted. 

The U.S. Department of Veterans Affairs can be included as a clinical site as long as they are not 1) acting as a performer/subperformer and 2) receiving any payment utilizing funds obligated on the award.

Reporting requirements will be addressed during post-award contract negotiations. Please feel free to include any questions on the award language in your submission.

The stated goal refers to measurement of improvements on the algorithmic level.

Proposer should define "degradation" in a way that makes sense for the specific use case. Likewise, the time window will likely depend on the use case because it may take different amounts of time to establish the surrogate ground truth for different use cases. For each use case, the metric(s) to define a degradation and the time window will be defined by TA2 performer. 

Time-to-correct alarm is the difference in time from time of occurrence of the data shift leading to performance degradation, to the time of its detection.

The template for the Solution Summary is now available on the program page: https://arpa-h.gov/research-and-funding/programs/precise-ai . Please click on the link for draft solicitation and then on the link that states "Note: There have been new actions to this contract opportunity. To view the most recent action, please click here."

You should choose one clinical track as described in the solicitation. Please note that if you choose the ""other track"" in Figure 4, your proposal must address both TA1 and TA2, or neither. Proposals that address TA4 or TA5 cannot address any of the other TAs. Please see the below message for more details. Each proposal responsive to this Solicitation can address a TA among TA1, TA2, TA3, TA4 and TA5, or a combination of TA1, TA2, and TA3. Proposals that address TA4 or TA5 cannot address any of the other TAs. An organization may act as a primary performer on one TA and a sub-performer on a separate proposal addressing the same TA. An organization may act as a sub-performer on multiple proposals addressing the same TA. If a proposal selects “other” for the clinical track, it must either address both TA1 and TA2, or neither. An organization or individual who is awarded for TA5 will not be awarded for any other TAs. An individual may submit no more than one proposal as PI. The same rules apply to summary solutions and full proposals.

The deadlines will be posted in the final solicitation.

The answer to both questions is yes.

Please note that the program is focused in AI-enabled decision support tools (AI-DSTs), where AI-DSTs are defined as (Section 1.3) "decision support tools that are used in a context where they provide recommendations for clinicians, and the clinicians are the final decision-makers." Within this context, you can choose any use case in the "other" category. Yes, to generate the surrogate ground truth for a radiographic algorithm, computational pathology can be an alternative to extracting the pathology results from the EHR.

In-person attendance is limited by the venue capacity and early registration is strongly recommended. In-person registration may close prior to the registration deadline due to the venue capacity being met. Registration for the virtual event is unlimited. The event is closed to the general public and media.

Yes there is a remote option for proposers day. Please see program page which linked to all relevant information: https://arpa-h.gov/research-and-funding/programs/precise-ai

The draft agenda is in the special notice, which is linked on the program page: https://arpa-h.gov/research-and-funding/programs/precise-ai. In the special notice (ARPA-H-SN-25-116.pdf), please see page 3. All registered attendees will receive a final agenda prior to the event kick-off. Please note that (1) Lunch will not be included; (2) There is no formal networking event the night before. (3) If you plan to bring a poster, the poster information is also in the special notice.

You may team as you wish, and we encourage teaming when it helps bring together expertise from different areas. Please that no one may be PI on more than one proposal.

Yes, the program is interested in the use of AI-DSTs in all settings, including outpatient settings and other types of clinics in addition to hospital settings.

The program emphasizes the collection of real data, as well as simulation studies to develop a methodology for the detection of AI model degradation and mitigation strategies. Please plan to select your AI-DST use case and the approaches that you will pursue in a way that optimizes your resources.

The program focuses on the potential effectiveness and innovation of the proposed methodologies and technologies, regardless of FDA approval/clearance of the AI-DST.

Automated analysis and detection of performance problems when the model is applied to specific subpopulations is within the scope of the PRECISE-AI program, even if there is no change over time in the frequency of that subpopulation. The focus of PRECISE-AI includes degradation detection, root cause analysis, and self-correction, which inherently involves monitoring and maintaining the performance of AI models across diverse clinical settings and subpopulations. This objective is emphasized within Technical Area 2 (TA2), specifically under Continuous Degradation Detection Tools (TA2.1) and AI-Based Root-Cause Analysis Tools (TA2.2), where there is an emphasis on distinguishing performance issues across different demographics. Addressing variations in model performance among subpopulations aligns with the program's goal of enhancing AI decision support tools' reliability and effectiveness in diverse clinical settings.

If there is a difference in performance among clinical sites due to differences in human errors in data entry among those sites, then this is in the scope of the program.

Yes, the same entity can submit multiple proposals, but an individual may submit no more than one proposal as PI. If a proposal selects “other” for the clinical track, it must either address both TA1 and TA2, or neither.

Timeliness of detection and correction of degradation in AI-DSTs is a consideration. However, this detection and correction of degradation is not formally required to happen in real time. For TA3, it is expected that effective communication with the clinical staff will require near real time communication.

The same institution can submit multiple proposals, but an individual may submit no more than one proposal as PI.  Proposers must submit a single proposal led by one PI under a single prime performer. There is no limit on the number of co-PIs. A proposal may target multiple TAs. Note if a proposal selects “other” for the clinical track, it must either address both TA1 and TA2, or neither.

The same institution can submit multiple proposals, but an individual may submit no more than one proposal as PI.

Yes--the intention is to incorporate live clinical data, and that is the reason why TA1 performers will collaborate with hospitals or other clinical care sites for data collection. However, existing data could be leveraged as well, for training, testing, or any other purpose.

There is no requirement for the structure and assignment of  co-PIs for a single proposal, but as described in the solicitation, proposers must submit a single proposal led by one PI under a single prime performer. Note that no individual may be PI on more than one proposal.

Yes the anticipated due date for solution summaries is November 7th. Proposers' day slides are now available on the PRECISE-AI program page.

Please follow the instructions on Section 4 (Team Organization and Capabilities) of the solution summary template.

Yes, when the solicitation is updated, it will be updated on the same sam.gov link as the draft solicitation.

Vendors can submit a proposal as a prime (as the main PI) with site involvement.

As stated in the ISO, it is desired that all non-commercial software generated by the program be provided as deliverables to the Government with open-source or unlimited rights. However, there is room to keep parts of the solutions as private intellectual property. In particular, if the performer uses software components that were not generated by the program, other alternatives will be considered.

An organization can be involved in multiple submissions, but note that an individual may not be PI on multiple submissions. An organization or individual who is awarded for TA5 will not be awarded for any other TAs. 

Although we would prefer that the 2 additional uses cases for Phase 2 be defined at the time of submission for the Solution Summary or for the Full Proposal, this is not a requirement.

The recording and slides will be posted on the PRECISE-AI program page.

A partial update may be an update that modifies the AI-DST model when certain conditions are met, but keeps the AI-DST model unchanged when those conditions are not met. As described in the program solicitation, for AI-enabled models marketed for clinical use, an update to the clinically deployed model can only be permissible if the update is included as part of an FDA-cleared pre-determined change control plan (PCCP) and the conditions in the PCCP are satisfied.

Yes dermatology applications can be a clinical track in the "other" category. As discussed in the solicitation document, it is preferred to  limit the time horizon for data elements which are  used to define the surrogate ground truth, so that model performance monitoring can be performed in as little time as possible after the acquisition of the input data for the AI-DST.

There is no prior preference for any one clinical track.

We have no preference for the size of the team. Teams should be structured to meet the goals laid out in the technical plan for the proposal.

The ISO document describes AI-DSTs as "AI-DSTs are decision support tools that are used in a context where they provide recommendations for clinicians, and the clinicians are the final decision-makers."

Yes data truthing is a permitted method provided it helps achieve the goals of the program.

Although general uncertainty quantification methods come with their advantages, there is no requirement to use the same methods across use cases.

Prior FDA approval and deployment is not required, but prior and well established baseline performance of the AI-DST is required.

Phase 3 will require performers to re-propose as self-selected teams that collectively address all the TAs. The proposers are expected to describe a plan for Phase III in their submitted proposal, but the focus should be  Phases 1 and 2.

You do not need to have teaming finalized for the solution summary. You can provide a rough estimate of costs you expect to incur for both your own institution and your possible teaming partner.

Yes, you can include support letters as part of your Volume 1 in your full proposal (not included in the page limit).

Apart from citations, hyperlinks to additional proposal content, including videos, will be ignored.

Yes you can include visuals in both the solution summary and the final proposal.

We will not be sharing the attendee list to protect privacy.

The template's font size should be adjusted so that your submission is in English with 12-point Avenir Next LT Pro Light font.

Yes, if you can describe methods to measure the metrics described in the ISO (for example, including, but not limited to agreement of surrogate ground truth with manual ground truth), then automated contouring tools count as AI-DSTs. For automated contouring, since the surrogate ground truth may not come from the EHR, it is recommended  that you submit to both TA1 and TA2.

Yes, distinct hospitals in the same hospital system count as separate clinical sites. However, please note that it is preferred that the proposals  describe innovative, comprehensive surrogate ground truth labeling methods capable of spanning different health centers/clinics.

No, the cover page does not count towards the page limit for both the solution summary. Please check the first page of Appendix A in sam.gov.

Proposals with partners can be combined if this is expected to result in an overall better proposal. There is no formal requirement to combine with partners, but please note that when you submit different proposals, one of the proposals may be recommended for funding while the other may not.

No, there is no preference for clinical versus research AI-DST. However, the performance of any AI-DST included in your proposal must have been already established with sound studies.

There is no preference for one type of organization over another. We are looking for strong technical approaches and teams capable of reaching the goals laid out.

Yes, there is no size consideration for companies.

As the special notice states, "If proposing to multiple TAs, the proposer should provide a summary table for the total effort, and a table for each TA." The table for the total effort should assume that all TAs in the solution summary will be funded. The individual TAs tables should also assume the other TAs will be funded (capturing possible efficiencies associated with having multiple TAs).

The restrictions on FFRDCs apply the same to all FFRDCs.

The solution summary template requires a BOE for the total project. Please note that the BOE  is not included in the page limit, and you are free to include costs per phase if you choose to do so.

This is not permitted. The organization must remain the same.

For budget, we are looking for plans that meet the goals of the TA(s) you are applying to and budgets that fit those plans.

We do not have a preset template for the key personnel resumes.

The minimum suggested margin is 0.5". You are free to use larger margins (eg 1") and generally structure as desired.

You must fill out the table as required in Section 5, Basis of Estimate in the solution summary template for each TA and total of all TAs being proposed. Further justification is welcomed but not required for the solution summary.

Costs should be written up in each solution summary, irrespective of the existence of others. Any streamlining will be done afterwards. Remember that an organization may be a sub-contractor on multiple proposals, but may not be a PI on more than one.

Yes, a condensed version of the NIH Biosketch or the ScienCV format would be acceptable, as long as it conforms to the requirement of being within one single page per key personnel. Here is the specific guideline from the special notice:

"Identify key personnel, and provide no more than 5 single-page, key personnel resumes (as a separate attachment) for the identified individuals (resumes will not count against the page limit)."

Solution summary contents will not be shared outside of the Government and its support contractors. Proposers will retain all rights to the ideas presented in the solution summaries.

The solution summary special notice states that 4pm on November 7th is the deadline.

The PI or anyone at your institution of your choice can submit. 

Yes indirect costs are allowable for all categories.

Yes, a federally negotiated rate can be used. Indirect rates can be applied on all budget categories in accordance with your federally negotiated rate agreement.

Yes, labor hours should list the number of hours of labor.

Awards will depend on the quality of proposals received and funding available.

The goal of TA1 is to extract a surrogate ground truth that can be used in the assessment of AI-enabled decision support tools (AI-DSTs). Please note that if you choose the "other track" in Figure 4, your proposal must address both TA1 and TA2. Please also note that training / developing new AI-DST models is out-of-scope for both TA1 and TA2.

The aim of surrogate ground truth extraction is to evaluate AI-DSTs. A time series waveform data by itself is out-of-scope unless it is used for the performance evaluation of an existing AI-DST.

All TA1 performers are expected to share the surrogate ground truth with TA2 and TA3 performers through the TA4 infrastructure. While all data used for inference is not required to be shared, some may be helpful or needed.

The requirements for TA1 data sources include providing sufficient information to obtain input data and surrogate ground truth which meet the metrics defined in the solicitation and that will enable TA2 performers to meet their metrics. Importantly, please refer to the items listed in the solicitation that are under "TA1 proposals will address the following topics."

the requirements of two and four clinical use cases, for phases I and II respectively, apply to each individual performer for TA2, not collectively.

A2 performers will be free to supplement data received from TA1 performers.  Potential TA2 performers in this situation may want to considering also applying to TA1 in their proposal to maximize their potential impact. 

Performers to TA2 should  have two use cases (two distinct AI-DSTs) on which to apply their work effectively within Phase I. That number increases to 4 use cases in phase II. For the purpose of proposals, identifying the two specific Phase II use cases within your clinical track is desired, but not required. Since it is possible that there will be multiple TA1 performers for each clinical track, it is anticipated that data and labels for your thoracic CT applications will be covered. If you have relatively rare condition as your use case, you may consider applying to both TA1 and TA2.

Performers to TA2 should are expected to have two use cases (two distinct AI-DSTs) on which to apply their work effectively within Phase I. That number increased to 4 use cases in phase II. For the purpose of proposals, identifying the two specific Phase I use cases within your clinical track is desired but not required. You can write about challenges and opportunities with your potential use cases. If you are not applying to TA1, you can discuss your solution in TA2 while abstracting away finer details of the upstream efforts of TA1.

This was stated in error. Organizations that are selected to perform on TA4 are still eligible to perform on TA1-TA3. Please refer to the solicitation for full information.

Since TA2 is expected to use the surrogate ground truth extracted by TA1, we do not recommend making TA2 broader than TA1, especially if you select the "other" clinical track. However, TA1 can be broader than TA2.

Clinician education is not in scope for TA3.

The topics that TA4 and TA5 proposals are required to address are listed in the program solicitation.

Although TA4 performers are not required to provide data, a TA4 participant that can provide access to data from a large number of clinical sites is welcome.

ARPA-H will not provide any training or testing data for the program.

There is no prior assumption that the architecture must be the same or different. There are many possible creative ways to solve this problem and we are looking for the most promising approaches.

TA1 and TA4 both have responsibilities to ensure reliable access to relevant data for TA2 and TA3 performers.

Yes, collaboration is a large component of the program for all performers.

Each proposal responsive to this solicitation can address a TA among TA1, TA2, TA3, TA4 and TA5, or a combination of TA1, TA2, and TA3. Proposals that address TA4 or TA5 cannot address any of the other TAs. If a proposal selects “other” for the clinical track, it must either address both TA1 and TA2, or neither.  An organization or individual who is awarded for TA5 will not be awarded for any other TAs.

Yes, there should be one Section 1-4 that covers the three TAs.

The number of clinical applications for TA3 is the same as the number of clinical applications for TA2, across phases.

Using historical data does not constitute human subjects so you should select no the in the Human Subjects Research field.

The required total number of clinical use cases applies to each individual performer irrespective of the clinical track(s) selected. For example, in Phase II, you can have 4 use cases all of which may belong to a single track.

The numbers stated apply to the total number of clinical uses cases. That required number does not change from phase II to phase III.

TA1 performers are ultimately responsible for generating surrogate ground truth for their clinical use cases, and sharing it with TA2 performers. This requires that TA2 performer to gain access to the data needed to generate surrogate ground truth. Gaining access may be done independently or in collaboration with TA4 performers, who are responsible for setting up the shared infrastructure across TAs.

The scope of the program does not specify technology or specific methods to achieve the program goals. Generating synthetic data can be a step toward achieving goals across TAs, potentially.

Partial awards are a possibility.

Each proposal responsive to this Solicitation can address a TA among TA1, TA2, TA3, TA4 and TA5, or a combination of TA1, TA2, and TA3. Proposals that address TA5 cannot address any of the other TAs. An organization may act as a primary performer on one TA and a sub-performer on a separate proposal addressing the same TA. An organization may act as a sub-performer on multiple proposals addressing the same TA.  If a proposal selects “other” for the clinical track, it must either address both TA1 and TA2, or neither.  An organization or individual who is awarded for TA5 will not be awarded for any other TAs.

Field studies do not need to be randomized clinical trials, but they need to measure the performance of the AI-clinician team.

TA1, TA2, and TA3 can be combined into a single solution summary. if you want to also propose to work on TA4 and/or TA5, then you can submit a separate solution summaries for those. Proposals that address TA4 or TA5 cannot address any of the other TAs.

The two use cases in phase I and four use cases in phase II can indeed belong to a single clinical track. 

MIR and PET-CT are considered to both be in the "other" category.

Each TA-1 performer will propose their own  AI methods for surrogate ground truth extraction in their proposal. As summarized in the "Program Timeline and Milestones" table in the ISO, TA1 performers need to operationalize their software for 2 use cases and 3 clinical sites in Phase 1, and increase the number of use cases to 4 in phase 2. For a team proposing to both TA1 and TA2, this can be satisfied by using your own use cases and AI-DST models. If you are submitting to TA1 only, your ground plans and ground truth extraction methods are expected to be general enough to apply to any AI-DST proposed by a TA2 performer for a clinical track listed in Figure 4. Note that it is possible that adjustments are made in negotiations or during the period of performance. 

Phase 1 of TA1-TA4 is considered basic research. Phase 2 of TA1-TA4 is considered applied research. The appropriate exemptions will apply.

There are no predefined limits for required compute in the program. We do have negotiated agreements for cloud services, which we may use in the program. However, for the purpose of the TA4 budget, assume no special negotiated rates.