skip to main content

NSF INCLUDES Shared Measures

Request to Submit Project Data

Contact Us

Progress Addressing the Design Elements

A hallmark of NSF INCLUDES is the use of the five design elements of collaborative infrastructure, a process by which partner organizations (1) engage their community to formulate a shared vision of what can be accomplished collaboratively; (2) provide a platform for collaborative action; (3) develop common goals, objectives, metrics, and data collection procedures to measure shared progress and inform decision making; (4) develop structures across partner organizations to enhance coordination, communication, and visibility; and (5) establish the capacity for the expansion, sustainability, and scaling of their shared efforts. Each NSF INCLUDES project uses this framework to accelerate its efforts to address systemic barriers to diversity, equity, and inclusion in STEM.

Shared Vision

NSF INCLUDES projects identify a common agenda that reflects a collective understanding of the BP challenge and links to existing research, promising practices, and/or to the previous and ongoing activities of partnering organizations. Large scale networks and partnerships develop a strategic plan to address the BP challenges, including technical infrastructure, which facilitates the implementation of a set of specified activities to achieve targeted outcomes. (Source: NSF Program Solicitation 19-600.)

The graphics below portray survey findings for Shared Vision using respondents as the unit of analysis. In addition to showing results for all survey respondents (i.e., Overall), the charts disaggregate data by the number of years that have elapsed since a respondents’ projects first received NSF INCLUDES funding (i.e., Year 2 or Year 3 of Project Funding). The use of Year of Project Funding allows us to examine which aspects of Shared Vision projects are addressing at a given point in their life cycle.

Our project has a plan that addresses systemic barriers to broadening participation in STEM

Nearly three-fourths (72%) of respondents indicated that their project had developed (or had made significant progress in developing) a plan for addressing systemic barriers to broadening participation in STEM. Respondents from Alliances with 2 years of funding were more likely to report significant progress or achievement in this area (76%, compared with 68% for those from projects with 3 years of funding).

  • This has been achieved
  • We have made significant progress
  • We have made some progress
  • We have not started yet
  • Project has no current plans to do this

Our project’s goals are informed by an assessment of the participant population’s needs

Nearly all (92%) respondents indicated that their project’s goals had been informed by an assessment of the needs of their participant population. Respondents from Alliances with 3 years of funding were more likely to somewhat or strongly agree with this statement (95%, compared with 89% for those from projects with 2 years of funding).

  • Strongly agree
  • Somewhat agree
  • Somewhat disagree
  • Strongly disagree

All of our core partners are involved in the process of developing our project’s goals

Nearly all (94%) respondents somewhat or strongly agreed that all of their core partners were involved in the process of developing their project’s goals. Respondents from Alliances with 2 years of funding were more likely to strongly agree with this statement (60%, compared with 49% for those from projects with 3 years of funding).

  • Strongly agree
  • Somewhat agree
  • Somewhat disagree
  • Strongly disagree

Partnerships

NSF INCLUDES projects consist of a set of primary organizations and additional partners that come together locally, regionally, nationally, by disciplinary focus, or by other multisector categories. Partners are supported by an independent “backbone” or support organization to help coordinate and facilitate the collaboration. (Source: NSF Program Solicitation 19-600.)

The graphics below portray survey findings for Partnerships using respondents as the unit of analysis. In addition to showing results for all survey respondents (i.e., Overall), the charts disaggregate data by the number of years that have elapsed since a respondents’ projects first received NSF INCLUDES funding (i.e., Year 2 or Year 3 of Project Funding). The use of Year of Project Funding allows us to examine which aspects of Partnerships projects are addressing at a given point in their life cycle.

Our project has a plan that clearly specifies each partner’s role

More than four-fifths (81%) of respondents indicated that their project had developed (or had made significant progress in developing) a plan that clearly specified each partner’s role. There was no meaningful difference by year of funding.

  • This has been achieved
  • We have made significant progress
  • We have made some progress
  • We have not started yet
  • Project has no current plans to do this

Our project adds new partners to address a given need (e.g., to access crucial expertise and/or additional participants)

Nearly three-fifths (59%) of respondents indicated that their project had added (or had made significant plans to add) new partners to address a given need. Respondents from Alliances with 3 years of funding were more likely to report significant progress or achievement in this area (65%, compared with 55% for those from projects with 2 years of funding).

  • This has been achieved
  • We have made significant progress
  • We have made some progress
  • We have not started yet
  • Project has no current plans to do this

The sum of our core and supporting partners represent the range of institutions needed to achieve our project’s goals

Most (85%) respondents somewhat or strongly agreed that the sum of their core and supporting partners represented the range of institutions needed to achieve the goals of their project. Respondents from Alliances with 3 years of funding were more likely to strongly agree with this statement (53%, compared with 43% for those from projects with 2 years of funding).

  • Strongly agree
  • Somewhat agree
  • Somewhat disagree
  • Strongly disagree

The sum of our core and supporting partners reflect the diversity of our participant population

Nearly three-fourths (71%) of respondents somewhat or strongly agreed that the sum of their core and supporting partners reflected the diversity of their participant population. There was no meaningful difference by year of funding.

  • Strongly agree
  • Somewhat agree
  • Somewhat disagree
  • Strongly disagree

My organization has clear goals for its contribution on the project

More than four-fifths (84%) of respondents somewhat or strongly agreed that their organization had clear goals for its contribution to the project. Respondents from Alliances with 3 years of funding were more likely to strongly agree with this statement (77%, compared with 17% for those from projects with 2 years of funding).

  • Strongly agree
  • Somewhat agree
  • Somewhat disagree
  • Strongly disagree

My organization is committed to implementing the project’s approach

Nearly all (95%) respondents somewhat or strongly agreed that their organization was committed to implementing their project’s approach. Respondents from Alliances with 3 years of funding were more likely to strongly agree with this statement (85%, compared with 33% for those from projects with 2 years of funding).

  • Strongly agree
  • Somewhat agree
  • Somewhat disagree
  • Strongly disagree

My organization changes its activities to better align with the project’s approach

Most (89%) respondents somewhat or strongly agreed that their organization had changed its activities to better align with the project’s approach. Respondents from Alliances with 3 years of funding were more likely to strongly agree with this statement (58%, compared with 44% for those from projects with 2 years of funding).

  • Strongly agree
  • Somewhat agree
  • Somewhat disagree
  • Strongly disagree

Goals & Metrics

NSF INCLUDES projects have well-defined, relevant goals and measurable objectives and outcomes, including progress indicators; they outline the role the “backbone” or support organization plays in collecting and coordinating data on outcomes from the proposed partner institutions. This statement of goals and measurable objectives includes describing the types of data that might be collected and how data will be used. Projects have an evaluation plan that use benchmarks, indicators, logic models, road maps, or other evaluative methods to document progress toward goals, objectives, and outcomes. (Source: NSF Program Solicitation 19-600.)

The graphics below portray survey findings for Goals & Metrics using respondents as the unit of analysis. In addition to showing results for all survey respondents (i.e., Overall), the charts disaggregate data by the number of years that have elapsed since a respondents’ projects first received NSF INCLUDES funding (i.e., Year 2 or Year 3 of Project Funding). The use of Year of Project Funding allows us to examine which aspects of Goals & Metrics projects are addressing at a given point in their life cycle.

Our project has participatory processes to refine its measures, indicators, metrics, and/or data collection methods

More than three-fourths (77%) of respondents indicated that their project had (or have made significant progress toward developing) a participatory process to refine its measures, indicators, metrics, and/or data collection methods. Respondents from Alliances with 3 years of funding were more likely to report significant progress or achievement in this area (84%, compared with 70% for those from projects with 2 years of funding).

  • This has been achieved
  • We have made significant progress
  • We have made some progress
  • We have not started yet
  • Project has no current plans to do this

Our project has the capacity to track progress across all partners (e.g., protocols, common metrics)

More than three-fifths (62%) of respondents indicated that their project had (or had made significant progress toward developing) the capacity to track progress across all partners. Respondents from Alliances with 3 years of funding were marginally more likely to report significant progress or achievement in this area (64%, compared with 60% for those from projects with 2 years of funding).

  • This has been achieved
  • We have made significant progress
  • We have made some progress
  • We have not started yet
  • Project has no current plans to do this

Our project uses data to make regular improvements

More than three-fifths (61%) of respondents indicated that their project had achieved (or had made significant progress toward) using data to make regular improvements. Respondents from Alliances with 3 years of funding were more likely to report significant progress or achievement in this area (24%, compared with 20% for those from projects with 2 years of funding).

  • This has been achieved
  • We have made significant progress
  • We have made some progress
  • We have not started yet
  • Project has no current plans to do this

All of our core partners are involved in the process of making sense of findings that emerge from the project’s analysis of shared measurement data

Most (89%) respondents somewhat or strongly agreed that all of their core partners were involved in the process of making sense of findings that emerged from their project’s analysis of their shared measurement data. Respondents from Alliances with 3 years of funding were slightly more likely to strongly agree with this statement (41%, compared with 33% for those from projects with 2 years of funding).

  • Strongly agree
  • Somewhat agree
  • Somewhat disagree
  • Strongly disagree

My organization is involved in the process of making sense of data collected by the project

Nearly three-fourths (74%) of respondents somewhat or strongly agreed that their organization was involved in the process of making sense of data collected by the Alliance. Respondents from Alliances with 3 years of funding were more likely to strongly agree with this statement (54%, compared with 17% for those from projects with 2 years of funding).

  • Strongly agree
  • Somewhat agree
  • Somewhat disagree
  • Strongly disagree

Leadership & Communication

Projects include internal and external communication plans and explain how they develop leadership capacity within and among all partnering organizations. They describe their strategy to engage participating organizations in change management. (Source: NSF Program Solicitation 19-600.)

The graphics below portray survey findings for Leadership & Communication using respondents as the unit of analysis. In addition to showing results for all survey respondents (i.e., Overall), the charts disaggregate data by the number of years that have elapsed since a respondents’ projects first received NSF INCLUDES funding (i.e., Year 2 or Year 3 of Project Funding). The use of Year of Project Funding allows us to examine which aspects of Leadership & Communications projects are addressing at a given point in their life cycle.

Our project’s leadership structure leverages the collective knowledge of partners and other stakeholders

Nearly all (93%) respondents somewhat or strongly agreed that their project’s leadership structure leveraged the collective knowledge of partners and other stakeholders. Respondents from Alliances with 2 years of funding were more likely to strongly agree with this statement (76%, compared with 63% for those from projects with 3 years of funding).

  • Strongly agree
  • Somewhat agree
  • Somewhat disagree
  • Strongly disagree

Our project leadership has structures in place to encourage full participation by all partners

Nearly all (95%) respondents somewhat or strongly agreed that their project’s leadership had structures in place to encourage full participation by all partners. Respondents from Alliances with 2 years of funding were more likely to strongly agree with this statement (64%, compared with 43% for those from projects with 3 years of funding).

  • Strongly agree
  • Somewhat agree
  • Somewhat disagree
  • Strongly disagree

Our project has internal procedures that minimize power imbalances among partners

Nearly all (92%) respondents somewhat or strongly agreed that their project had internal procedures in place that minimize power imbalances among partners. Respondents from Alliances with 2 years of funding were more likely to somewhat or strongly agree with this statement (98%, compared with 86% for those from projects with 3 years of funding).

  • Strongly agree
  • Somewhat agree
  • Somewhat disagree
  • Strongly disagree

Our project leadership is willing to engage in frank and open discussions when areas of disagreement exist

Nearly all (92%) respondents somewhat or strongly agreed that their project leadership was willing to engage in frank and open discussions when areas of disagreement exist. Respondents from Alliances with 2 years of funding were more likely to somewhat or strongly agree with this statement (98%, compared with 85% for those from projects with 3 years of funding).

  • Strongly agree
  • Somewhat agree
  • Somewhat disagree
  • Strongly disagree

Our project leadership provides opportunities for building relationships across partners

Nearly all (95%) respondents somewhat or strongly agreed that their project leadership provided opportunities for building relationships across partners. Respondents from Alliances with 2 years of funding were more likely to strongly agree with this statement (71%, compared with 58% for those from projects with 3 years of funding).

  • Strongly agree
  • Somewhat agree
  • Somewhat disagree
  • Strongly disagree

Our project’s decision-making processes are transparent to those inside the project

Most (89%) respondents somewhat or strongly agreed that their project’s decision-making processes were transparent to those inside the project. Respondents from Alliances with 2 years of funding were more likely to strongly agree with this statement (59%, compared with 39% for those from projects with 3 years of funding).

  • Strongly agree
  • Somewhat agree
  • Somewhat disagree
  • Strongly disagree

Our Project's decisions are informed by input from our participant population (e.g., through representation by members of the participant population on a steering committee)

More than four-fifths (83%) of respondents somewhat or strongly agreed that their project’s decisions were informed by input from the participant population. Respondents from Alliances with 2 years of funding were slightly more likely to strongly agree with this statement (51%, compared with 46% for those from projects with 3 years of funding).

  • Strongly agree
  • Somewhat agree
  • Somewhat disagree
  • Strongly disagree

All of our core partners collaborate with each other to align their actions

Most (88%) respondents somewhat or strongly agreed that all core partners collaborated with each other to align their actions. Respondents from Alliances with 2 years of funding were more likely to strongly agree with this statement (54%, compared with 39% for those from projects with 3 years of funding).

  • Strongly agree
  • Somewhat agree
  • Somewhat disagree
  • Strongly disagree

All of our core partners regularly seek advice from one another (e.g., effective strategies for addressing a given challenge)

Nearly all (91%)respondents somewhat or strongly agreed that all core partners regularly seek advice from one another. Respondents from Alliances with 2 years of funding were more likely to strongly agree with this statement (48%, compared with 40% for those from projects with 3 years of funding).

  • Strongly agree
  • Somewhat agree
  • Somewhat disagree
  • Strongly disagree

My organization shares its challenges, setbacks, and lessons learned with other partners on the project

Nearly all (89.4%) respondents somewhat or strongly agreed that their organizations shared their challenges, setbacks, and lessons learned with other partners on the project. Respondents from Alliances with 3 years of funding were more likely to strongly agree with this statement (46%, compared with 17% for those from projects with 2 years of funding).

  • Strongly agree
  • Somewhat agree
  • Somewhat disagree
  • Strongly disagree

My organization seeks advice from other partners on the project (e.g., effective strategies for addressing a given challenge)

Nearly all (94%) respondents somewhat or strongly agreed that their organization seeks advice from other partners on the project. Respondents from Alliances with 3 years of funding were more likely to strongly agree with this statement (50%, compared with 33% for those from projects with 2 years of funding).

  • Strongly agree
  • Somewhat agree
  • Somewhat disagree
  • Strongly disagree

Expansion, Sustainability, & Scale

Projects include plans that will lead to the expansion, sustainability, and scale of their activities. They describe their overall contribution to broadening participation in the nation’s scientific and engineering workforce. For large scale partnerships and networks, a “backbone” or support organization facilitates and sustains these efforts over the long term with support from the NSF INCLUDES Coordination Hub. (Source: NSF Program Solicitation 19-600.)

The graphics below portray survey findings for Expansion, Sustainability, & Scale using respondents as the unit of analysis. In addition to showing results for all survey respondents (i.e., Overall), the charts disaggregate data by the number of years that have elapsed since a respondents’ projects first received NSF INCLUDES funding (i.e., Year 2 or Year 3 of Project Funding). The use of Year of Project Funding allows us to examine which aspects of Expansion, Sustainability, & Scale projects are addressing at a given point in their life cycle.

Our project contributes to the field’s knowledge base about effective strategies for broadening participation in STEM

Nearly three-fourths (70%) of respondents indicated that their project had contributed (or had made significant progress toward contributing) to the field’s knowledge base about effective strategies for broadening participation in STEM. Respondents from Alliances with 3 years of funding were more likely to report significant progress or achievement in this area (80%, compared with 61% for those from projects with 2 years of funding).

  • This has been achieved
  • We have made significant progress
  • We have made some progress
  • We have not started yet
  • Project has no current plans to do this

Project has a written plan that outlines a strategy for sustaining activities beyond the current award period

Just under one-third (30%) of respondents indicated that their project had (or had made significant progress toward developing) a written plan that outlined a strategy for sustaining activities beyond the current award period. Respondents from Alliances with 3 years of funding were more likely to report significant progress or achievement in this area (33%, compared with 28% for those from projects with 2 years of funding).

  • This has been achieved
  • We have made significant progress
  • We have made some progress
  • We have not started yet
  • Project has no current plans to do this

Project has secured funding beyond the current award period

Just over one-tenth (12%) of respondents indicated that their project had secured (or had made significant progress toward securing) funding beyond the current award period. Respondents from Alliances with 3 years of funding were more likely to report significant progress or achievement in this area (16%, compared with 9% for those from projects with 2 years of funding).

  • This has been achieved
  • We have made significant progress
  • We have made some progress
  • We have not started yet
  • Project has no current plans to do this

Our project has a strategic vision of what activities will be sustained beyond the current award period

Nearly three-fourths (73%) of respondents somewhat or strongly agreed that their project had a strategic vision of what activities would be sustained beyond the current award period. Respondents from Alliances with 2 years of funding were more likely to somewhat or strongly agree with this statement (83%, compared with 62% for those from projects with 3 years of funding).

  • Strongly agree
  • Somewhat agree
  • Somewhat disagree
  • Strongly disagree

Because the survey was administered for the first time in spring 2021, we presently have no data on respondents’ perceptions of progress at the end of the first year of NSF INLCUDES funding. Going forward, we will obtain Year 1 data from initiatives that are just beginning their work on an NSF INCLUDES-funded project.) As such, we are currently unable to provide information about the relative progress that respondents would have reported for their initial year.

In theory, one would expect that Alliances with more years of NSF INCLUDES funding would report more progress around the operationalization of a given design element. However, we are somewhat cautious when making such comparisons because it is possible that the characteristics of Alliances funded in a given cohort differ (e.g., in terms of the maturity and complexity of their partnership structure, the range of barriers they are designed to address, the characteristics of their participant population, and the complexity of their approach). In addition, respondents’ perspectives concerning their accomplishments (or the progress they still need to make) about a given design element may shift as they recognize the complexity of a given issue—with respondents realizing more work is needed as they begin to delve more deeply into a particular task.