skip to main content

NSF INCLUDES Shared Measures

Submit Project Data

Contact Us

Progress Addressing the Design Elements

A hallmark of NSF INCLUDES is the use of the five design elements of collaborative infrastructure, a process by which partner organizations (1) engage their community to formulate a shared vision of what can be accomplished collaboratively; (2) provide a platform for collaborative action; (3) develop common goals, objectives, metrics, and data collection procedures to measure shared progress and inform decision making; (4) develop structures across partner organizations to enhance coordination, communication, and visibility; and (5) establish the capacity for the expansion, sustainability, and scaling of their shared efforts. Each NSF INCLUDES project uses this framework to accelerate its efforts to address systemic barriers to diversity, equity, and inclusion in STEM.

Shared Vision

NSF INCLUDES projects identify a common agenda that reflects a collective understanding of the BP challenge and links to existing research, promising practices, and/or to the previous and ongoing activities of partnering organizations. Large scale networks and partnerships develop a strategic plan to address the BP challenges, including technical infrastructure, which facilitates the implementation of a set of specified activities to achieve targeted outcomes. (Source: NSF Program Solicitation 19-600.)

The graphics below portray survey findings for Shared Vision using respondents as the unit of analysis. In addition to showing results for all survey respondents (i.e., Overall), the charts disaggregate data by the number of years that have elapsed since a respondents’ projects first received NSF INCLUDES funding (i.e., Year 2 or Year 3 of Project Funding). The use of Year of Project Funding allows us to examine which aspects of Shared Vision projects are addressing at a given point in their life cycle.

Our project has a plan that addresses systemic barriers to broadening participation in STEM

  • This has been achieved
  • We have made significant progress
  • We have made some progress
  • We have not started yet
  • Project has no current plans to do this

Our project’s goals are informed by an assessment of the participant population’s needs

  • Strongly agree
  • Somewhat agree
  • Somewhat disagree
  • Strongly disagree

All of our core partners are involved in the process of developing our project’s goals

  • Strongly agree
  • Somewhat agree
  • Somewhat disagree
  • Strongly disagree

Partnerships

NSF INCLUDES projects consist of a set of primary organizations and additional partners that come together locally, regionally, nationally, by disciplinary focus, or by other multisector categories. Partners are supported by an independent “backbone” or support organization to help coordinate and facilitate the collaboration. (Source: NSF Program Solicitation 19-600.)

The graphics below portray survey findings for Partnerships using respondents as the unit of analysis. In addition to showing results for all survey respondents (i.e., Overall), the charts disaggregate data by the number of years that have elapsed since a respondents’ projects first received NSF INCLUDES funding (i.e., Year 2 or Year 3 of Project Funding). The use of Year of Project Funding allows us to examine which aspects of Partnerships projects are addressing at a given point in their life cycle.

Our project has a plan that clearly specifies each partner’s role

  • This has been achieved
  • We have made significant progress
  • We have made some progress
  • We have not started yet
  • Project has no current plans to do this

Our project adds new partners to address a given need (e.g., to access crucial expertise and/or additional participants)

  • This has been achieved
  • We have made significant progress
  • We have made some progress
  • We have not started yet
  • Project has no current plans to do this

The sum of our core and supporting partners represent the range of institutions needed to achieve our project’s goals

  • Strongly agree
  • Somewhat agree
  • Somewhat disagree
  • Strongly disagree

The sum of our core and supporting partners reflect the diversity of our participant population

  • Strongly agree
  • Somewhat agree
  • Somewhat disagree
  • Strongly disagree

My organization has clear goals for its contribution on the project

  • Strongly agree
  • Somewhat agree
  • Somewhat disagree
  • Strongly disagree

My organization is committed to implementing the project’s approach

  • Strongly agree
  • Somewhat agree
  • Somewhat disagree
  • Strongly disagree

My organization changes its activities to better align with the project’s approach

  • Strongly agree
  • Somewhat agree
  • Somewhat disagree
  • Strongly disagree

Goals & Metrics

NSF INCLUDES projects have well-defined, relevant goals and measurable objectives and outcomes, including progress indicators; they outline the role the “backbone” or support organization plays in collecting and coordinating data on outcomes from the proposed partner institutions. This statement of goals and measurable objectives includes describing the types of data that might be collected and how data will be used. Projects have an evaluation plan that use benchmarks, indicators, logic models, road maps, or other evaluative methods to document progress toward goals, objectives, and outcomes. (Source: NSF Program Solicitation 19-600.)

The graphics below portray survey findings for Goals & Metrics using respondents as the unit of analysis. In addition to showing results for all survey respondents (i.e., Overall), the charts disaggregate data by the number of years that have elapsed since a respondents’ projects first received NSF INCLUDES funding (i.e., Year 2 or Year 3 of Project Funding). The use of Year of Project Funding allows us to examine which aspects of Goals & Metrics projects are addressing at a given point in their life cycle.

Our project has participatory processes to refine its measures, indicators, metrics, and/or data collection methods

  • This has been achieved
  • We have made significant progress
  • We have made some progress
  • We have not started yet
  • Project has no current plans to do this

Our project has the capacity to track progress across all partners (e.g., protocols, common metrics)

  • This has been achieved
  • We have made significant progress
  • We have made some progress
  • We have not started yet
  • Project has no current plans to do this

Our project uses data to make regular improvements

  • This has been achieved
  • We have made significant progress
  • We have made some progress
  • We have not started yet
  • Project has no current plans to do this

All of our core partners are involved in the process of making sense of findings that emerge from the project’s analysis of shared measurement data

  • Strongly agree
  • Somewhat agree
  • Somewhat disagree
  • Strongly disagree

My organization is involved in the process of making sense of data collected by the project

  • Strongly agree
  • Somewhat agree
  • Somewhat disagree
  • Strongly disagree

Leadership & Communication

Projects include internal and external communication plans and explain how they develop leadership capacity within and among all partnering organizations. They describe their strategy to engage participating organizations in change management. (Source: NSF Program Solicitation 19-600.)

The graphics below portray survey findings for Leadership & Communication using respondents as the unit of analysis. In addition to showing results for all survey respondents (i.e., Overall), the charts disaggregate data by the number of years that have elapsed since a respondents’ projects first received NSF INCLUDES funding (i.e., Year 2 or Year 3 of Project Funding). The use of Year of Project Funding allows us to examine which aspects of Leadership & Communications projects are addressing at a given point in their life cycle.

Our project’s leadership structure leverages the collective knowledge of partners and other stakeholders

  • Strongly agree
  • Somewhat agree
  • Somewhat disagree
  • Strongly disagree

Our project leadership has structures in place to encourage full participation by all partners

  • Strongly agree
  • Somewhat agree
  • Somewhat disagree
  • Strongly disagree

Our project has internal procedures that minimize power imbalances among partners

  • Strongly agree
  • Somewhat agree
  • Somewhat disagree
  • Strongly disagree

Our project leadership is willing to engage in frank and open discussions when areas of disagreement exist

  • Strongly agree
  • Somewhat agree
  • Somewhat disagree
  • Strongly disagree

Our project leadership provides opportunities for building relationships across partners

  • Strongly agree
  • Somewhat agree
  • Somewhat disagree
  • Strongly disagree

Our project’s decision-making processes are transparent to those inside the project

  • Strongly agree
  • Somewhat agree
  • Somewhat disagree
  • Strongly disagree

Our Project's decisions are informed by input from our participant population (e.g., through representation by members of the participant population on a steering committee)

  • Strongly agree
  • Somewhat agree
  • Somewhat disagree
  • Strongly disagree

All of our core partners collaborate with each other to align their actions

  • Strongly agree
  • Somewhat agree
  • Somewhat disagree
  • Strongly disagree

All of our core partners regularly seek advice from one another (e.g., effective strategies for addressing a given challenge)

  • Strongly agree
  • Somewhat agree
  • Somewhat disagree
  • Strongly disagree

My organization shares its challenges, setbacks, and lessons learned with other partners on the project

  • Strongly agree
  • Somewhat agree
  • Somewhat disagree
  • Strongly disagree

My organization seeks advice from other partners on the project (e.g., effective strategies for addressing a given challenge)

  • Strongly agree
  • Somewhat agree
  • Somewhat disagree
  • Strongly disagree

Expansion, Sustainability, & Scale

Projects include plans that will lead to the expansion, sustainability, and scale of their activities. They describe their overall contribution to broadening participation in the nation’s scientific and engineering workforce. For large scale partnerships and networks, a “backbone” or support organization facilitates and sustains these efforts over the long term with support from the NSF INCLUDES Coordination Hub. (Source: NSF Program Solicitation 19-600.)

The graphics below portray survey findings for Expansion, Sustainability, & Scale using respondents as the unit of analysis. In addition to showing results for all survey respondents (i.e., Overall), the charts disaggregate data by the number of years that have elapsed since a respondents’ projects first received NSF INCLUDES funding (i.e., Year 2 or Year 3 of Project Funding). The use of Year of Project Funding allows us to examine which aspects of Expansion, Sustainability, & Scale projects are addressing at a given point in their life cycle.

Our project contributes to the field’s knowledge base about effective strategies for broadening participation in STEM

  • This has been achieved
  • We have made significant progress
  • We have made some progress
  • We have not started yet
  • Project has no current plans to do this

Project has a written plan that outlines a strategy for sustaining activities beyond the current award period

  • This has been achieved
  • We have made significant progress
  • We have made some progress
  • We have not started yet
  • Project has no current plans to do this

Project has secured funding beyond the current award period

  • This has been achieved
  • We have made significant progress
  • We have made some progress
  • We have not started yet
  • Project has no current plans to do this

Our project has a strategic vision of what activities will be sustained beyond the current award period

  • Strongly agree
  • Somewhat agree
  • Somewhat disagree
  • Strongly disagree

Because the survey was administered for the first time in spring 2021, we presently have no data on respondents’ perceptions of progress at the end of the first year of NSF INLCUDES funding. Going forward, we will obtain Year 1 data from initiatives that are just beginning their work on an NSF INCLUDES-funded project.) As such, we are currently unable to provide information about the relative progress that respondents would have reported for their initial year.

In theory, one would expect that Alliances with more years of NSF INCLUDES funding would report more progress around the operationalization of a given design element. However, we are somewhat cautious when making such comparisons because it is possible that the characteristics of Alliances funded in a given cohort differ (e.g., in terms of the maturity and complexity of their partnership structure, the range of barriers they are designed to address, the characteristics of their participant population, and the complexity of their approach). In addition, respondents’ perspectives concerning their accomplishments (or the progress they still need to make) about a given design element may shift as they recognize the complexity of a given issue—with respondents realizing more work is needed as they begin to delve more deeply into a particular task.