skip to main content

NSF INCLUDES Shared Measures

Contact Us

Progress Addressing the Design Elements

A hallmark of NSF INCLUDES is the use of the five design elements of collaborative infrastructure, a process by which partner organizations (1) engage their community to formulate a shared vision of what can be accomplished collaboratively; (2) provide a platform for collaborative action; (3) develop common goals, objectives, metrics, and data collection procedures to measure shared progress and inform decision making; (4) develop structures across partner organizations to enhance coordination, communication, and visibility; and (5) establish the capacity for the expansion, sustainability, and scaling of their shared efforts. Each project funded or co-funded by NSF INCLUDES uses this framework to accelerate its efforts to address systemic barriers to diversity, equity, and inclusion in STEM.

This page presents aggregated responses from the nine Alliances that participated in the Hub’s Collaborative Infrastructure Survey. Additionally, 42 stakeholders across six NSF INCLUDES Planning Grants completed the survey in 2022.

A hallmark of NSF INCLUDES is the use of the five design elements of collaborative infrastructure, a process by which partner organizations (1) engage their community to formulate a shared vision of what can be accomplished collaboratively; (2) provide a platform for collaborative action; (3) develop common goals, objectives, metrics, and data collection procedures to measure shared progress and inform decision making; (4) develop structures across partner organizations to enhance coordination, communication, and visibility; and (5) establish the capacity for the expansion, sustainability, and scaling of their shared efforts. Each NSF INCLUDES project uses this framework to accelerate its efforts to address systemic barriers to diversity, equity, and inclusion in STEM.

Shared Vision

NSF INCLUDES projects identify a common agenda that reflects a collective understanding of the broadening participation (BP) challenge and links to existing research, promising practices, and/or to the previous and ongoing activities of partnering organizations. Large scale networks and partnerships develop a strategic plan to address the BP challenges, including technical infrastructure, which facilitates the implementation of a set of specified activities to achieve targeted outcomes. (Source: NSF Program Solicitation 19-600.)

The graphics below portray survey findings for Shared Vision using Alliance respondents as the unit of analysis. In addition to showing results for all Alliance respondents (i.e., Overall), the charts disaggregate data by the number of years that have elapsed since respondents’ Alliances first received NSF INCLUDES funding. The use of Year of Project Funding allows us to examine which aspects of Shared Vision Alliances are addressing at a given point in their life cycle.

VIEW

Our project has a plan that addresses systemic barriers to broadening participation in STEM

  • This has been achieved
  • We have made significant progress
  • We have made some progress
  • We have not started yet
  • Project has no current plans to do this

Our project’s goals are informed by an assessment of the participant population’s needs

  • Strongly agree
  • Somewhat agree
  • Somewhat disagree
  • Strongly disagree

All of our core partners are involved in the process of developing our project’s goals

  • Strongly agree
  • Somewhat agree
  • Somewhat disagree
  • Strongly disagree

Partnerships

NSF INCLUDES projects consist of a set of primary organizations and additional partners that come together locally, regionally, nationally, by disciplinary focus, or by other multisector categories. Partners are supported by an independent “backbone” or support organization to help coordinate and facilitate the collaboration. (Source: NSF Program Solicitation 19-600.)

The graphics below portray survey findings for Partnerships using Alliance respondents as the unit of analysis. In addition to showing results for all Alliance respondents (i.e., Overall), the charts disaggregate data by the number of years that have elapsed since respondents' Alliances first received NSF INCLUDES funding. The use of Year of Project Funding allows us to examine which aspects of Partnerships Alliances are addressing at a given point in their life cycle.

VIEW

Our project has a plan that clearly specifies each partner’s role

  • This has been achieved
  • We have made significant progress
  • We have made some progress
  • We have not started yet
  • Project has no current plans to do this

Our project adds new partners to address a given need (e.g., to access crucial expertise and/or additional participants)

  • This has been achieved
  • We have made significant progress
  • We have made some progress
  • We have not started yet
  • Project has no current plans to do this

The sum of our core and supporting partners represent the range of institutions needed to achieve our project’s goals

  • Strongly agree
  • Somewhat agree
  • Somewhat disagree
  • Strongly disagree

The sum of our core and supporting partners reflect the diversity of our participant population

  • Strongly agree
  • Somewhat agree
  • Somewhat disagree
  • Strongly disagree

My organization has clear goals for its contribution on the project

  • Strongly agree
  • Somewhat agree
  • Somewhat disagree
  • Strongly disagree

My organization is committed to implementing the project’s approach

  • Strongly agree
  • Somewhat agree
  • Somewhat disagree
  • Strongly disagree

My organization changes its activities to better align with the project’s approach

  • Strongly agree
  • Somewhat agree
  • Somewhat disagree
  • Strongly disagree

Goals & Metrics

NSF INCLUDES projects have well-defined, relevant goals and measurable objectives and outcomes, including progress indicators; they outline the role the “backbone” or support organization plays in collecting and coordinating data on outcomes from the proposed partner institutions. This statement of goals and measurable objectives includes describing the types of data that might be collected and how data will be used. Projects have an evaluation plan that use benchmarks, indicators, logic models, road maps, or other evaluative methods to document progress toward goals, objectives, and outcomes. (Source: NSF Program Solicitation 19-600.)

The graphics below portray survey findings for Goals & Metrics using Alliance respondents as the unit of analysis. In addition to showing results for all Alliance respondents (i.e., Overall), the charts disaggregate data by the number of years that have elapsed since respondents’ Alliances first received NSF INCLUDES funding. The use of Year of Project Funding allows us to examine which aspects of Goals & Metrics Alliances are addressing at a given point in their life cycle.

VIEW

Our project has participatory processes to refine its measures, indicators, metrics, and/or data collection methods

  • This has been achieved
  • We have made significant progress
  • We have made some progress
  • We have not started yet
  • Project has no current plans to do this

Our project has the capacity to track progress across all partners (e.g., protocols, common metrics)

  • This has been achieved
  • We have made significant progress
  • We have made some progress
  • We have not started yet
  • Project has no current plans to do this

Our project uses data to make regular improvements

  • This has been achieved
  • We have made significant progress
  • We have made some progress
  • We have not started yet
  • Project has no current plans to do this

All of our core partners are involved in the process of making sense of findings that emerge from the project’s analysis of shared measurement data

  • Strongly agree
  • Somewhat agree
  • Somewhat disagree
  • Strongly disagree

My organization is involved in the process of making sense of data collected by the project

  • Strongly agree
  • Somewhat agree
  • Somewhat disagree
  • Strongly disagree

Leadership & Communication

Projects include internal and external communication plans and explain how they develop leadership capacity within and among all partnering organizations. They describe their strategy to engage participating organizations in change management. (Source: NSF Program Solicitation 19-600.)

The graphics below portray survey findings for Leadership & Communication using Alliance respondents as the unit of analysis. In addition to showing results for all Alliance respondents (i.e., Overall), the charts disaggregate data by the number of years that have elapsed since respondents' Alliances first received NSF INCLUDES funding. The use of Year of Project Funding allows us to examine which aspects of Leadership & Communication Alliances are addressing at a given point in their life cycle.

VIEW

Our project’s leadership structure leverages the collective knowledge of partners and other stakeholders

  • Strongly agree
  • Somewhat agree
  • Somewhat disagree
  • Strongly disagree

Our project leadership has structures in place to encourage full participation by all partners

  • Strongly agree
  • Somewhat agree
  • Somewhat disagree
  • Strongly disagree

Our project has internal procedures that minimize power imbalances among partners

  • Strongly agree
  • Somewhat agree
  • Somewhat disagree
  • Strongly disagree

Our project leadership is willing to engage in frank and open discussions when areas of disagreement exist

  • Strongly agree
  • Somewhat agree
  • Somewhat disagree
  • Strongly disagree

Our project leadership provides opportunities for building relationships across partners

  • Strongly agree
  • Somewhat agree
  • Somewhat disagree
  • Strongly disagree

Our project’s decision-making processes are transparent to those inside the project

  • Strongly agree
  • Somewhat agree
  • Somewhat disagree
  • Strongly disagree

Our Project's decisions are informed by input from our participant population (e.g., through representation by members of the participant population on a steering committee)

  • Strongly agree
  • Somewhat agree
  • Somewhat disagree
  • Strongly disagree

All of our core partners collaborate with each other to align their actions

  • Strongly agree
  • Somewhat agree
  • Somewhat disagree
  • Strongly disagree

All of our core partners regularly seek advice from one another (e.g., effective strategies for addressing a given challenge)

  • Strongly agree
  • Somewhat agree
  • Somewhat disagree
  • Strongly disagree

My organization shares its challenges, setbacks, and lessons learned with other partners on the project

  • Strongly agree
  • Somewhat agree
  • Somewhat disagree
  • Strongly disagree

My organization seeks advice from other partners on the project (e.g., effective strategies for addressing a given challenge)

  • Strongly agree
  • Somewhat agree
  • Somewhat disagree
  • Strongly disagree

Expansion, Sustainability, & Scale

Projects include plans that will lead to the expansion, sustainability, and scale of their activities. They describe their overall contribution to broadening participation in the nation’s scientific and engineering workforce. For large scale partnerships and networks, a “backbone” or support organization facilitates and sustains these efforts over the long term with support from the NSF INCLUDES Coordination Hub. (Source: NSF Program Solicitation 19-600.)

The graphics below portray survey findings for Expansion, Sustainability, & Scale using Alliance respondents as the unit of analysis. In addition to showing results for all Alliance respondents (i.e., Overall), the charts disaggregate data by the number of years that have elapsed since respondents' Alliances first received NSF INCLUDES funding. The use of Year of Project Funding allows us to examine which aspects of Expansion, Sustainability, & Scale Alliances are addressing at a given point in their life cycle.

VIEW

Our project contributes to the field’s knowledge base about effective strategies for broadening participation in STEM

  • This has been achieved
  • We have made significant progress
  • We have made some progress
  • We have not started yet
  • Project has no current plans to do this

Project has a written plan that outlines a strategy for sustaining activities beyond the current award period

  • This has been achieved
  • We have made significant progress
  • We have made some progress
  • We have not started yet
  • Project has no current plans to do this

Project has secured funding beyond the current award period

  • This has been achieved
  • We have made significant progress
  • We have made some progress
  • We have not started yet
  • Project has no current plans to do this

Our project has a strategic vision of what activities will be sustained beyond the current award period

  • Strongly agree
  • Somewhat agree
  • Somewhat disagree
  • Strongly disagree

In theory, one would expect that Alliances with more years of NSF INCLUDES funding would report more progress around the operationalization of a given design element. However, it is possible that the characteristics of Alliances funded in a given cohort differ (e.g., in terms of the maturity and complexity of their partnership structure, the range of barriers they are designed to address, the characteristics of their participant population, and the complexity of their approach). In addition, respondents’ perspectives concerning their accomplishments (or the progress they still need to make) around a given design element may shift as they recognize the complexity of a given issue—with respondents realizing more work is needed as they begin to delve more deeply into a particular task. Finally, the composition of respondents within a given Alliance may change over time.