The Role of Bronfenbrenner ’ s Ecological Systems Theory in Enhancing Interpretive Consistency in Mixed Research 1

651 | P a g e M a r c h 2 2 , 2 0 1 4 The Role of Bronfenbrenner’s Ecological Systems Theory in Enhancing Interpretive Consistency in Mixed Research1 Anthony J. Onwuegbuzie Sam Houston State University Department of Educational Leadership and Counseling Box 2119, Huntsville, Texas 77341-2119 E-Mail: tonyonwuegbuzie@aol.com Kathleen M. T. Collins University of Arkansas at Fayetteville Department of Curriculum & Instruction 241 Graduate Education Bldg.Fayetteville, AR 72701 E-Mail: kxc01@uark.edu Abstract One of the nine major threats to legitimation (i.e., the degree that integration of findings leads to credible and defensible meta-inferences) is sample legitimation integration (Onwuegbuzie & Johnson, 2006). Addressing this form of legitimation requires the researcher to maintain interpretive consistency between the selected sampling design and the inferences made from the ensuing findings. To facilitate researchers‘ efforts to address interpretive consistency, in this article, we provide a meta-sampling framework that is structured in accordance to the dimensions of Bronfenbrenner‘s (1979) ecological systems model. In this meta-framework, the four dimensions of the model are juxtaposed to various types of generalizations, sampling-based considerations, and mixed sampling criteria. Application of this inclusive framework is appropriate for the conduct of quantitative, qualitative, and mixed research.

Since these ancient times, virtually all research representing the social, behavioral, and health sciences has yielded findings that led to generalizations that are made by the researcher somewhere on the particularistic-universalistic continuum (Onwuegbuzie, Slate, Leech, & Collins, 2009). At one end of the continuum (i.e., particularistic) are research studies that yield generalizations that are used to describe phenomena that occur, or have occurred, in a specific context. At the other end of the continuum (i.e., universalistic) are research studies that yield generalizations that are used to describe phenomena that can be applied to broad contexts--whether the context either represents people other than the study participants or represents different situations, locations, or time periods for the same study participants‖ (Onwuegbuzie et al., 2009, p. 28). Thus, for instance, studying perceptions, behaviors, or experiences pertaining to one individual leads to generalizations that lie somewhere toward the particularistic end of the particularistic-universalistic continuum, whereas studying the same construct on a large sample of individuals yields generalizations that lie somewhere toward the universalistic end of the particularistic-universalistic continuum. Conversely, the mid-point of the particularistic-universalistic continuum represents generalizations used to describe phenomena that occur, or have occurred, in a specific context but that can be applied to broad [er] contexts.
Among researchers representing the quantitative research tradition (i.e., hereafter referred to as quantitative researchers), there appears to be unequivocal agreement that generalization is an important goal of research. However, as concluded by Williams (2000), many-if not most-researchers representing the qualitative research tradition (i.e., hereafter referred to as qualitative researchers) question the degree that generalization is possible. As stated by Williams (2000): The interpretivist attitude to generalisation is rather like that of the Victorian middle classes towards sex. They do it, they know it goes on, but they rarely admit to either. A few, Denzin (1983), or Guba and Lincoln (1982Lincoln ( , 1994, are more explicitly in denial, claiming that generalisation is impossible. However for the most part, papers reporting on results of research using interpretive methods will make generalizing statements about findings whilst not commenting upon the basis upon which such generalisations might be justified. Text books, or methodological works on interpretivism (or more broadly qualitative research) will not usually refer to generalisation, or it will be addressed obliquely in the context of ‗external validity', ‗transferability', ‗confirmability'(see Ward-Schofield 1993 for a discussion of this). (p. 210) Further, as surmised by Williams (2000), -Generalisation seems to be inevitable in interpretivist research. Indeed, virtually every reported study will contain at least some kinds of generalising claim‖ (p. 210). More specifically, Onwuegbuzie and Leech (2007) posited the following: While it is true that in many situations qualitative researchers are not interested in generalizing findings beyond the people they directly study, we contend that in virtually every qualitative study, one or more of the following generalizations nearly always are made: (a) from the sample of words to the voice; (b) from the sample of observations to the truth space; (c) from the words of key informants to the voice of the other sample members; (d) from the words of sample members to those of one or more individuals not selected for the study; or (e) from the observations of sample members to the experience of one or more individuals not selected for the study. (p. 107) Indeed, as soon as a qualitative researcher uses a phrase such as The [emergent] themes were…., he or she is making some form of generalization because the researcher has reduced each participant's voice (i.e., data) to codes (e.g., inductive codes, deductive codes, abductive codes, interpretive codes) to themes-with each theme representing some form of generalization of a part of each participant's story, which, in turn, represents some form of generalization of each participant's lived experience that is of interest to the researcher. Such a generalization takes place regardless of the sample size; that is, even if the qualitative inquiry involves one participant, some form of generalization still takes place. Onwuegbuzie et al. (2009) identified the following five major types of generalizations that researchers can make (a) external (statistical) generalizations (i.e., making generalizations, judgments, predictions, or inferences on data yielded from a representative statistical [i.e., optimally large and random] sample to the population from which the sample was drawn [i.e., universalistic generalizability]); (b) internal (statistical) generalizations (i.e., making generalizations, judgments, predictions, or inferences on data obtained from one or more representative or elite study participants [e.g., key informants, sub-sample members] to the sample from which the participant[s] was selected [i.e., particularistic generalizability]); (c) analytic generalizations (i.e., -the investigator is striving to generalize a particular set of [case study] results to some broader theory‖; Yin, 2009, p. 43) and are -applied to wider theory on the basis of how selected cases ‗fit' with general constructs‖; Curtis, Gesler, Smith, & Washburn, 2000, p. 1002); (d) case-to-case transfer (i.e., making M a r c h 2 2 , 2 0 1 4 generalizations, judgments, or inferences from one case to another [similar] case; Miles & Huberman, 1994); and (e) naturalistic generalization (i.e., each reader makes generalizations entirely, or at least in part, from her/his personal or vicarious experiences; Stake & Trumbull, 1982). As noted by Onwuegbuzie et al. (2009), both qualitative and quantitative research studies, and studies utilizing mixing techniques can generate any of these five types of generalization. Further, Williams (2000) and Payne and Williams (2005) discussed the notion of moderatum generalization. In particular, Payne and Williams (2005) defined moderatum generalizations as those generalizations that resemble the modest, pragmatic generalizations drawn from personal experience which, by bringing a semblance of order and consistency to social interaction, make everyday life possible. Indeed, a strong claim can be made that in qualitative research…such moderatum generalizations are unavoidable. (p. 296) Similarly, a small sample size of key informants (e.g., n = 1) should not be used to make internal statistical generalizations to a relatively large group of sample members from which the key informant(s) were drawn. Further, a small sample size (e.g., n = 1) should not be used to make analytic generalizations about a phenomenonespecially when the goal of the interpretivist researcher is to provide an insight into an issue or to redraw a generalization, as is the case for instrumental case studies (Stake, 2005). Also, findings are not transferable (e.g., case-to-case transfer) if the interpretivist researcher has not conducted persistent observation or prolonged engagement (cf. Lincoln & Guba, 1985) such that an inadequate amount of data (e.g., interview data, observational data) has been collected, thereby leading to what Onwuegbuzie and Leech (2007) referred to as observational bias (i.e., -when the data collectors have obtained an insufficient sampling of behaviors or words from the study participant(s)‖ [p. 235]).

Prevalence of Interpretive Inconsistency in Published Studies
Unfortunately, when interpreting their findings, a significant proportion of qualitative, quantitative, and mixed researchers make inferences that represent interpretive inconsistency. In what follows, we will present studies in which the prevalence of interpretive inconsistency has been examined in studies representing all the research traditions.

Prevalence of Interpretive Inconsistency in Qualitative Research
With respect to qualitative research, Onwuegbuzie and Leech (2010) conducted a mixed research study to examine the generalization practices in all empirical qualitative research articles published in The Qualitative Report (TQR), a reputable qualitative journal, between its inception in 1990 and 2006. Of the 273 articles that were published in TQR during this period, 125 represented empirical articles. Onwuegbuzie and Leech's quantitative analysis of all 125 empirical qualitative articles revealed that a significant proportion (i.e., 29.6%) of these studies evidenced interpretive inconsistency by containing generalizations beyond the underlying sample that were made inappropriately by the author(s). Onwuegbuzie and Leech's (2010) qualitative analysis of these empirical studies identified several types of over-generalizations that occurred, which included the author(s) making general recommendations for future practice and providing general policy implications based only on a few participants.
In addition to reducing the justification that a given set of findings in a qualitative study can be subjected to one or more of the six aforementioned types of generalization (i.e., external statistical, internal statistical, analytic, case-to-case, naturalistic, moderatum)-thereby compromising external credibility (i.e., -the confirmability and transferability of findings and conclusions‖; Onwuegbuzie & Leech, 2007, p. 235), an inappropriate sampling design also unduly affects what Onwuegbuzie and Leech (2007) labeled as internal credibility, which represents the -the truth value, applicability, consistency, neutrality, dependability, and/or credibility of interpretations and conclusions within the underlying setting or group‖ (Onwuegbuzie & Leech, 2007, p. 234 Morse, 1995) will not be reached, whether data saturation (i.e., occurring when information occurs so repeatedly that the interpretivist researcher can anticipate it and whereby the collection of more data appears to have no additional interpretive value; Sandelowski, 2008) or theoretical saturation (i.e., occurring when the interpretivist researcher can assume that her/his emergent theory is developed sufficiently to fit any future data collected; Sandelowski, 2008). The lack of saturation makes it more difficult to obtain thick description (Geertz, 1973), which, in turn, makes it more difficult for knowledge construction and meaning making to occur. As an example, using data from a study wherein 60 women in two West African countries were interviewed in depth, Guest, Bunce, and , who systematically documented the degree of data saturation and variability over the course of their thematic analysis, found that saturation occurred within the first 12 interviews, with the core elements for meta-themes emerging as early as after six interviews. Apart from M a r c h 2 2 , 2 0 1 4 demonstrating the role that sample size can play in the data saturation process, these findings indicate that, for their data, saturation would not have occurred if a sample size less than six had been used. Thus, an inadequate sampling designwhich represents lack of rigor-leads to interpretive inconsistency, which, in turn, compromises the inference quality associated with a set of qualitative findings.

Prevalence of Interpretive Inconsistency in Quantitative Research
With respect to quantitative research, Onwuegbuzie and Daniel (2005) examined the generalization practices in all manuscripts submitted to Research in the Schools-a nationally/internationally peer-reviewed journal, over a 2-year period. Of the 52 manuscripts involved, nearly one third (i.e., 30.0%) evidenced interpretive inconsistency by containing generalizations beyond the underlying sample that were made inappropriately by the author(s). These authors were making external statistical generalizations in their quantitative research reports based on sample sizes as small as 20!
In addition to reducing the external validity (i.e., -the extent to which the study results can be generalized to and across populations of persons, settings, times, outcomes, and treatment variations‖; Johnson & Christensen, 2010, p. 585) of a given set of findings in a quantitative study, an inappropriate sampling design also unduly affects internal validity (i.e., the -approximate validity with which we infer that a relationship between two variables is causal‖; Cook & Campbell, 1979, p. 3). Specifically, small sample sizes result in low statistical power (i.e., the conditional probability of rejecting the null hypothesis when the alternative hypothesis is true) for conducting null hypothesis significance tests (NHSTs) (Schmidt & Hunter, 1997). Unfortunately, a substantive proportion of quantitative research studies involve the use of NHSTs with inadequate statistical power. Consistent with this assertion, Schmidt and Hunter (1997) reported that -the average [hypothesized] power of null hypothesis significance tests in typical studies and research literature is in the .40 to .60 range… [with] .50 as a rough average‖ (p. 40), which indicates that approximately one half of all statistical tests in the social and behavioral science literature will be statistically nonsignificant. As noted by Schmidt and Hunter (1997), -This level of accuracy is so low that it could be achieved just by flipping a (unbiased) coin!‖ (p. 40).Thus, an inadequate sampling design in quantitative research studies-which also represents lack of rigor-leads to interpretive inconsistency, which, in turn, compromises the inference quality associated with a set of quantitative findings.

Prevalence of Interpretive Inconsistency in Mixed Research
Qualitative research and quantitative research are not the only research traditions that induce relatively high levels of interpretive inconsistency. Some mixed research studies also possess the trait of interpretive inconsistency, which compromises the integrity of the meta-inferences and generalizations. Indeed, after searching for mixed research articles that were labeled as such by the author(s) across 15 electronic bibliographic databases (e.g., Academic Search Premier [EBSCOHost]) that represented all years for which records existed until 2006, Collins et al. (2007) identified 121 studies representing nine fields in the social or health sciences (i.e., education, nursing, social work, public health, psychology, medicine, sociology, business, library science). Collins et al. then examined these 121 mixed research studies to assess the level and nature of interpretive consistency-documenting relatively high levels of interpretive inconsistency. In particular, of the 54 studies that included a quantitative and/or a qualitative phase involving a sample size of 30 or less participants, 53.7% contained meta-inferences that represented inappropriate external statistical generalizations-yielding interpretive inconsistency. Collins et al.'s (2007) data also revealed that 24.8% of the mixed research studies contained sample sizes for the quantitative phase(s)/component(s) that lacked sufficient statistical power (i.e., < .80) to conduct a NHST. Although the authors of some of these studies, nonetheless, did document statistically significant findings, it is clear that, in most cases, the sample size(s) provided a threat to the internal validity of the findings, as well as to the external validity. Thus, an inadequate sampling design in mixed research studies-which also represents lack of rigorleads to interpretive inconsistency, which, in turn, compromises the meta-inference quality associated with the combined set of qualitative and quantitative findings.

Major Purpose of Article
When making inferences in mono-method research (i.e., quantitative research or qualitative research), one of the six aforementioned types of generalizations typically is used. However, when quantitative research approaches and qualitative research approaches are used within the same study-as is the case when mixed research studies are conducted (Johnson, Onwuegbuzie, & Turner, 2007)-then, more than one of these types of generalizations often is involved, increasing the complexity of the inferences made. For example, the quantitative findings might yield external (statistical) generalizations, assuming that the underlying sample is representative of the population to which the generalizations are made; and the qualitative findings might yield internal (statistical) generalizations, analytic generalizations, or case-to-case transfer, depending on the goal of the qualitative phase(s)/component(s). These two sets of inferences yielded by the quantitative and qualitative findings then would be combined into meta-inferences (i.e., a set of inferences that represent a coherent whole; Tashakkori & Teddlie, 2003), which typically involve making more complex generalizations than is the case for mono-method studies, especially when it involves combining inferences from different sub-samples with different sample sizes whose sub-samples were selected via different sampling schemes. As such, Onwuegbuzie and Johnson (2006) identified as one of their nine major threats to legitimation (i.e., the degree that integration of findings leads to credible and defensible meta-inferences) in mixed research studies what they called sample legitimation integration, which represents the extent to which the relationship between the qualitative and quantitative sampling designs yields quality meta-inferences. A pivotal component of this process is the selection of the sampling design. M a r c h 2 2 , 2 0 1 4 As contended by Payne and Williams (2005), when designing their studies, researchers -should plan for anticipated generalizations…and generalization[s] should be more explicitly formulated within a context of supporting evidence‖ (p. 295) [emphasis in original]. Indeed, planning for anticipated generalizations should help researchers avoid making generalizations that reflect interpretive inconsistency. With this in mind, in what follows, we provide a meta-sampling framework for enhancing inference quality in mixed research. This meta-framework is structured in accordance to the dimensions of Bronfenbrenner's (1979) ecological systems model, and these dimensions are juxtaposed to the types of generalizations and sampling criteria (Collins, 2010) pertinent to mixed research.

Toward a Sampling Meta-Framework for Enhancing Inference Quality in Mixed Research
Recently, Onwuegbuzie, Collins, and Frels (2013) provided a framework for designing and categorizing qualitative, quantitative, and mixed research studies that utilizes Bronfenbrenner's (1979) ecological systems theory. As outlined by Onwuegbuzie Bronfenbrenner's (1979) four levels that they coined as micro-research studies (i.e., Level 1: research wherein one or more persons or groups are studied within his/her/their immediate environment[s]), meso-research studies (i.e., Level 2: research wherein one or more persons or groups are studied within other systems in which the he/she/they spends time), exo-research studies (i.e., Level 3: research wherein one or more persons or groups are studied within systems by which the he/she/they might be influenced but of which he/she/they is not directly a member), and macro-research studies (i.e., Level 4: research wherein one or more persons or groups are studied within the larger cultural world or society surrounding him/her/them). A microresearch, meso-research, exo-research, and macro-research study each can be represented by the qualitative, quantitative, or mixed research tradition, depending on elements such as the philosophical assumptions and stances underlying the research study, research goal, research objective, research questions, the type of data collected, and, very importantly, the sampling design. However, mixed research studies are more likely to involve phases that represent different levels. This likelihood is actualized by a recent study designed by Kohler's (2011), who investigated the role of school size in incidents of violence among Texas middle schools. The study's design involved a quantitative phase, whereby Kohler examined the relationship between the incidents of school violence and the size of middle schools among all 842 middle schools in the state of Texas)-representing macro-research (i.e., Level 4). The qualitative phase involved studying the perceptions of school violence of seven principals from different-sized middle schools to ascertain whether they believe school size plays a role in the number of incidents of school violence, as well as whether their perceptions align with the quantitative discipline data related to fighting, assaults, and aggravated assaults-representing micro-research (i.e., Level 1).
Onwuegbuzie et al.'s (2013) re-conceptualization has important implications for generalization because it helps the researcher bound the inquiry with respect to the generalizability of the findings. Indeed, Onwuegbuzie et al. differentiated what they termed as within-level generalization from across-level generalization. Within-level generalization represents one of the six types of generalizations of the findings being made within the same level (i.e., micro, meso, exo, macro) and across-level generalization, stemming from mixed research studies, wherein one of the six types of generalizations of the findings from one research level (e.g., Level 1) is combined with one of the six types of generalizations of the findings from another research level (e.g., Level 4) to yield meta-inferences-as was the case in Kohler's (2011) study.

Sampling Considerations for Selection of Generalizations
In this article, we build on Onwuegbuzie et al.'s (2013) framework by outlining six considerations that we suggest quantitative, qualitative, and mixed researchers alike could make at the research conceptualization stage of their research studies in order to facilitate decisions pertinent to selecting appropriate generalizations (i.e., enhance internal validity/credibility and external validity/credibility). These six considerations are labeled as probabilistic orientation, emtic orientation, abductive orientation, intrinsic versus instrumental orientation, particularistic versus universalistic orientation, and philosophical clarity.

Probabilistic Orientation Consideration
In the prevalence rate studies discussed earlier, compelling evidence of interpretive inconsistency was documented in a significant proportion of these empirical research articles. It appears that researchers who make interpretive inconsistent generalizations do not realize the influential role that probability plays in the generalization process. Yet, each of the six generalization types represent an adequate level of confidence that the phenomenon captured by the underlying data is M a r c h 2 2 , 2 0 1 4 both real-in the subjective, objective, or intersubjective sense-and stable (e.g., reliable, trustworthy) for the study participants from whom the data are collected, and, in the case when external or internal statistical generalizations are made, for people other than these study participants. For example, in the case where saturation is reached (and a researcher surely would have less confidence in making inferences based on one 1-minute interview than on several interviews each lasting for more than 1 hour), the researcher has confidence that the data have occurred sufficiently repeatedly for an analytic generalization to be made. However, confidence is based on some form of probability-whether it reflects the theoretical method of assigning probabilities (i.e., via p values), the relative frequency method of assigning probabilities (i.e., via internal replication [e.g., bootstrapping or jackknife techniques, cross-validation]), or-as is typically the case when interpreting qualitative findings-the subjective method of assigning probabilities (i.e., subjective probabilities). This method of assigning probabilities reflects the researcher's personal judgment about how likely a particular finding is stable for the participant(s), but rather than being based on any precise computation, it represents a reasonable assessment by the researcher or by the researcher and participant(s), in the case where knowledge is coconstructed. Simply put, external statistical generalizations should be based, at least to some degree, on either theoretical probability or empirical probability, whereas analytic generalizations, case-to-case transfer, and naturalistic generalizations should be based on subjective probability. Thus, awareness of one's probabilistic orientation likely would prevent a mixed researcher, for example, from making external statistical generalizations based on subjective probability.

Emtic Orientation Consideration
Recently, Onwuegbuzie (2012) coined the phrase emtic perspective or orientation to denote the point at which emic and etic viewpoints are maximally interactive. In terms of generalization, an emtic orientation is most likely to yield meta-inferences that reflect a combining of external statistical generalization(s) with other qualitative-based generalizations such as analytic generalization. Such an orientation is more likely to enhance what Onwuegbuzie and Johnson (2006) refer to as inside-outside legitimation, which represents the extent to which the mixed researcher accurately presents and appropriately incorporates the insider's view and the observer's view for purposes such as understanding, description, and explanation. Thus, awareness of one's emtic orientation likely would prevent a mixed researcher from making interpretive inconsistent generalizations.

Abductive Orientation Consideration
Abductive logic or reasoning involves oscillating between inductive and deductive logic (Morgan, 2007). Thus, determining the type of logic that would underlie a mixed research study (i.e., inductive vs. deductive vs. abductive) helps the mixed researcher make sampling design decisions because, for example, using abductive logic typically would necessitate a larger sample size for the quantitative phase or theoretical sampling (i.e., process of collecting more data from the same or different participants to gain a deeper understanding of participants who have already been selected) for the qualitative phase than would using inductive or deductive logic. Thus, awareness of one's abductive orientation likely would prevent a mixed researcher from making interpretive inconsistent generalizations.

Intrinsic Versus Instrumental Orientation Consideration
An important decision for the mixed researcher to make during the conceptualization stage pertains to the purpose for selecting participants. Thus, the mixed researcher should decide (a) whether all participants in the quantitative and qualitative phases are selected intrinsically, whereby the purpose is to obtain a rich understanding of each target (e.g., illustrative, deviant) participant, such that each participant is selected because in all its particularity and ordinariness, he/she is of interest (Stake, 2005); (b) whether all participants in the quantitative and qualitative phases are selected instrumentally, whereby, as noted previously, the purpose is to provide an insight into an issue or to redraw a generalization; or (c) a combination of intrinsic and instrumental selection is involved. Such a decision would help the mixed researcher select an appropriate sampling design by determining an appropriate sample size (e.g., small sample sizes for intrinsic selection), sampling scheme (e.g., selection of representative samples, although not necessarily large samples, for instrumental selection), and number of observational units per participant such that some form of saturation is reached with respect to the qualitative data (i.e., within-case saturation for intrinsic selection and cross-case saturation for instrumental selection4). Thus, awareness of one's intrinsic versus instrumental orientation likely would prevent a mixed researcher from making interpretive inconsistent generalizations.

Particularistic Versus Universalistic Orientation Consideration
As noted previously, another important decision for the mixed researcher to make during the conceptualization stage is whether the study should yield generalizations that are used to describe phenomena that occur, or have occurred, in a specific context (particularistic), whether the study should yield generalizations that are used to describe phenomena that can be applied to broad contexts (universalistic), or some combination of both. Such a decision would help mixed researchers select an appropriate sampling design that, in turn, yields interpretive consistent generalizations. Thus, awareness of one's particularistic versus universalistic orientation likely would prevent mixed researchers from making interpretive inconsistent generalizations.

Philosophical Clarity Consideration
Collins, Onwuegbuzie, and Johnson (2012) introduced the notion of philosophical clarity, which they conceptualized as a quality criterion that represents the degree to which -the researcher is aware of and articulates her/his philosophical M a r c h 2 2 , 2 0 1 4 proclivities in terms of philosophical assumptions and stances in relation to all components, claims, actions, and uses in a mixed research study‖ (p. 855). According to Collins et al. (2012), philosophical clarity drives a researcher's choice of paradigm, which, in turn, impacts the decisions and actions made by a mixed researcher. Lack of philosophical clarity has the potential to affect inference quality. In particular, mixed researchers who do not have adequate philosophical clarity are likely to be undecided about what constitutes an appropriate generalization for which to strive, which is a function of several research components that include the following: (a) the mixed research goal (i.e., predict; add to the knowledge base; have a personal, social, institutional, and/or organizational impact; measure change; understand complex phenomena; test new ideas; generate new ideas; inform constituencies; or examine the past; Newman, Ridenour, Newman, & DeMarco, 2003); (b) the consumers (i.e., audience) of the mixed research study (e.g., stakeholders); (c) the mode(s) of presenting the findings (e.g., dissertation/thesis, journal article, technical report, book, book chapter, conference paper, YouTube video); (d) probabilistic orientation (as discussed previously); (e) emtic orientation (as discussed previously); (f) abductive orientation (as discussed previously); intrinsic versus instrumental orientation (as discussed previously); and (g) particularistic or universalistic orientation (as discussed previously). The five criteria comprising Collins's (2010) integrative sampling typology facilitate mixed researchers' efforts to clarify their sampling decisions when conceptualizing the design and implementing procedures to collect and to analyze data, and to report the study (e.g., journal articles). There is an interactive relationship existing among the five criteria, and when these criteria are applied to the conduct of a mixed research study, the likelihood of the mixed researcher achieving interpretive consistency in the interpretive phase of the study is elevated. Specifically, the typology focuses attention on the relationship of the samples in terms of time orientation of the phases (concurrent and sequential) (Criterion 1); relationship between the samples (i.e., identical, parallel, nested, multilevel) in terms of sample selection (Criterion 2); type of generalization selected by the mixed researcher (Criterion 3); types of data collected (quantitative, qualitative, transformed) in response to the research question (Criterion 4); and emphasis of approach (dominant, dominant-less, equal) in terms of the mixed researcher formulating credible meta-inferences and applicable generalizations (Criterion 5).
Moreover, to reconcile interpretive inconsistencies, the mixed researcher can strategize a -representativeness/saturation tradeoff‖ (Teddlie & Tashakkori, 2009, p. 184) approach. Depending on the type of data collected and the selected generalization, the mixed researcher can make an informed decision to place more emphasis on data collected in one phase or, in situations when collecting one form of data and conducting cross-over analyzes (Onwuegbuzie & Combs, 2010), place more emphasis on one form of analyses when determining conclusions and meta-inferences. The tradeoff reflects the degree that these decisions focus on the representativeness (quantitative component) or conversely on saturation (qualitative component). Figure 1 illustrates our proposed sampling meta-framework. Contained in this figure are the four levels pertaining to Bronfenbrenner's (1979) ecological systems model, which we believe should help contextualize the decisions that the mixed researcher makes to apply the ensuing generalizations to the findings. Also, in this table are the six sampling considerations (i.e., probabilistic orientation, emtic orientation, abductive orientation, intrinsic vs. instrumental orientation, particularistic vs. universalistic orientation, and philosophical clarity). It should be noted that any of the six generalization types can be made at any of the four levels of research, depending on these six considerations. Finally, this figure incorporates Collins's (2010) five sampling criteria applicable to the utilization of sampling designs in a mixed inquiry. Applying these criteria to the sampling process elevates the degree to which the findings are warranted and the process is transparent.

Conclusions
The role of sampling in mixed research still is an underdeveloped area, with only a handful of works published to date (i.e., Collins, 2010;Collins et al., 2006Collins et al., , 2007Teddlie & Yu, 2007). This was the motivation behind the present article. Moreover, the overall goal of this article was to provide a sampling meta-framework for enhancing inference quality in mixed research that is driven by Bronfenbrenner's (1979) ecological systems theory. We contend that our meta-framework is flexible because it can be utilized by both qualitative researchers and quantitative researchers. In any case, we hope that our meta-framework represents a step in the appropriate direction for enhancing rigor in mixed research studies (as well as in qualitative research and quantitative research studies), thereby reducing the incidence of interpretive inconsistent generalizations being made, and, ultimately, enhancing understanding (i.e., increased Verstehen) of phenomena. In fact, we contend that attaining these goals is not merely an issue of rigor but, more importantly, is an issue of ethics, and is dependent on the values of the mixed researcher as exemplified by the researcher's decisions made throughout the mixed research process. M a r c h 2 2 , 2 0 1 4