top of page
Search

Upward Bullying in the Workplace: A Phenomenological Study

Bolling – Volume 10, Issue 1 (2019)

e-Journal of Social & Behavioural Research in Business

Vol. 10, Iss. 1, June 2019, pp: 61-77.

”http://www.ejsbrb.org”

Upward Bullying in the Workplace: A Phenomenological Study

Jerry A. Bolling

School of Advance Studies

University of Phoenix

Upper Marlboro, MD 20772 USA

Email: drjabolling@gmail.com

Abstract

Purpose: The purpose of this paper is to explore and describe 14 enrolled agent (EA) managers’ perceived lived experiences of upward bullying (UB) by subordinates.

Design/ methodology/ approach: The qualitative interpretive phenomenological analysis approach is used in this paper to provide a rich and robust hermeneutic analysis of the perceived lived experiences for EA managers.

Results/ findings: The conversion of the raw data (i.e., analysis, interpreting, coding, horizontalizing, & clustering the horizons) into a coherent textual description of the phenomenon produced five themes of workplace bullying (WB), upward bullying (UB), antecedents of bullying, resources, and prevention.

Key words: Enrolled agent; phenomenology; status in organization and management theory; upward bullying; workplace bullying.

JEL Classification: C61; J24; M12

PsycINFO Classification: 3620

FoR Code: 1401; 1503

ERA Journal ID#: 123340

© e-JSBRB Vol.10, Iss.1 (2019) 61

Bolling – Volume 10, Issue 1 (2019)

Introduction

Upward bullying (UB) is not new to the lexicon of harassment in the workplace. It is often misunderstood and unrecognizable as a viable term and construct under the harassment banner by the masses. For instance, some men and many women described to their loved ones a sickening feeling they experienced in the workplace at the hands of an unrequested lover. As such, sexually harassed workers were unable to explain the magnitude and psychological impact the encounters had on their abilities to cope with these unknown phenomena of job pressures and job security. Once the actions and actors of the then unknown phenomenon of sexual harassment was named, defined, and explained the experiences and overtures initiated by an unrequested lover in the workplace was clearly understood by others.

Likewise, UB parallels sexual harassment in the workplace. In the sense, many men and women in managerial roles described to their loved ones a sickening feeling they experienced in the workplace at the hands of their undermining and underperforming subordinates. Much like sexually harassed workers, the bullied manager is unable to explain the magnitude and psychological impact the encounter had on his or her abilities to cope with these unknown phenomena of job pressures and job security. Thus, until the actions and actors of upward bullying is fully explored and described like its harassment counterparts: sexual harassment, school bullying, and cyber bullying. UB will continue to be an unrecognizable component of bullying behaviour in the workplace.

Upward Bullying


Instinctively, the narrative for bullying of any kind (e.g., cyber, school, & work) is often reported as a powerless individual falling prey to a powerful individual (Patterson, Branch, Barker, & Ramsay, 2018). In the workplace this power imbalance is often viewed as a manager or an individual with authority bullying staff members with little to no authority in the organization. The dynamics of UB in the workplace differs from the traditional views of workplace bullying (WB). Branch, Ramsay, and Barker (2006, 2007) noted UB is the act of a subordinate staff member inflicting intentional or unintentional bullying behaviours toward his or her manager. Birks, Budden, Stewart, and Chapman (2014), Branch et al. (2006, 2007), and Wallace, Johnston, and Trenberth (2010) agreed components of UB is passive-aggressive in nature, which includes refusing to accept or complete work assignments, missing work, showing up late for work and meetings, spreading professionally damaging rumours, withholding information, and false claims of work persecution in attempts to avoid disciplinary actions, and sabotaging the manager.

In a 2017 survey study, The Workplace Bullying Institute (WBI) approximated of the 161,616,000 domestic workforce positions in the United States: 37% were unaware of bullying, 25% had no experiences with bullying, 19% witnessed bullying, 10% suffered some form of bullying, and 9% acknowledged currently being bullied (Namie, 2017). Additionally, the survey cited 70% males and 30% females as the perpetrators of bullying, 61% of the bullies being managers, and 63% acted alone, with 6% bullying the manager (Namie, 2017). Wallace et al. noted in their 2010 investigative study ‘Bullying the boss: The prevalence of upward bullying behaviours’, 70.6% of the respondents (n=491) indicated subordinate-initiated bullying behaviour toward the managers.

UB is often a silent but toxic form of WB, whereas, most managers will not report being bullied by staff for fear of receiving the professional criticism of ‘being soft’ or ‘a pushover’ manager (Birks et al, 2014). In addition to a manager staying silent, Birks et al. (2014) believed UB often thrives in the workplace when an organization struggles with dynamic changes of personnel, downsizing, and mergers. However, changes set aside, Birks et al. (2014) believed the real problem with UB is a lack of support from senior leadership; thus, leaving the manager vulnerable to visceral attacks by subordinates. For many bullied managers, UB feels and looks different to what he or she recognizes as WB with no one or nowhere to turn for relief or support (Birks et al., 2014).

© e-JSBRB Vol.10, Iss.1 (2019) 62


Bolling – Volume 10, Issue 1 (2019)

Managers bullied in the workplace by their subordinates are a growing group and merits attention (Branch et al., 2007). In their qualitative study, Branch et al. (2007) interviewed 24 managers, nine managers experiencing no issues with UB, and 15 managers having direct experiences of UB concluded organizational factors of change, environment, and power directly contributed to issues of UB. Patterson, Branch, Barker, and Ramsay (2018) agreed a manager’s dependency on his or her subordinate could tip the power scale and cause a power imbalance; thereby, resulting in UB. Patterson et al. (2018) interviewed six participants in an exploratory study, producing three major themes: a) loss of legitimate power, b) coercive power, and c) structural power contributing to the under-researched topic of UB.

The goal of this study was to build upon UB knowledge gained through prior analyses and literature research, reports, and studies providing a rich and robust hermeneutic analysis of lived experiences for enrolled agent (EA) managers. This qualitative interpretive phenomenological analysis (IPA) study focused on two semi-structured, open-ended central questions: 1) what are the lived experiences for EAs bullied by subordinate and 2) what are the lived experiences for EAs recognition of being bullied by subordinates when seeking supportive help from other managers. Schat and Kelloway (2003) noted further investigation of workplace aggression should merit the attention of the organization’s leadership, governing legislative authorities, and social researchers when an infraction of any kind is committed in the workplace against coworkers, supervisors, or management. As such, the responses from the two semi-structured, open-ended central questions might provide organizational leadership with additional insight on UB, which is not offered in the current literature.

Literature Review

Since the 1980s when Heinz Leymann introduced the phenomenon of adult bullying in the workplace (hereto referred as workplace bullying), researchers and theorists with varying academic backgrounds, disciplines, and diverse fields of studies contributed to the conceptual, contextual, and intellectual knowledge of bullying literature. Nevertheless, Hershcovis (2011) noted despite the proliferation of articles, researches, and studies on workplace aggression, incivilities, and stressors such as Einarsen (1999) WB; Einarsen and Raknes (1997) victimization of men; Emdad, Alipour, Hagberg, and Jensen (2013) witnessing bullying; Freeman (2014) adolescents bullying; Hanish et al., (2013) school bullying; Jenkins, Winefield, and Sarris (2011) accusations of bullying; Hollis (2015) cost of bullying; Hutchinson and Hurley (2013) emotional intelligence of bullying; Keashly (2001) emotion abuse and bullying; Messias, Kindrick, and Castro (2014) cyber bullying and other constructs examining the mistreatment of human capital in the workplace, bullying continues to be a major problem. Bible (2012) noted bullying in the workplace stretches across different industries of corporate, government, family-owned, and non-profit spanning geographic areas of local, national, and global, and social categories of age, health, political ideology, profession, race, religion, and sex. As such, the typology of bullying centers around three conceptual themes of cyber bullying, school bullying, and WB.

Cyber bullying is the latest construct added to the harassment umbrella with bullying behaviour toward others in society, school, and the workplace. Research studies of Burnett, Yozwiak, and Omar (2013), Gofin and Avitzour (2012), Hollis (2016), and Jacobs, Völlink, Dehue, and Lechner (2015) defined cyber bullying as the targeting of individuals or groups using a proliferation of information technologies (IT) appliances such as computers, email, or social media as a means to bully, harass, or intimidate other individuals or groups. Grundherr, Geisler, Stoiber, and Schäfer (2017) stated school bullying happens when one student or group of students bully and victimize other students. Additionally, Drydakis (2014), Homel (2013), Sansone, Lam, and Wiederman (2013) noted school bullying is a precursor to adult bullying, which is often a continuation of misbehaved, misguided, uncontrolled, and uncorrected youthful acts from the classroom spawning into the workplace as adult aggressions. As such, the destructive, insidious, and malicious antecedent behaviours that prior and new researchers found in cyber and school bullying is a constant challenge for staff and management alike, as bullying problems invaded the workplace. Bender and Lösel (2011), Bhukhanwala (2014), and Walser de Lara (2016) agreed the manifestations of school bullying over time will appear in adulthood, as anti-social behaviours considered not appropriate in mainstream societies and especially in the workforce.

© e-JSBRB Vol.10, Iss.1 (2019) 63

Bolling – Volume 10, Issue 1 (2019)

Although UB is not new, it is an under-researched topic. For instance, Branch et al. (2006, 2007) believed WB literature was missing a composite of UB agreeing their analyses of prior literature, studies, and reports on WB focused on the single construct of the victim’s accounting of the downward bullying. As such, a paradigm shift occurred within the WB literature, which included perpetrators and bystanders as participants in the bullying equation no longer relying on the accuser or targeted individual as the focal point of WB. Jenkins et al. (2011), Jenkins et al. (2012), and Rhodes et al. (2010) studies explored bullying from the perspective of the accused perpetrators and bystanders. Lian, Ferris, Morrison, and Brown (2014) agreed in their cross-lagged data study an abusive reciprocal relationship exists between subordinates and managers; whereas, the subordinate is equally responsible for bullying and in particular instances is the primary culprit of inflicting bullying tendencies within the workplace.


Wallace et al. (2010) noted there is a paucity amount of information existing in the literature regarding UB. While, Branch et al. (2007) said to understand the broad impact of WB; attention should give way to the totality of the bullying incident from the accused, accuser, and bystander at every level of input. As such, this author used search engines (e.g., Google, Google Scholar, ERIC, etc.) to produced more than 1,650,000 WB results. Interestingly enough, the search engines populated less than ten studies using ‘upward bullying’, ‘upwards bullying’, ‘bottom-up bullying’, ‘bullying the manager’, and ‘subordinate-initiated bullying’ within the descriptor’s field (See Table 1). Additionally, the use of current and historical UB and WB studies ranging from types, definitions, impact, origination, and legislation mentioned some form of WB existed during the Industrial Age and existing in the Information Age.

Table 1:

Researched Peer-Reviewed Upward Bullying Articles:

Articles Used in this ResearchTitle Author(s) Year Times CitedBullying the boss: The prevalence of upward bullying behaviours Wallace, Johnston, &Trenberth 2010 11Turning the tables: The growth of upward bullying in nursing academia Birks, Budden, Stewart, & Chapman 2014 10Causes of upwards bullying: Managers' perspectives Branch, Ramsay, & Barker 2006 12Managers in the firing line: Contributing factors to workplace bullying by staff Branch, Ramsay, & Barker 2007 104Playing with power: Examinations of types of power used by staff members in workplace bullying Patterson, Branch, Ramsay, & Barker 2018 0Articles Not Used in this ResearchUs and them: Understanding upwards bullying through the lens of social identify theory Branch, Ramsay, Sheehan, & Barker 2004 2Exploration of upward bullying: An interview study Branch, Ramsay, Sheehan, & Barker 2005 3The bullied boss: A conceptual exploration of upward bullying Branch, Ramsay, & Barker 2008 21



© e-JSBRB Vol.10, Iss.1 (2019) 64

Bolling – Volume 10, Issue 1 (2019)

Methodology

The subjective nature of qualitative research is ideal for any researcher focusing on humanistic or idealistic approaches when analyzing, describing, exploring, interpreting, or understanding the attitudes, behaviours, beliefs, interactions, and perceptions of others (Vibha, Bijayini, & Sanjay, 2013). As such, qualitative methods are non-quantitative, as its focus is to use language as a descriptive mechanism to explain a researchable phenomenon (Mohammadi, Shekari, Banar, and Ajili, 2014). Within qualitative studies, there are several different methods originating from other disciplines of studies (Connelly, 2010) such as phenomenology. Phenomenology is an embodiment of humankind’s experiences, which represents their ‘lifeworld’ through physical juxtaposition of their body, senses, and reality of assumptions (Connelly, 2010).


As a philosophical pursuit, phenomenology is the brainchild of theorist Edmund Husserl (1859-1938), the renowned father of phenomenology (Tuohy, Cooney, Dowling, Murphy, & Sixsmith, 2013). Husserl’s radical approach for phenomenology was to extract and determine what the phenomena meant to the individual (Moustakas, 1994) (i.e., ‘what does it mean to be an inmate’, ‘what does it mean to miscarry a child’, or what does it mean to be an amputee’). Husserl pursuit of phenomenology evolved from the participants’ conscious experiences of emotions, judgments, perceptions, etc. (Gill, 2014) along with their intuitive experiences of the phenomenon starting point and extracting its essential experiences and essences of what the individual experienced (VanScoy & Evenstad, 2015).

Phenomenology is broken into two widely used approaches of descriptive and interpretive analysis and at the heart of the two approaches is how a researcher decides on bracketing the collected information for analysis (Tuohy, Cooney, Dowling, Murphy, and Sixsmith, 2013). Descriptive or eidetic phenomenology describes the phenomenon through the consciousness of the participants (Jackson, Vaughan, & Brown, 2018) by describing the phenomenon’s general characteristics rather than the individual’s experiences (Giorgi, 2010). Interpretive or hermeneutics phenomenology looks “to describe, understand, and interpret participants’ experiences (Tuohy et al., 2013, p. 18). Hermeneutics research goes beyond the mere description of core concepts and essences found in descriptive analysis, as it looks for meanings embedded within the extracted data to give a composite of the participants’ common life practices (Lopez & Willis, 2004).


Population and Sample

For this study, this researcher selected the Maryland/District of Columbia Society of Enrolled Agents (MD/DC SEA) (the researcher is an active board member and treasurer) because the organization has approximately 300 members on its active roster with an estimate 50% being managers. Enrolled agents (EA) are certified tax officials licensed by the Internal Revenue Services (IRS) to assist taxpayers in their efforts to prepare and file business and personal income taxes (IRS, 2018 & National Association of Enrolled Agents (NAEA), 2018). The inclusion criterion for participation were: a) affiliation to MD/DCSEA, b) current or previous employment, c) a current or previous manager with subordinate staff members, d) beyond the age of majority (18), and e) a willingness to speak candidly, honestly, and openly about their lived experiences of being bullied by their subordinates in the workplace.

The availability of tax professionals is questionable during the Internal Revenue Service’s tax filing season, which runs January 1 through April 15 every year. As a courtesy to the EAs, this researcher waited until the end of the tax season before requesting members to join this IPA study. At the first spring meeting following the conclusion of the 2017 IRS tax-filing season, clearance from the University of Phoenix’s Institutional Review Board, and the permission of MD/DC SEA’s president, this researcher asked members of MD/DC SEA to join this IPA study. The communication secretary sent emails to the active roster of EAs stating the intentions for this study that participants would need to meet the established criterion, sign a consent form, agree to a one-hour audio recorded, face-to-face interview session, and answer 27 open-ended questions. In a 12-day span, the researcher fielded over 90 calls and emails discussing the criterion and desires of the study and whether the EAs were suitable to join the study.

© e-JSBRB Vol.10, Iss.1 (2019) 65

Bolling – Volume 10, Issue 1 (2019)

This study used the convenience sampling technique, which is a common type of sampling technique used in qualitative studies because of its inherent convenience component (Farrokhi & Mahmoudi-Hamidabad, 2012). A total of 14 EAs (nine males & five females) with education levels of two doctorates, seven masters, three bachelors, and two high school diplomas signed consent and confidentiality forms and joined this IPA study. Malterud, Siersma, and Guassora, (2016) stated there is no standard sample size or guideline for qualitative researchers to use in determining their sample sizes. Vasileiou, Barnett, Thorpe, and Young (2018) added qualitative sample sizes often tend to be small.


Brown and Kimball (2013) sampled 11 participants in their article ‘Cutting to live: A phenomenology of self-harm’ producing 11 themes. Strayhorn (2014) study on Korean American Gay men in college had a paucity amount of four participants with two themes a) ‘went to college to live out’ and b) ‘encountered gay racism/racialized homophobia’. Eleven fathers participated in Yu, Wang, Chen, and Mu (2011) study the ‘Lived experiences of fathers in cross national marriages’ producing four themes. Creswell (2013) specified the required population for a phenomenological exploration could range from three or four interviewees to hundreds sharing their lived experiences. Crouch and McKenzie (2006) added phenomenologists are becoming comfortable using smaller sample sizes for interview-based studies.


Collection and Analysis

The collection phase began with each EA receiving a copy of the semi-structured open ended questions allowing the EA an opportunity to preview the questions, give reflective thought, and possibly provide a thick and rich detail of their lived experiences of UB. The face to-face, one-on-one, and one-hour audio-recorded interviews took place in library conference rooms, private homes, private offices, and one conducted over the telephone. The meeting accommodations simulated a private and secure interview area for the EAs to speak freely. On the day of the scheduled interviews, the researcher greeted each manager cheerfully, held a refresher conversation detailing the purpose, intent, and desire for this IPA study, reviewed and signed the confidentiality and informed consent forms. On average, each interview lasted about one-hour and the EAs answered each question without any regrets or trepidations.


After the completion of the interviews, the researcher applied gender-neutral pseudonym names for added anonymity, and completed a manual word-for-word transcription of the audio-recordings. On average it took five hours to transcribe the audio-recordings into 14 single-typed Microsoft Word documents (118 pages in total). The EAs received, reviewed, and validated the transcription from their interview session for authenticity with all transcripts going uncorrected. In addition, separate file packets were created to house the hard-copy documentation (e.g., confidentiality and consent forms, questionnaire sheet, field notes, word for-word transcript, and validation email of transcript) for ease of analysis, sorting, and retrieval.

The data analysis used for this study was Moustakas (1994) modified version of van Kaam’s 1959 and 1966 method of analysis for phenomenological data, which is horizonalization, reduction of invariant constituents, clustering and thematizing invariant constituents, final identification, construct textural description, construct structural description, construct textural-structural description, and develop composite description (Moustakas, 1994, pp. 121- 122). The researcher developed initial coding concepts and themes to test against the EAs’ conversations and built a matrix of codes and themes throughout the text for comparison. The use of the NVivo matrix further assisted the researcher in arranging, labelling, and organizing the collected data; thereby, recognizing and stripping away irrelevant items, questions, statements, and topics that produced five themes (i.e., WB, UB, antecedents of UB, resources, and preventions).


Validity

In qualitative writing, there are multiple designs and approaches requiring researchers to provide succinct clarity and integrity throughout their study, which adds credibility to the researcher’s work. The credibility for qualitative study rises with an appropriateness of the literature review, instrumentation, data collection, data analysis, recommendations, and

© e-JSBRB Vol.10, Iss.1 (2019) 66

Bolling – Volume 10, Issue 1 (2019)

conclusion (Leung, 2015). As such, part of the trustworthiness of the qualitative analysis is the researcher’s transferability of raw contextual data converted into rich, thick, and vivid meaning. Phenomenology: The question of validity and reliability (2011) stated:

“It should be noted that in phenomenology, objectivity is a bias, for phenomenology suspends belief in the objective world, it studies the subject and keeps its focus on the subject alone while turning away completely from the object. So, if a researcher establishes something about the inner experience of a person, or a group of people, that conclusion cannot be validated objectively” (para. 5).


Thus, what does validity look like in a phenomenological study? Validity means what we infer or conclude from a premise, actually follows from the premise. A way to make inference of truth holding up in this phenomenological study is to assume the participants’ shared lived experiences, as known to them, are authentic, legitimate, and in full truth of their perceptions. From this researcher’s perspective, the logic or meaning derived from participants’ responses seemed plausible and logical as deductive logic is employed. In this study of UB in the workplace, theory has provided a means by which the reader can connect the topic and concomitant behaviour and make sense out of the theoretical underpinning of the study. In essence, internal validity exist because the conclusions of this study follow from the data collected and the external validity of the study suggests that there is an opportunity to generalize the findings of this study to similar situations.


Results and Emerging Themes

Five themes emerged from the data analysis of the recorded interviews and field notes 1) Workplace Bullying, 2) Upward Bullying, 3) Antecedents of Upward Bullying, 4) Resources, and 5) Prevention techniques along with sub-themes that aligned with the study’s two research questions. What are the lived experiences of managers bullied by subordinate staff members when carrying out their stated goals, mission, or vision of the organization? What are the lived experiences of managers bullied by subordinate staff members when seeking supportive help from other managers? As such, the following responses from the 14 EA managers produced five themes.


Theme 1: Workplace Bullying

Cowan (2012) noted for many organizations there are multiple definitions, descriptions, and labels generated for the term WB. Saunders, Huynh, and Goodman-Delahunty (2007) stated the average layperson holds a misrepresentation of a working definition of WB. As such, this poor representation of defining WB is a prevailing thought that aligns with previous studies (Carbo & Hughes, 2010; & NgaleIlongo, 2015) attention, discussion, and views when defining the term WB. The EAs defined (with varying degrees of success) their definition of WB proved fruitful in establishing a solid baseline understanding of WB. A large majority (79%) of EAs defined WB with clarity noting WB is an individual covertly or overtly exhibiting behaviours with an explicit goal to belittle, intimidate, or create inappropriate work behaviour against another in the workplace; whereas, a small minority (21%) of EAs misunderstood or knew about WB as a concept. Evan and Parker were incredulous to WB as a real and working composite of bullying in the workplace. Evan, “This is my first time hearing about WB.” While Parker was unable to associate behaviour in the workplace as bullying tendencies, “It’s a performance issue but I never would have thought that the behaviour would be called bullying.”

WB is an innocuous term often creating misguided thoughts of abuse, harassment, and violence existing on many school playgrounds associated with hitting, kicking, punching, pushing, and shoving. The problem with WB is defining the term and how the antecedent actions of bullying affect the overall goals of an organization. WB is not always readily apparent to the target, accuser, bystander, or in the case of this study, the EA manager charged with the professional welfare of the organization’s human capital and resources. Thus, establishing a baseline for the 14 EAs’ lived experiences of WB was vital to the evolutionary progression towards closing the gap found in upward bullying literature (See Figure 1).

© e-JSBRB Vol.10, Iss.1 (2019) 67

Bolling – Volume 10, Issue 1 (2019)

Figure 1:

Baseline Understanding of Workplace Bullying


Theme 2: Upward Bullying in the Workplace

Wallace et al. (2010) agreed UB occurs when a subordinate staff member inflicts intentional or unintentional acts of bullying towards his or her manager. Theme 2 allowed the EAs to explore beyond their knowledge and awareness of WB and delve into a new construct of counterproductive workplace behaviours known as UB. It was important to have the EAs define UB, explore their awareness of being a target, and any associated emotional awareness of being bullied by subordinates. Twenty-one percent of the EAs were able to provide a clear, succinct, and irrefutable definition of UB. Kerry, “It is pretty descriptive from the title…an individual bullying the supervisor.” Lee, “A manager being undermined by employees.” A slightly higher rate (36%) of EAs was unaware, unsure, and unable to define UB and conversely recognize the actions of their subordinates as bullying. Bailey, “This is a very new concept”, Hunter, “I heard of bullying before, not UB”, and Kerry, “I have never heard that [term] before” aligns with prior studies indicating a paucity amount of information on UB, thus, translating into the EAs lack of definable awareness of the term UB.

As such, having the EAs define, explore, and recognize UB from an emotional state of being the ‘Target’ allowed the researcher to construct a line of demarcation between accusations of WB and being the target of bullying in the workplace (See Figure 2).

© e-JSBRB Vol.10, Iss.1 (2019) 68

Bolling – Volume 10, Issue 1 (2019)

Figure 2:

The Line of Demarcation between Workplace Bullying and Upward Bullying

UB is an innocuous term often creating misguided thoughts of managers applying undue, unnecessary, and unwarranted abuse against their staff. Birks et al. (2014) and Branch et al. (2006, 2007) rationalized those misguided assumptions arises because there is not an adequate amount of information existing in the literature regarding the phenomenon. Thus, establishing the line of demarcation between WB and UB aligns with the knowledge building required for the evolutionary progression towards closing the gap found in upward bullying literature.

Theme 3: Antecedents of Upward Bullying

Occasionally, the interdependency relationship of the manager-subordinate becomes blurred when the manager relies on the subordinate’s expertise, knowledge, and skill to achieve the goals, mission, and vision of the organization. As such, Wallace et al. (2010) asserted the subordinate-manager creates an interdependency relationship of undue access, manipulation, and withholding of information from the manager is a form of UB. Francis added being a younger supervisor empowered a subordinate to use their years of experiences and service as a form of intimidation. Educational status in the workplace creates lines of demarcation in terms of authority, assignments, income, etc. Blair witnessed educated subordinates not adhering to the instructional commands of a non-educated superior. Wollard (2011) wrote a point of

© e-JSBRB Vol.10, Iss.1 (2019) 69

Bolling – Volume 10, Issue 1 (2019)

disengagement begins with an employee’s perceived self-actualization that threatens his or her emotional, mental, and physical work life, which spills into what Tyler and Blader (2000) noted as counterproductive workplace behaviours (CWB) (i.e., aggressive tones, body language, harassing emails, & rumours).

Whether the disengagement starts with manifestation of health related issues, non supportive language, body language, or other CWB actions those actions often produce warning signs that are indicative of the antecedents aligned with disengaged employee theory (Tyler & Blader). Casey, “It’s not vulgarity, it’s just disrespectful tone, it’s loud, its disruption.” Furthermore, many managers come from family oriented backgrounds prewired with emotional triggers, which could be manipulated. Evan remembered a subordinate using emotional triggers from prior conversations when the two were contemporaries stating, “You use to share everything with me now you are one of them”. Other EAs experienced the filing of frivolous EEO claims, harassment charges, and requesting union representation as antecedents of bullying. Hunter, “He just say those things and walk out and filed all types of things up the wazoo; complaints, lawsuit, come after my name, defamation of character, and you know all types of things”. Similar to Themes 1 and 2, Theme 3 allowed the EAs to move beyond their limited perceived lived experiences of being bullied to understanding the interrelatedness and connectivity of the theories between the status in the organization and management theory, counterproductive workplace behaviour theory, disengaged employees theory, and other antecedents and theories of UB (See Figure 3).

Figure 3:

Antecedents and Theories of Upward Bullying


Theme 4: Resources

Managers are often put in unattainable, uncomfortable, and unwinnable situations making it easy to understand why UB goes underreported (Birks et al., 2014). The aftermath for most managers unwilling to ask for help is the fear of appearing weak could hamper their ability to effectively manage the finance, human capital, and operational resources. Bohns and Flynn (2009) reported managers oftentimes have problems seeking help or involving others in workplace issues because it is awkward and uncomfortable. When asked if seeking help from another manager was a viable option. The EAs split, 50% sought aid, encouragement, or relief while 50% believed nothing could be done or any resources where available. Blair, “I am unaware of any colleague, senior manager, or agency’s entity available, able, or willing to address the needs of managers”. Casey rationalized asking for guidance indicated a flaw in the mechanics of managing the project. Kerry believed the actions of the subordinates did not

© e-JSBRB Vol.10, Iss.1 (2019) 70

Bolling – Volume 10, Issue 1 (2019)

affect the unit to the degree of requesting additional resources, therefore, “I did not ask for help”. Morgan felt the agency lacked the resources to combat UB, “They don’t give no classes on what you talking ‘bout”.

Addison, “The agency’s resources didn’t establish what avenues, protocols, or requirements to initiate when dealing with THE BULLY” (air quotes used). The preferred guidance among the EAs was support in the form of communication, training, and united protocols with reasonable support mechanics. Bonaccio, O’Reilly, O’Sullivan, and Chiocchio (2016), DeIuliis (2016), and Delcambre (2010) agreed communication and training are active ingredients to creating a healthy organization. Hunter believed communication and some level of handholding would soothe the difficulties of dealing with bullies. Addison, Alex, Bailey, Dana, and Evan maintained the preferred guidance or resources required for them in quailing issues of UB is training, “I would prefer some type of training class dealing with the real issues supervisors face daily”. Theme 4 allowed the EAs to apply ideations of managerial support and resources was vital to the evolutionary progression towards closing the gap found in upward bullying literature (See Figure 4).

Figure 4:

Managers’ Awareness of Available Resources

Theme 5: Prevention

Wiedmer (2011) opined the practices of WB are preventable and the keys to winning this insufferable and malicious practice rest with executive managers and the establishment of appropriate policies, practices, and proper enforcement mechanisms, which is a prevailing thought of previous WB prevention recommendations of Keller, Budin, and Allie (2016) and Strandmark and Rahm (2014). Theme 5 explored preventable actions, mechanisms, and techniques of solving UB (See Figure 5). The EAs (71%) agreed communication could be an effective tool in thwarting the actions of UB. Lee, “If upward bullying becomes a known behaviour that is acknowledged and some measures being put in place…it would occur less”. Casey, “I think its awareness of the conversations that are happening in the organization and when you need to as managers not be afraid of stepping in and de-escalate a situation”. Blair, “I think managers could just make everyone aware UB does exist.”

© e-JSBRB Vol.10, Iss.1 (2019) 71

Bolling – Volume 10, Issue 1 (2019)

Figure 5:

Organizational Prevention Techniques


The EAs (86%) believed their respective organizations could employ educational and training tools to solve the problems of UB. When training the workforce, Addison and Morgan verbalized UB should move from the shadows of obscurity to the forefront of the workplace lexicon. Addison paralleled the issue of UB to the previous civil liberties campaign, which implemented intervention-prevention protocols to change the ideations of gender, preference, and sexual orientation in the workplace. Morgan likened UB to younger siblings bullying older siblings adding organizations could use a similar ‘Stop the Bullying’ ground roots campaign of communication, education, and training mechanisms enacted through schools, media, and the Internet. Theme 5 allowed the EAs to explore ideations of preventable self-awareness, managerial, and organizational mechanisms was vital to the evolutionary progression towards closing the gap found in upward bullying literature.

Discussion

Beyond the fact that UB is not new to the lexicon of harassment in the workplace, most WB literature focuses on downward (manager to staff) bullying. On the other hand, Branch et al., (2006, 2007) contrasted UB as an understudied phenomenon in workplace literature. A problem for 79% of EAs was their inabilities to properly recognize, define, and understand UB. UB is not only silent, oftentimes it is a numbing sensation of failures and self-doubting behaviour because managers are unaware of the antecedents and related theories of UB (See Figure 3), which is easily misunderstood as disgruntle, incompetent, and malcontent employees with bad attitudes and improper work ethics. Anecdotally and metaphorically managers subjected to UB existentially violates the fourth rung of Maslow’s hierarchy of self-esteem needs of achievement, confidence, and respect from others.

The real problem with UB is its easy to understand why UB goes underreported because managers are often put in unattainable, uncomfortable, and unwinnable situations as managers do not want to appear weak, soft, or a pushover (Birks et al., 2014) a belief is grounded in prior studies. The analysis, findings, and results of this IPA study revealed 50% of the EAs were unable and unwilling to seek assistance because they were embarrass, the situation was unique, or have unsupportive leadership. The analysis indicated UB is a solvable problem; whereas, 71% of the EAs agreed the implementation of a viable communication, education, and training plan could thwart CWB issues that contribute to UB when organizational leaders take a supportive and active role in their human capital workforce.

Recommendations and Future Studies

The analysis, findings, and results of this IPA study indicated the presence of UB by subordinate staff members in the workplace does exist. As such, this researcher recommends leadership develop a knowledge based UB Cycle (See Figure 6) encompassing the following areas of defining and understanding the terms WB and UB, establish a demarcation line between the

© e-JSBRB Vol.10, Iss.1 (2019) 72

Bolling – Volume 10, Issue 1 (2019)

terms, identify antecedents of UB, dedicate resources, and build training programs to solve issues of UB.

Figure 6:

The Upward Bullying Cycle

Workplace

Bullying

Prevention

Resources

Upward

Bullying

Antecedents

of Upward

Bullying

An important step for most leadership beyond establishing a zero tolerance level is to recognize and understand UB is real, take it seriously, and act responsively and swiftly. Leadership might establish anti-WB governances, protocols, and regulations codifying a particular code of conduct amplifying the organization’s stance of instituting a zero tolerance towards UB in particular and WB in general. Leadership could establish an annual UB training course for their workforce, which might coincide with expectations of their established anti-WB governances, protocols, and regulations.

Future researchers could adopt and emulate this researcher’s methodology, design, questions, population, and sampling techniques to strengthen the analysis of managers bullied by subordinates. Within the WB literature, the social phenomenon of WB expanded from the victims’ accounting of bullying to other participants such as the accused and bystanders. As such, future studies involving UB in the workplace would need to move beyond the managers’ narratives of being bullied; insomuch as, exploring why subordinates bully their managers, other antecedents of upward bullying, and the impact (ineffectiveness) on the culture and organization. Additionally, future studies might explore the impact of the organization’s psychological, psychosocial, and physical wellbeing when implementing intervention-prevention protocols as a deterrent to UB.

Limitations and Delimitations

A limitation existed for not selecting a quantitative or mixed method research study over the chosen qualitative method. Ingham-Broomfield (2015) noted using the word quantitative in its own right conjures thoughts of amounts, counting, or measurable metrics. As such, the researcher explored the conceptual, contextual, and intellectual description of 14 EA managers bullied by subordinate staff. A limitation exists after multiple attempts to recruit more participants; the researcher garnered 14 EA managers to give descriptions of their lived experiences of being bullied. Thus, generalization of the findings for this study presents a limitation.

Conversely, delimitation existed in the selection of 14 EAs using the convenience sampling technique. A larger group of managers, a higher quantity range, or the use of the

© e-JSBRB Vol.10, Iss.1 (2019) 73

Bolling – Volume 10, Issue 1 (2019)

snowball sampling would suffice by producing additional collected data, which would have amplified the findings. In retrospect, additional research questions could have facilitated more data collection exploring experiences, memories, narratives, perceptions, and stories of upward bullying in the workplace. A final delimitation for this IPA study was the selected locale for this study as it was limited to the metropolitan area of the District of Columbia.

Conclusion

The purpose of this qualitative IPA study was to explore and describe 14 EA’s lived experiences of UB in the workplace by subordinate staff members and build upon the upward bullying knowledge gained through extensive literature research, previous research analyses, reports, and studies; thereby, providing a rich and robust hermeneutic analysis. The analysis, findings, and results of the EAs’ lived-experiences allowed for an authentic conversion of the raw data (i.e., analysis, interpreting, coding, horizontalizing, and clustering the horizons) into a coherent textual description of the phenomenon producing five themes of WB, UB, antecedents of UB, resources, and prevention.

Upward bullying is organic in nature, perceptional to those it harms, and subjective for others who encounter the actors and actions of the phenomenon. The prevailing thought for many of the female EAs and two males expressed their pre-understanding of UB was “I don’t get bullied”. However, their post-understanding of UB led to a self-discovery and overpowered realism the EAs were in fact victims of UB and expressed frustration over their gullibility and naiveté for not recognizing and mislabelling the tendencies of UB. Finally, a prevailing belief is this study would add to the paucity amount of UB literature; thereby, allowing future researchers to transfer their conceptualization, contextualization, and intellectualization to the phenomenon of UB in the workplace.

References

Bender, D., & Lösel, F. (2011). Bullying at school as a predictor of delinquency, violence and other anti social behaviour in adulthood. Criminal Behaviour and Mental Health: CBMH, 21(2), 99-106. doi:10.1002/cbm.799

Bhukhanwala, F. (2014). Theater of the oppressed in an after-school program: Middle school students' perspectives on bullying and prevention. Middle School Journal, 46(1), 3-12.

Bible, J. D. (2012). The jerk at work: Workplace bullying and the law's inability to combat it. Employee Relations Law Journal, 38(1), 32-51.

Birks, M., Budden, L. M., Stewart, L., & Chapman, Y. (2014). Turning the tables: the growth of upward bullying in nursing academia. Journal of Advanced Nursing, 70(8), 1685-1687.

doi:10.1111/jan.12317

Bohns, V. K., & Flynn, F. J. (2009). “Why didn’t you just ask?” Underestimating the discomfort of seek-help. Journal of Experimental Social Psychology, 46 (2010), 402-409. doi:10.1016/j.jesp.2009.12.015

Bonaccio, S., O’Reilly, J., O’Sullivan, S. L., & Chiocchio, F. (2016). Nonverbal behavior and communication in the workplace: A review and an agenda for research. Journal of Management, 42(5), 1,044- 1,074. doi:10.1177/0149206315621146

Branch, S., Ramsay, S., & Barker, M. (2006). Causes of upwards bullying: Managers’ perspectives. In proceedings of the twentieth ANZAM conference management: Pragmatism, philosophy, and priorities, Central Queensland University: Queensland, Australia.

Branch, S., Ramsay, S., & Barker, M. (2007). Managers in the firing line: Contributing factors to workplace bullying by staff - an interview study. Journal of Management and Organization, 13(3), 264-281.

Brown, T. B., & Kimball, T. (2013). Cutting to live: A phenomenology of self-harm. Journal of Marital and Family Therapy, 39(2), 195-208. doi:10.1111/j.1752-0606.2011.00270.x

Burnett, Amy L., Yozwiak, J. A., & Omar, H. A. (2013). Cyber bullying: A brief review. International Journal of Child and Adolescent Health, 6(4), 465-468.

Carbo, J., & Hughes, A. (2010). Workplace bullying: Developing a human rights definition from the perspective and experiences of targets. Working USA, 13(3), 387-403.

© e-JSBRB Vol.10, Iss.1 (2019) 74

Bolling – Volume 10, Issue 1 (2019)

Connelly, L. M. (2010). What is phenomenology? Medsurg Nursing, 19(2), 127-128.

Cowan, R. L. (2012). Defining workplace bullying from the human resource professional’s perspective. Management Communication Quarterly, 26 (3), 377-403. doi:10.1177/0893318912439474

Creswell, J. W. (2013). Qualitative inquiry & research design: Choosing among five approaches (3rd ed.). Thousand Oaks, CA: Sage.

Crouch, M., & McKenzie, H. (2006). The logic of small samples in interview-based qualitative research. Social Science Information, 45(4), 483–499.

DeIuliis, D. (2016). Workplace communication. Communication Research Trends, 35(1), 3.

Delcambre, P. (2010). Written and oral communication in the workplace – Deployment, stabilized forms of interactions, and workload: An organizational approach. Management Communication Quarterly, 24(4) 635-642. doi:10.11770893318910380374

Drydakis, N. (2014). Bullying at school and labour market outcomes. International Journal of Manpower, 35(8), 1185-1211. doi:10.1108/IJM-08-2012-0122

Einarsen, S. (1999). The nature and causes of bullying at work. International Journal of Manpower, 20(1), 16-27.

Einarsen, S., & Raknes, B. I. (1997). Harassment in the workplace and the victimization of men. Violence and Victims, 12(3), 247-63.

Emdad, R., Alipour, A., Hagberg, J., & Jensen, I. B. (2013). The impact of bystanding to workplace bullying on symptoms of depression among women and men in industry in sweden: An empirical and theoretical longitudinal study. International Archives of Occupational and Environmental Health, 86(6), 709-16. doi:10.1007/s00420-012-0813-1

Farrokhi, F., & Mahmoudi-Hamidabad, A. (2012). Rethinking convenience sampling: Defining quality criteria. Theory & Practice in Language Studies, 2(4), 784-792. doi:10.4304/tpls.2.4.784-792

Freeman, G. G. (2014). The implementation of character education and children's literature to teach bullying characteristics and prevention strategies to preschool children: An action research project. Early Childhood Education Journal, 42(5), 305-316. doi:10.1007/s10643-013-0614-5

Gill, M. J. (2014). The possibilities of phenomenology for organizational research. Organizational Research Methods, 17(2), 118-137. doi:10.1177/1094428113518348

Giorgi, A. (2010). Phenomenology and the practice of science. Existential Analysis: Journal of the Society for Existential Analysis, 21(1), 3-22.

Gofin, R., & Avitzour, M. (2012). Traditional versus Internet bullying in junior high school students. Maternal and Child Health Journal, 16(8), 1625-35. doi:10.1007/s10995-012-0989-8

Grundherr, M., Geisler, A., Stoiber, M., & Schäfer, M. (2017). School bullying and moral reasoning competence. Social Development, 26(2), 278-294. doi:10.1111/sode.12199

Hanish, L. l., Bradshaw, C. P., Espelage, D. L., Rodkin, P. C., Swearer, S. M., & Horne, A. (2013). Looking Toward the Future of Bullying Research: Recommendations for Research and Funding Priorities. Journal of School Violence, 12(3), 283-295. doi:10.1080/15388220.2013.788449

Hershcovis, M. S. (2011). Incivility, social undermining, bullying...oh my! A call to reconcile constructs within workplace aggression research. Journal of Organizational Behavior, 32(3), 499-519. doi:10.1002/job.689

Hollis, L. P. (2015). Bully university? The cost of workplace bullying and employee disengagement in American higher education. SAGE Open, April- June, 1-11. doi:10.1177/2158244015589997

Hollis, L. P. (2016). Cybershaming-technology, cyber bullying, and the application to people of color. The coercive community college: Bullying and its costly impact on the mission to serve underrepresented populations (pp. 125-135). Emerald Group Publishing Limited.

Homel, J. B. (2013). Does bullying others at school lead to adult aggression? The roles of drinking and university participation during the transition to adulthood. Australian Journal of Psychology, 65(2), 98-106. doi:10.1111/ajpy.12002

Hutchinson, M., & Hurley, J. (2013). Exploring leadership capability and emotional intelligence as moderators of workplace bullying. Journal of Nursing Management, 21(3), 553-562. doi:10.1111/j.1365-2834.2012.01372.x

Ingham-Broomfield, R. (2015). A nurses' guide to quantitative research. Australian Journal of Advanced Nursing (Online), 32(2), 32-38.

© e-JSBRB Vol.10, Iss.1 (2019) 75

Bolling – Volume 10, Issue 1 (2019)

Internal Revenue Service. (2018). Regulations Governing Practice before the Internal Revenue Service. Retrieved from http://www.irs.gov/pub/irs-utl/pcir230.pdf

Jackson, C., Vaughan, D. R., & Brown, L. (2018). Discovering lived experiences through descriptive phenomenology. International Journal of Contemporary Hospitality Management, 30(11), 3309- 3325. doi:10.1108/IJCHM-2017-0707

Jacobs, N. C. L., Völlink, T., Dehue, F., & Lechner, L. (2015). The development of a self-report questionnaire on coping with cyber bullying: The cyber bullying coping questionnaire. Societies, 5(2), 460-491. doi:10.3390/soc5020460

Jenkins, M. F., Zapf, D., Winefield, H., & Sarris, A. (2012). Bullying allegations from the accused bully's perspective. British Journal of Management, 23(4), 489-501. doi:10.1111/j.1467-

8551.2011.00778.x

Jenkins, M., Winefield, H., & Sarris, A. (2011). Consequences of being accused of workplace bullying: An exploratory study. International Journal of Workplace Health Management, 4(1), 33-47. doi:10.1108/17538351111118581

Keashly, L. (2001). Interpersonal and systemic aspects of emotional abuse at work: The target's perspective. Violence and Victims, 16(3), 233-68.

Keller, R., Budin, W. C., & Allie, T. (2016). Task force to address bullying. American Journal of Nursing, 116(2), 52-58.

Leung, L. (2015). Validity, reliability, and generalizability in qualitative research. Journal of Family Medicine and Primary Care, 4(3), 324-328.

Lian, H., Ferris, D. L., Morrison, R., & Brown, D. J. (2014). Blame it on the supervisor or the subordinate? Reciprocal relations between abusive supervision and organizational deviance. Journal of Applied Psychology, 99(4), 651-664. doi:10.1037/a0035498

Lopez, K. A., & Willis, D. G. (2004). Descriptive versus interpretive phenomenology: Their contributions to nursing knowledge. Qualitative Health Research, 14(5), 726–735. doi:10.1177/1049732304263638

Malterud, K., Siersma, V. D., & Guassora, A. D. (2016). Sample Size in Qualitative Interview Studies: Guided by Information Power. Qualitative Health Research, 26(13), 1753–1760.

doi:10.1177/1049732315617444

Messias, E., Kindrick, K., & Castro, J. (2014). School bullying, cyber bullying, or both: Correlates of teen suicidality in the 2011 CDC youth risk behavior survey. Comprehensive Psychiatry, 55(5), 1063- 1068. doi:10.1016/j.comppsych.2014.02.005

Mohammadi, A., Shekari, A., Banar, N. L., & Ajili, Z. G. (2014). Analysis of pedagogy focusing on van Manen's qualitative research method of phenomenology. Asian Journal of Research in Social Sciences and Humanities, 4(11), 273-284.

Moustakas, C. (1994). Phenomenological research methods. Thousand Oaks, CA: Sage.

Namie, G. J. (2017). 2017 WBI U.S. September 2017. Retrieved from

http://www.workplacebullying.org/wbiresearch/wbi-2017-survey/

National Association of Enrolled Agents. (2018). What is an Enrolled Agent? Retrieved from http://www.nea.org/taxpayers/what-enrolled-agent

NgaleIlongo, F. (2015). Defining workplace bullying in institutions of higher learning. Journal of Organisation and Human Behaviour, 4(1) 6-12.

Patterson, E., Branch, S., Barker, M., & Ramsay, S. (2018). Playing with power: Examinations of types of power used by staff members in workplace bullying – A qualitative interview study. Qualitative Research in Organizations and Management: An International Journal, 13(1), 32-52. doi:10.1108/QROM-10-2016-1441

Phenomenology: The question of validity and reliability (2011). Retrieved from

http://pakphilosophy.blogspot.com/2011/04/phenomenology-question-of-validity-and.html

Rhodes, C., Pullen, A., Vickers, M. H., Clegg, S. R., & Pitsis, A. (2010). Violence and workplace bullying: What are an organization's ethical responsibilities? Administrative Theory & Praxis, 32(1), 96-115.

Sansone, R. A., Lam, C., & Wiederman, M. W. (2013). Victims of bullying in childhood, criminal outcomes in adulthood. International Journal of Psychiatry in Clinical Practice, 17(1), 69-72.

doi:10.3109/13651501.2012.693185

© e-JSBRB Vol.10, Iss.1 (2019) 76

Bolling – Volume 10, Issue 1 (2019)

Saunders, P., Huynh, A., & Goodman-Delahunty, J. (2007). Defining workplace bullying behaviour professional lay definitions of workplace bullying. International Journal of Law and Psychiatry, 30(4-5), 340-350. doi:10.1016/j.ijlp.2007.06.007

Schat, A.C.H., & Kelloway, E.K. (2003). Reducing the adverse consequences of workplace aggression and violence: The buffering effects of organizational support. Journal of Occupational Health Psychology, 8(2), 110–122.

Strandmark, M., & Rahm, G. (2014). Development, implementation and evaluation of a process to prevent and combat workplace bullying. Scandinavian Journal of Public Health, 42 (Suppl 15), 66-73. doi:10.1177/1403494814549494

Strayhorn, T. L. (2014). Beyond the model minority myth: Interrogating the lived experiences of Korean American gay men in college. Journal of College Student Development, 55(6), 586-594.

Tuohy, D., Cooney, A., Dowling, M., Murphy, K., & Sixsmith, J. (2013). An overview of interpretive phenomenology as a research methodology. Nurse Researcher, 20(6), 17-20.

Tyler, T. R., & Blader, S. (2000). Cooperation in Groups: Procedural justice, social identity and behavioral engagement. Psychology Press: New York, NY.

VanScoy, A., & Evenstad, S. B., (2015). Interpretative phenomenological analysis for LIS research. Journal of Documentation, 71(2), 338-357. doi.10.1108/JD-09-2013-0118

Vasileiou, K., Barnett, J., Thorpe, S., & Young, T. (2018). Characterising and justifying sample size sufficiency in interview-based studies: Systematic analysis of qualitative health research over a 15- year period. BMC Medical Research Methodology, 18 doi:10.1186/s12874-018-0594-7

Vibha, P., Bijayini, J., & Sanjay, K. (2013). Qualitative research. Perspectives in Clinical Research. 4(3), 192.

Wallace. B., Johnston, L., & Trenberth, L. (2010). Bullying the boss: The prevalence of upward bullying behaviours. The Australian and New Zealand Journal of Organisational Psychology, 3(1), 66-71. doi.10.1375/ajop3.1.66

Walser deLara, E. (2016). Bullying Scars: The impact on adult life and relationships. Oxford University Press.

Wiedmer, T. L. (2011). Workplace bullying: Costly and preventable. Delta Kappa Gamma Bulletin, 77(2), 35-41.

Wollard, K. K. (2011). Quiet desperation: Another perspective on employee engagement. Advances in Developing Human Resources, 13(4), 526-537. doi:10.1177/1523422311430942

Yu, S., Wang, M. N., Chen, Y. & Mu, P. (2011). The lived experiences of fathers in cross-national marriages. Journal of Nursing Research, 19(3), 210-219. doi: 10.1097/JNR.0b013e318229c3ee.

© e-JSBRB Vol.10, Iss.1 (2019) 77


47 views0 comments

Opmerkingen


bottom of page