Is your research valid?

Permit me to use an automotive-related example to describe the concept of validity. Imagine that you purchased a new car from a dealership four years ago. This morning you dropped back to them for its annual service. Mid-way through the day you get a call from them.

“We’ve completed the service and your car is ready to collect. Incidentally, we notice that your car is due to undergo its National Car Test next month. When we were checking your tyre pressure, we noticed your two front tyres did not have enough thread on them. They will need replacing if you are to pass the test. By the way, we can do that for you, for €185. Oh, and your windscreen wiper is not cleaning your rear window fully. This might cause you to fail the test as well. If you are interested, we can also fix that for just €50. And one other thing, we recommend you change your brake discs after 100,000 km. Your car already has 105,000 km the clock! It’s just advisory, but if you want to be on the safe side, we can replace them for €550 …”.

At this stage, you might rightly be asking yourself, are these ‘real’ issues, or is the dealer trying it on? If I had left it to another service centre, would they be saying the same? How do I know they are telling the truth? Do I really need to spend another €785 on top of the €99 I am already paying for the service? What are the ‘facts and what is ‘interpretation’ or ‘opinion’? In other words, is what they are saying valid?

Validity is the level at which a study – whether it is a health check on your car, or data collected from a questionnaire or interview – measures what it was meant to measure. For example, if I want my car to be constantly running as good as when I bought it from the dealer, I will want it regularly serviced and maintained. At the beginning of each service, I will want them to ‘measure’ key parameters against which the car should perform, and then ensure that the car continues to perform within them. If this is the case, I will probably consider these points to be valid. In other words, I will treat each ‘measure’ as appropriate and each point as a ‘fact’. I will also then be able to say to myself that the measures are valid, and the outcome of the measures are valid. Therefore, I must get all three things – tyres, windscreen wipers and brake discs – fixed, or replaced, on my car.

If, on the other hand, I simply want my car to be safe, and to pass the National Car Test, then I might question the validity of what is said. For example, is it appropriate that the dealer measure tyre thread depth at all? Is tyre thread depth measured in the National Car Test? What exactly does ‘wiper is not cleaning the rear window fully’ mean? Is the brake disk measure (100,000 km) valid? Who is the ‘we’ in ‘we recommend’? With this mind-set, I take a more questioning stance. In other words, I want to check the validity of everything that is said. This includes, for example, what is being measured, how it is being measured, and who is doing the measuring?

To check for validity, I might double-check the legal requirements for tyre thread depth on the National Car Test website. I might also request a second opinion from a mechanic on the windscreen wiping function. For example, is it cleaning 95%, 90% or 50% of the windscreen? Alternatively, I might consider asking who is making claims about the brake disc requirements?

I might form the view that the first two items (replacement tyres and windscreen wipers) are valid, as I know my vehicle will fail the National Car Test if I do not replace them. However, my brake discs are another matter. The claim that they need to be 100,000 km might be valid using the dealer’s checklist. The car manufacturer, however, might recommend changing at 110,000 km, the Automobile Association recommend replacement at 130,000 km, and I might form the opinion that the brakes are working fine (personally valid). Given the doubts about the validity of the ‘replace at 100,000 km’ claim, I may put off having my brake discs replaced for now.

Or I may ascertain, by checking the National Car Test requirements, that the dealer is trying to make a quick-buck and none of these points are valid. In this case, I will not get any parts replaced, or work done.

In research, we consider data to be valid when two criteria are met. First, it measures what it was intended to measure. Second, it uses an appropriate method of measurement.

Designing research that is valid requires you to continuously ask:

  • Is my investigation providing appropriate answers to my research questions (or hypotheses)?
  • If it is, am I using suitable methods to obtain those answers?
  • Am I measuring the right things?
  • Am I measuring these things in the right way?

What, then, is the key to achieving validity?

Achieving validity with quantitative methods involves being able to establish, as best you can, the facts. Using statistical methods, you may be able to, for example, calculate variances between different sites and ascertain if ‘answers’ are the same, or different. For instance, if the same data results from tests at two different sites, it is more likely to be valid. The extent to which you can generalise from your sample to the population can be validated, in part, by providing an appropriate figure for statistical significance alongside a core statistic such as a total or an average score.

With qualitative methods, validity involves making a rational judgement (not an opinion) about applicability of your analysis and findings when transferring from your specific research context to another context. There may be different interpretations (other service centres, another mechanic’s and more importantly, yours, as per the example above) to be considered. You will need to back up any assertions made by explaining why they are valid (for example, a worn-out disc brake is an engineering certainty).

For both quantitative and qualitative methods, some form of triangulation is advisable. That is, collecting data that provides the same or similar answers from two or more sources increases validity among other things. A simple technique used widely in research is to get a second opinion, to question groups, rather than individuals, or to review results with a panel of experts.

In addition, you should always apply a sense check. Ask yourself: does my answer look/sound right? Obviously, the answer will always involve some element of interpretation, but you can rely on your experience and expertise (or that of others) to guide you. Finally, if you find that something is invalid, you should explain why this is the case (that is, clarify your reasoning).

While in most business and humanities research it is rare to be able to ensure 100% validity (as we are often dealing with shifting naturalistic, personal and social phenomenon), we should aim to make our research as valid as possible. This is achieved by ensuring that we are measuring the right thing and measuring it the right way.

Thesis Upgrade’s practical research guidance can help you measure the right things and measure them the right way.

Is your degree worth it?

It can seem, as a student, that you are under financial attack. Globally, college registration fees and tuition rates are constantly and continuously rising. The cost of college accommodation is steadily increasing. Living expenses for students always seem to run ahead of national inflation statistics. Add to that, the fact that you may be studying and earning little, while your friends enjoy a weekly or monthly pay cheque from their job. Or what about the forgotten hidden costs of spending time in the library instead of relaxing at the gym or the cinema? Factors such as these are compelling current and prospective students to question whether doing a degree programme is worth it, at least financially.

The traditional wisdom is that a college degree will lead to a higher income and better standard of life. If, however, you take a shorter-term outlook and confine yourself to entry-level salaries and employment opportunities relevant to your degree, you will most likely find that is not the case. In relative terms, globally, entry-level salaries for graduates are lower than they have been ever. Furthermore, the prospects for advancement are less certain. Even if you take a longer-term view, you may never recover the true costs of going to college over your lifetime of working!

These considerations have led to a long-standing debate within the academic community. For example, should students view college only as a resource to prepare themselves for the future workforce? Or is the aim to create engaged and enlightened citizens, capable of thinking critically and communicating clearly, and ultimately able to thrive in their well-being? If you follow the former logic, due to the high, and ever-increasing, financial, social, and mental expenditures involved, the investment does not seem to match the return for many graduates.

If, however, you look at the more expansive perspective of college life, a positive view emerges. The reality is that college is about both preparing people for a job (and helping them advance their careers and earnings) and to thrive in their overall lives. A Gallup-Bates College Study (in the USA) affords some insight into what students maybe thinking:

  • 80% of college graduates say it is ‘important to derive a sense of purpose from their work’; yet only 38% of graduates strongly agree they have ‘discovered work that has a satisfying purpose’.
  • For graduates with low levels of purpose in their work, only 6% are ‘thriving in their overall well-being’; while graduates with high purpose in their work are ten times more likely to be ‘thriving in their well-being’ (59%)!
  • The top two drivers of a graduate achieving purpose in their work are when they have an applied job or internship while studying for their degree, and when they have someone who encourages their goals and dreams during college. Graduates who are reflective are 67% are also more likely to have purposeful work.

These findings are true for all generations of graduates, but especially for Gen Z and Millennials. They are more likely to want to derive purpose from their work than from other sources in life. Both these generations, in looking back on their college experience, are likely to regret not having had real-life work experiences while studying.

So, graduates value both purpose and work, and, in fact, find the most purpose in and from work. Colleges can, for instance, help graduates achieve purposeful, applied work by including applied studies and work experiences while the students are studying with them. They can further assist by teaching students the knowledge and skills needed to care about their own well-being, and by having faculty, staff, and students who embrace a culture of caring about each other’s goals and dreams. Instilling a classically liberal arts element as an integral part of learning (that is, teaching students how to be reflective) is also powerfully linked to job success and satisfaction.

The purpose of college is not just job training, but also includes career and life training. Your job is not just about a salary; it is also about obtaining experiences that has a purpose. Helping graduates achieve purposeful work experiences may indeed be the purpose of modern college.

If we want to answer the question of whether college is worth it, we need to start by asking ‘what is the purpose of college?’ Reflecting on that may well be the key to unlocking the next era of higher and further education, as well as the societal well-being and economic prosperity for our nation.

Being consistent throughout your data collection and analysis instils trust and believability

When assessing a thesis or dissertation, one of the many issues examiners look for is consistency. They like to be able to follow a golden thread that stitches the disparate sections of your document together. One way in which you can do this is to ensure uniformity in how you design, execute and analyse your primary research.

Here are examples from three students Thesis Upgrade has worked with. They have taken different approaches to their data collection and analysis, but all have ensured robust consistency throughout the process.

John Shelly, an MBA student, adopted a quantitative approach. “My research design involved establishing if there was a relationship between investment in training in innovation techniques and the innovation outputs from small and medium-sized companies. I collected quantitative data through an online questionnaire. It was easy for business managers to complete it, as their answers were based on facts they could obtain from their own organisations. For example, spend on innovation training, quantities of new products developed, actual market demand, prices achieved by new products, and so on. The data I gathered were empirical and led me to test a range of hypotheses I had formulated using quantitative, mathematical techniques. Having a quantitative (mainly numerical) data set meant I used statistical functions and formula (totals, sums, averages, modal values) for my analysis. These analyses showed that there was a positive relationship between two variables. That is, the more money spent on training, the more likely it was for new products to be produced and sold. Using a quantitative method from research design, through to data collection and collation, and on to analysis and findings, helped me to demonstrate consistency in my approach”.

By contrast, Sonia Broadchurch, who was studying for a BSc in Human Resource Management, took a qualitative approach: “My research examined what some people view as a soft issue: how managers use performance appraisal meetings to motivate staff”, she said. “In my case, there were multiple realities to deal with. Each of our managers, and every worker, subjectively saw the purpose and outcomes of appraisal meetings differently. They construct their own viewpoints through ascribed meaning. It is easy to fall into the trap of gathering and analysing data that is unstructured and apparently meaningless. I therefore set out to collect data – words, phrases, sentences and exchanges – by being consistent in the questions I asked at interview. I continued to maintain this uniformity when I analysed the data thematically. The emerging themes helped me to substantiate the claim that appraisal meetings could be used effectively to motivate staff”.

Some students, like Charlie Reyners, conduct mixed methods research. “My study involved exploring the equalities, and inequalities, of internal promotions in government offices. My initial, quantitative research involved administering questionnaires to managers who interviewed internal candidates. The findings indicated that managers believed promotions occurred following a rigorous and objective interview process. Similar questionnaires administered to interviewees showed that few internal candidates felt the same. I followed up the contrasting data sets with two focus group discussions with the interviewers (managers) and interviewees (internal staff candidates). This approach allowed me to explore the issue qualitatively and in depth. Both interviewers and interviewees began to question how objective the decision making was. By confronting the ‘reality’ initially presented by the managers we began to uncover the hidden effects of different values, ideologies, biases and influences on the selection process. I had originally intended to conduct only quantitative analysis, but later added to that with critical interpretive qualitative analysis collected from the two focus groups”.

How to ensure consistency in your research is a key decision that is, ideally, made when designing your fieldwork or study. It involves choosing whether to carry out quantitative or qualitative research, or inductive versus deductive research, and making numerous decisions that might influence the data collection and analysis process. Sometimes researchers, such as Charlie, find it necessary to change their approach mid-process, as the information they have collated indicates they have insufficient data of the appropriate type to draw findings and a conclusion. In these cases, an additional data collection process should be undertaken, and an abductive form of analysis, which combines all the data sets, then follows.

Thesis Upgrade’s Analysing and Interpreting Your Data is a useful, downloadable, digital publication. It contains easy-to-understand information and straightforward explanations to help you interrogate, analyse and interpret your collected data. Buy now for immediate use.

Thesis titles matter

One of the first things that students need to do when they start a thesis or dissertation is to give it a title. Many students, however, spend too little time thinking about it and simply name their research in a generic way.

Consider this title: ‘A research study on market segmentation in the auto sector’. This a useful title, but it could be expanded upon and strengthened. It states the obvious. We already know it is a research study before we pick it up to read it, so these words are largely redundant. The term market segmentation is useful, as it tells us broadly what it is about, but could we be more specific about which type, or types, of segmentation are being written about. For instance, what does the word ‘auto’ mean – automatic, automotive? Is it the global auto business, or the EU business, or the UK, or …? Too often, students fail to put enough thought into a thesis title, resulting in inconsistent, vague or unfocused descriptions that confuse, or worse, deceive potential readers.

Titles for theses should reflect how newspaper headlines are written. They should attract attention from a specific, known audience and deliver a comprehensible description about the content of the full document. Along with the abstract, titles are what initially catch the eyes of people searching for content within your field. And for citation searchers, your thesis title is amongst limited information they will see when they are searching around your topic.

Here are eight tips on how to draw attention to your research and get your thesis done!

Follow your institute’s requirements:

Thesis titles need to meet certain standards and be in the correct format before being considered suitable for submission. For example, universities may require titles to adhere to the APA style of referencing for academic papers. This means no more than twelve words in length, title case, centred vertically and horizontally, no abbreviations or contractions, and no special formatting apart from bolding (that is, omit italics, underline, etc.).

Emulate the best:

By the time you have completed reviewing literature for your thesis you will have come across numerous book, journal and paper titles. Review these to explore how established and widely cited researchers in the same field are crafting their titles. It is acceptable to imitate the structure of appealing titles, but copying them verbatim will look suspicious, and probably negatively affect your impact. Skim those you have been attracted to, then look for what it is that captures your attention and interest.

Be clear, but comprehensive:

Whether you emulate or not, outline what the reader will get out of your thesis simply and precisely. Revealing the essence of your document immediately in the title will make the entire document’s purpose clear and your objective understandable. Your title should be transparent about the research you accomplished, not the whole topic you studied. Identify what was specifically investigated, how the research unfolded, and who it involved. In other words, make sure your title describes the specific topic, the research/study method and scope, your primary results, and any samples involved if possible. These things help your audience by giving them reasons to continue reading with a more profound comprehension.

Work backwards, not forwards:

It is often effective to work in reverse, pointing to your results first. For example, knowledge sharing that promotes internal workplace communications: A case study on technological innovation in Ireland.

Include the essentials, leave out everything else:

Avoid being vague, but also being too verbose. Make the effort to pare down your title to only the fundamental parts. The word maximum for titles of research papers published in journals varies according to their specific requirements. However, authors typically aim to keep it below thirteen words in length. It could be useful to apply this same rule to your thesis.

Draft variants of your title:

Experiment with using different phrases and syntax overall. Once some versions have been written, you can read over them, see where redundant or useless words are being used and which versions are simpler and more to-the-point. Frequently, you will be able to produce the best title for your paper just by combining the greatest parts of different versions you drafted.

Keep it readable:

Most importantly, you want a title that is readable and easily understood even by those who may be a novice researcher in your field of study. Using complex, industry-specific terms, abbreviations, and even just hard-to-grasp or uncommon phrases will deter readers. Evade the temptation to use jargon – if you have jargon in your draft title, simply take it out. A title full of extraneous information will turn readers away; a thought-provoking and descriptive research thesis title draws readers in.

Ask the experts:

For most researchers developing a thesis title is something that they will do once, twice or perhaps three times in their life. Others, such as supervisors, tutors and librarians deal with them day-in and day-out. They have the experience to know what makes an effective thesis title. Ask them for ideas, or show them yours, and be prepared for them to suggest you modify your title slightly. They tend to offer excellent advice!

For more practical guidance, take a look at Introducing and Concluding Your Thesis, a useful, downloadable, digital publication. It helps you succinctly introduce and conclude your research study, so that you make your thesis title matter. Buy now for immediate use.

Getting going with your thesis literature review

Many students undertaking a thesis find it difficult to know where to start when doing a literature review. According to our recent survey, most students feel overwhelmed by the sheer volume of literature available and are not sure how to collate it. One small step that can be taken is to categorise any literature you come across into one of three types (original, published or locator).

Original sources: These provide the first occurrence of a piece of work. For example, if you were going to do doctoral research on socialist economic policies, it is likely that you would need to look at foundation literature, such as notebooks compiled by Karl Marx. Ask yourself, who is the equivalent of Marx for my area? If, for example, your thesis is based on data analytics you might want to look at some of the original presentations and lectures in this area, such as John Tukey’s work on the future of data analysis, from the late 1950’s. Such work can be publicly available (like research theses, conference proceedings and some industry and government reports) or privately held (like unpublished manuscripts, notebooks, minutes from committee meetings and private emails). As the focus in these original sources is on new material, they are inclined to be detailed in nature and challenging to access. Despite this, it is, however, important to access original sources to get under the skin of a topic.

Published sources: Published sources tend to be aimed at a wider audience and are thus easier to access and more widely disseminated than original source material. Published sources can be academic in focus (academic books and journal articles), report-based (government publications), practitioner oriented (professional body reports) or all-purpose (general books, newspapers and magazines). A student of data analytics, for example, might look for John Tukey’s academic articles in this area, written in the early 1960’s and the many books, articles and other publications written and published since then.

Locator sources: These are sources that assist in accessing and attaining literature from the original and published sources. They are available in traditional formats (such as hard copy dictionaries, indexes and encyclopaedias) and from contemporary platforms (such as search engines, virtual academic database portals and publishing house portals). In addition to their retrieval function, these sources can also act as a convenient springboard for your literature search. This is because they offer the opportunity to gain an elementary understanding of a topic you may have limited knowledge on.

To help establish exactly which original, publisher and locator sources are available to you, it is a good idea to contact the person in your institute’s library responsible for the subject matter you are researching. This person should also be able to give you advice on carrying out your search efficiently and effectively.

It is also worthwhile asking this individual about the training your library provides concerning searching for relevant literature and sourcing suitable publications. Most colleges offer regular tours or induction sessions for their libraries (in-person and online). It is useful to know your way around physical and online catalogues, databases, etc. Time spent on this early on will save time later.

One final piece of advice: Do not rely on just one library. Use both your college library and libraries outside your college. For example, your local community library, libraries in professional institutes and government libraries. You may also want to access online repositories, such as the textbooks and toolbooks from Thesis Upgrade. For example, our Searching for Relevant Literature Toolbook will help you get going with your thesis literature review.

Thesis? It’s all Greek to me!

‘It’s all Greek to me’ is an expression we use when trying to say something that is not understandable. Many of us feel that way about the word ‘thesis’ – which has many, often conflicting, meanings. Small wonder then, that the origins of the word ‘thesis’ is found in the Greek language. It means to ‘put something forth’ or ‘make a proposition’, usually in an academic context. The word has its roots in the development of higher education in Athens, Greece, around 420 BCE (before common era).

Traditionally, education in Greece was familial, gendered and age-related. Fathers educated sons, mothers educated daughters, and older slaves educated younger slaves. More formal education began (for boys at least, females and slaves were excluded) in classical Athens around 500 BCE. It consisted of two parts. The first part is ‘gumnastike’ (physical education, strength, stamina, preparation for war) which took place in a ‘gymnasium’) and the second, ‘mousike’ (music, dance, lyrics, poetry). Students learned in a semi-structured way. For example, they would write by etching onto a wax-covered board, read through learning to memorise and recite poetry, and learn about legends such as Hesiod and Homer. Once a child reached adolescence, their formal education ended.

In contrast to the Athenians, military dominance was of greater importance to the Spartans. They structured their educational system as an extreme form of military boot camp, which they referred to as ‘agoge’. To them, the pursuit of intellectual knowledge was trivial, and academic learning, such as reading and writing, was kept to a minimum. A Spartan boy’s life was, instead, devoted almost entirely to his military school with the purpose of forming an almost indestructible Spartan soldier.

Over time the Athenian way became more popular. Society began to hold intellectual capacity with higher regard than physical Athens quickly became a global centre for the arts, learning and philosophy. Philosophers such as Socrates and the sophistic movement attracted to an influx of foreign scholars, teachers and their ‘higher education’ led to a growth of the equivalent of secondary and post-secondary schools. More focused fields of study included mathematics, astronomy, harmonics and dialect – all with an emphasis on the development of a student’s philosophical insight. It was necessary that individuals possessed the ability to take knowledge in a field and apply it towards an understanding based on logic and reason.

What has all of this to do with the word thesis? One of Socrates’ pupils, Plato, established the Academy (circa 387 BCE) which is often credited as the first University in the world. Plato prescribed a rigorous education a student must go through to attain true virtue, and thus understand reality for what it is. It included an elementary education in music, and poetry; physical training; three years of mandatory military training; ten years of mathematical science; five years of dialectic training; and fifteen years of practical political training. The few individuals who reach this upper level would, he claimed, become philosopher- kings, leaders of Plato’s ideal Republic, and even the world! A final step in their higher education involved scholars proving their worth by putting forward their thesis. This was a presentation, or document, submitted in support of candidature for one of these positions in the new world order.

Although the style and format of a modern thesis or dissertation is somewhat different to those of early Greek schools, modern theses generally retain similar elements. For example, writings by other, earlier authors are examined (the literature review); a thesis is proposed (today we have research aims, objectives, questions and hypotheses); structured ‘observations’ are offered (nowadays, research methodology and methods) and analysed (today referred to as data analysis); to produce findings and conclusions (proof of the thesis). Thanks to Plato and the ancient Greek education systems, we have the thesis. A ‘test’ of a scholar’s beliefs that is still widely in use in institutes of higher education today to assess a student’s suitability to become a leader in their community.

As a modern-day researcher, you do not need to study ancient Greek civilisation to find out how to go about Doing Your Thesis. You just need to click here for practical guidance!

Research on thesis proposals

Thesis Upgrade surveyed 60 postgraduate students who were starting their thesis journeys. The three most frequent areas of concern for these students were:

  • How do I define my research area?
  • Have I selected the right topic for me?
  • Have I written my title correctly?

Everyone feels confused at the beginning of the research process. This confusion probably means that you feel frustrated by your mental log jam. You may also feel anxious you are not making substantial progress with your thesis or dissertation. You are not alone!

Chiomo, a postgraduate MBA student, said “my main concern is the relevance of my research area – how could I be sure it was suitable for an MBA thesis?” Sonia, a postgraduate on an MSc in International Business Management programme added: “I am really worried about my topic, I need to narrow it down, to make it useful for real life or for my future career”. Anna, a final-year undergraduate, was alarmed to find that her supervisor would not be assigned until after she submitted her research proposal: “I was really worried about whether I have chosen a topic relevant to my study, whether it was too broad, or too narrow”.

Mark Campbell, Director with Thesis Upgrade commented: “It is not unusual for students to have to come up with a research area, or even a specific topic, themselves. Many colleges and universities do not have the resources to appoint supervisors at the research proposal stage. They wait to see what the students themselves propose, before allocating a supervisor who knows something about the subject area or proposed research method. We are finding an increasing number of students simply want to bounce their initial ideas off someone who is experienced with the research process – whether that be other students, or preferably a tutor or supervisor”.

Check out our Developing Your Research Proposal to calm your concerns. Buy now for an immediate confidence boost!

Research on thesis aims and objectives

Our survey of 60 postgraduate students offers some interesting insights into the top concerns they have during the early stages of their research. Once the initial topic have been selected, and research statement written, the next challenge students face is setting out their aims and objectives. Nearly two thirds of those questioned (64%) mentioned this as a key anxiety.

Research aims and objectives can be difficult to specify. Stephen, an MSc Human Resources Management student, maintained “I thought I knew what my objectives should be, but once I started to do my secondary research, I realised that this was a wide area and there were already many studies written about the topic. I had to refine my objectives again and again before my supervisor accepted them”. Bethany, a BA Social Studies student, assumed that finding a link between economic growth and social development in a specific African country would be an appropriate objective. When she was allocated to her supervisor, however, he said she would need to be more specific and much clearer: “It delayed the start of my research by two weeks, which was so disappointing – and inconvenient”, lamented Bethany.

Sue Mulhall, Director of Thesis Upgrade, notes “Most postgraduate students have some experience of setting research aims objectives. It can, however, be a daunting task for undergraduate students for whom undertaking a major research project is a first-time experience. Students often ask for advice on their aims, objectives and research questions. With proper support, most learners can produce clear aims and develop specific, measurable and realistic objectives. A suggestion the team at Thesis Upgrade often make is to begin by simply considering what is not known about the topic”, Sue recommends. “This apparently straightforward question can help to identify and critique where the key gaps in current knowledge are”.

Try out our toolbook packed full of practical tools to help you choose your thesis topic, write your research questions and formulate your hypotheses. It also shows you how to turn your dissertation idea into a realistic research proposal project plan. Buy our skills-based resource now for immediate use.