Failed your thesis? What next?

Your results have just been released. Hopefully you have been successful in your studies and are enjoying celebrating your achievement. Unfortunately, maybe the news you receive is not so positive. You get the unwelcome message that you have ‘failed’ your thesis or dissertation. Despite your undoubted disappointment, this should not stop you from continuing to study, completing your programme and obtaining your qualification. Simply pause and ask yourself: ‘what can I do next?’

First, and foremost, it is worth noting that failing a thesis or dissertation is not uncommon. After all, for most of your degree, many of of your assessments were short, directed tasks, which accounted for relatively small credits towards your overall qualification. When it comes to your thesis, or dissertation, however, you are often left to work alone, for a long period of time. During this time, you are using a process that is relatively new to you, and which comprises a large proportion of the credits you need to complete your programme.

Take note of the proverb put forward by Thomas H. Palmer in his Teacher’s Manual. ‘Tis a lesson you should heed: Try, try, try again. If at first you don’t succeed, try, try, try again’. Unless your college’s rules say otherwise, it is usual for students to have an opportunity to submit a revised thesis or dissertation, and/or do another viva voce (oral examination). You should most certainly ‘try again’!

Review your result

When ‘trying again’, you need to verify exactly what your result means. One student we worked with contacted us in a panic. They said they had ‘failed’ their dissertation. When we looked at their transcript, it simply said the mark was ‘withheld’ (some institutions use other wording, such as ‘pending’). This did not mean that they had failed, but, rather, there was a delay in publishing their result. This can happen if, for instance, the Examination Board did not have all the information they needed, such as requiring further evidence in support of the thesis. In such cases, you will be requested to furnish additional information within a specified (usually short) timeframe before you can obtain your overall result. Occasionally, if an Examination Board is not able to agree a mark/score on the available evidence, students can be offered a viva voce examination (an oral defence of their work).

Another common result given is ‘failed, entitled to resubmit’. This indicates that you have not passed your thesis, but can resubmit all, or part of your assessment work, for further consideration by the examiners. The results for such resubmissions are typically ‘capped’. This means that if you are successful in your resubmission, you will only be awarded the lowest available pass grade. Nonetheless, it is well worthwhile resubmitting, so you can complete your programme of studies and rightfully earn your degree.

Try again. Do better

Another proverb, this time from Samuel Beckett. ‘Ever tried. Ever failed. No matter. Try Again. Fail again. Fail better’.

Assuming your overall result is a ‘fail, entitled to resubmit’ you should view this as your opportunity to ‘do better’. To help, we recommend you seek feedback on your performance. It is important to do this immediately, as information on why students fail is not normally retained by institutions for too long after the official results are published.

In some cases, the Examinations Board provides feedback in the form of a written performance profile, or report, which identifies areas where marks have been gained or lost. Performance profiles vary in the amount of detail they offer, as the content is decided by the Examination Board, who, in turn, are advised by the institute’s general assessment regulations. You will, however, receive a score/mark on each component, or part, of the thesis. You may find minor differences between your overall, examinable, score and the component-level scores on your performance profile. This can occur when the Examination Board checks and revises borderline scores and amends the overall score, accordingly. In this situation, you can be confident that your awarded score/mark is correct.

Even if you do not receive written feedback, you may be able to approach your supervisor and/or examiner(s) to request some. Most are usually willing to offer verbal observations, but will rarely give written comments, as this might cut across decisions made by the Board. Again, you should try to do this quickly, as those who are most likely to be able assist you (such as your supervisor, internal examiner, or external examiner) may be moving on to the next cohort of students. It is important to seek not just feedback, but feed-forward. In other words, to discuss with your supervisor and/or examiner how you can improve your research study, document structure and/or presentation format, when you resubmit.

Other supports

In addition, you should establish what other supports may be available to help you resubmit your work. In some colleges, your supervisor remains in place until your second submission is complete. Whereas, in other institutions, you are automatically appointed an alternative supervisor to assist with resubmission. Even if no supervision is provided, you may have access to a counsellor or mentor to help you. Regardless of what is (or is not) available, it is always worthwhile checking. The more guidance you can obtain, at this stage, the better. If no support is forthcoming from your college, seek help from other students who have passed their thesis. Find out, who, in your class, was awarded a first/distinction. Would they share a copy of their thesis with you? Could you ask for their advice on how to improve yours?

Check the rules for resubmission

Make sure you know the rules for resubmission. You need to find out when, where, and in what format, you can resubmit. Questions you may need to pose are:

When should resubmission happen? This could vary from a couple of days, to up to six months after the original submission date. If you are not ready to rework your thesis or dissertation straight away, you may be able to postpone to the next opportunity for resubmission. This could, however, be as much as twelve months away. If you are ready, confirm who is responsible for registering you for your resubmission. In some institutions, this is done automatically by the faculty; in others, you will be responsible for your own registration.

What will your institution allow you to revise? Sometimes, it is only the presentation elements of your thesis document that can be changed (such as, structure, layout, typeface, and so on). Whereas in other cases, you are permitted a total rewrite of your study. This may occur, for example, when the content is of an appropriate standard, but it contains errors related to spellings, grammar, citing and referencing. Some institutions will only allow you to alter components for which you have a fail mark (such as, if your findings are poorly written up, you may be invited to revise that chapter/section only). Others may direct you to refine, or amend, a specific section (for example, your abstract).

The viva voce

In cases where you are invited to a viva voce, or a second presentation, it is usually because the research itself is passable, but the Examinations Board wants added assurances. For example, that you carried out the work yourself, or that you fully understood the research process you followed. This may mean that you will have to prepare for an in-depth, audio-visual presentation, and/or, participate in an oral questions and answer session. If your research is genuinely yours, and you understood what you did, this should not phase you. Make sure that you prepare in advance and practice, practice, practice. It would be helpful to have a copy of your thesis document ready, with key information marked up, so that you can refer to them easily.

If all else fails?

One final proverb comes to mind here. ‘If at first you don’t succeed; try, try again; then give up, and go and do something else instead’ (anon.).

If you are not eligible for a resubmission, or if they are not available for your thesis, you should check to see if you can gain credit for the research by retaking the whole module. That is, by doing new research and submitting that in place of your original thesis, or take another module, like a taught module, instead.

Finally. most institutions have an appeals mechanism, and this is the ultimate course of action you can take. With a thesis, or dissertation, it is rare to have the result of an Examinations Board overturned. The Appeals Board, however, may offer an alternative way of obtaining credit to complete your qualification, for example, by obtaining credits on another module.

 

Bibliography and reference management

When you are doing research, it can be useful to use a tool to create and sort citations and references. This is particularly so if you need to manage a substantial number of references, as is the case when you are doing a thesis. 

Bibliographic, or reference management tools, offer an uncomplicated way to collect and save details of the resources you have used in your work. These are software applications for scholars, researchers and authors to use for recording and utilising bibliographic citations (references) as well as managing project references either as an institution, research team or an individual. Once a citation has been recorded, it can be used time, and time, again, when generating bibliographies or lists of references.

Apart from organising references, most reference management software enable users to search references from online libraries. They normally consist of a database in which full bibliographic references can be entered, plus a system for generating selective lists of articles in the different formats required by publishers and scholarly journals.

Modern reference management packages can usually be integrated with word processors so that a reference list in the appropriate format is produced automatically as an article is written, reducing the risk that a cited source is not included in the reference list. They will also have a facility for importing the details of publications from bibliographic databases. They can generate references, reference lists and bibliographies in a variety of referencing styles.

“Selecting a tool to manage your references is straightforward. A first question to ask yourself, however, is whether you need to use one at all. All these tools require some investment of your time, so some form of time-cost v benefit analysis is essential”, suggests Carlotta Morillo, BSc Sustainable Management.

There are a range of free and paid for tools available, including web-based applications. Many free versions, however, may have limitations, such as the amount of storage space.

Manual handbooks

The simplest ones to use are downloadable files – effectively PDF handbooks that explain how to manually cite and reference, and which sometimes provide examples. Most educational institutions have these, and they are usually accessed from their website or intranet. Their big advantage is that they can be downloaded and kept on your laptop, or phone, or printed-off and kept in your study area for quick reference. The downside is that they do not perform the formatting for you, but just give advice on how to do it. If you are new to using referencing this can be a positive thing as it helps you to learn how to reference properly.

Word processor-embedded plugins

Word processing software you use, such as Word, may have in-built or plugin applications which allow you to generate references as you write. Although useful, experience suggests that these are not as accurate as full-blown tools.

Online websites

There are websites that allow you to instantly generate a single reference, often known as citation generators. The big advantage of these is that they are easy to access and use (so long as you are online!).

Browser extensions

If you are online, there are tools available that provide extensions to your browser which help you easily collect references.

Specialist applications

Many applications (usually in return for a licence fee) allow you to access them online and/or install them onto your device.

Wikipedia provides an excellent page containing an extensive comparison of tools you may find useful and many of the leading providers have useful explanatory videos on YouTube. At the present time, the best-known of these are Endnote, Mendeley, Papers and Zotero.

Effective bibliography and reference management is crucial when you are sourcing and reviewing your literature. Thesis Upgrade has some useful publications to help you search for relevant literature and critique your sourced literature.

Linking literature reviews to research questions and hypotheses

A question we are often asked is ‘how do I link my critique or literature review to my research questions or hypothesis?’ Recently, a variation of this question was asked by Will. He had submitted a first draft of his thesis to his supervisor for comment and was advised that what he had produced a literature summary, not a critical review.

When we looked over Will’s work, it became apparent that he was synopsising what had been written by others at the expense of building on it to develop his own arguments. A hallmark of academic research is that it relates to the work of other scholars and practitioners. To improve his review, Will needed to directly address his research questions using appropriate literature, while at the same time, embedding it in a critique of the work from previous researchers. Following our guidance, Will produced his own unique academic positions. This helped him answer his research questions and draw on the concepts, theories and models from others to substantiate his arguments.

Like Will’s situation, many dissertations begin with questions (who, what, where, when, why, how, or some combination of these) or hypothesis (a tentative statement, or explanation, for an observation, phenomenon, or problem that can be tested by further investigation). Prior to finalising your questions or hypotheses, you need to establish what is already known about the topic. in other words, what ‘gaps’ exist in that knowledge (that is, a critical review).

A critical review enables you to build a scaffold for your main arguments. This scaffold will, in turn, provide you with information to answer your questions or hypotheses.  A well-sourced critique should help you, firstly, set the context in which your research is located. If, for example, your broad research topic is ‘supply chain management’, the critique is partially an opportunity to introduce the topic, outline its history and bring the reader up to date on the latest thinking. This should facilitate you to align your research questions, or hypotheses, to what is already known about the topic.

Secondly, the critique should aid the identification, definition and exploration of your overall research problem in more depth. Searching for, reviewing and critiquing relevant books, journals and other material, permits you to situate your specific research within the accumulated archive of knowledge already associated with your topic. This helps to relate your questions or hypotheses to queries which have already been previously posited (and answered, or partially answered). Using this knowledge-base will also enable you to highlight seminal investigations, as well as describe, debate and explain the contributions of all key authors to the topic.

Exploring further, you should be able to recognise concepts and theories that are particularly relevant to your questions/hypotheses and discover other, similar inquiries associated with your topic. This literature should allow you to link your question or hypothesis and provide input into your conceptual and theoretical framework. It may assist you to establish specific research methodologies and methods that have been used in prior investigations. Furthermore, it should assist you to formulate and continuously improve your questions or hypotheses, making them more precise and relevant.

It is likely that you will return to your critical literature review many times on your thesis journey, updating, revising and rewriting it, while considering newly discovered ideas, concepts, theories and viewpoints. As you continuously review it, keep asking yourself: ‘How does this material contribute to the fundamental purpose of my thesis?’ The answer will help you maintain your focus on exactly what you want to achieve, why you want to do it, and how you intend doing it. Adopting this approach helps provide questions that lead to answers, which, in turn, add to and enhance existing knowledge. It also means you avoid two principal pitfalls when linking your thesis literature review to your research questions/hypotheses. ‘Brain dumps’ (describing everything you know about your topic) and ’sermons’ (lecturing readers who are subject matter specialists)!

Thesis Upgrade has produced an accessible and useful publication to help you effectively critique your sourced thesis literature. This digital downloadable resource contains easy-to-understand information and straightforward explanations. Buy now for immediate use.

Still stuck for a research idea? Use your internship, summer vacation or part-time job for inspiration

College students often work part-time, or take vacation jobs, to fund their studies. They also participate in internships to gain experience and get an ‘in’ to an organisation. Apart from providing much-needed money to finance your studies, you can use these experiences to identify, develop and test out your research ideas.

Here are some examples of job-led, theses and dissertations we have come across. Some, which generated, successful, start-up businesses.

Following his primary degree in creative and cultural studies, Stefan went to San Francisco on a J-1 visa. He initially worked as an assistant manager in a bar and then for a large hotel chain. In both places, Stefan noticed missed opportunities by waiting staff to upsell due to a combination of inadequate processes and poor communication. Subsequently, Stefan developed an idea for a mobile application (app) that allows restaurant managers to stay connected with their waiters and the kitchen. The app merges an easy-to-use point of sale system with a two-way communications platform. Stefan subsequently enrolled on a postgraduate programme in business and entrepreneurship to gain an understanding of what was involved in setting up a new enterprise. His consultancy research project identified a further need for the app – to deal with the level of waste in the sector. For example, between 4% to 10% of food ordered never reaches a customer’s table, so food waste in hotel restaurants runs into hundreds of thousands of euros a year. Following completion of his postgraduate studies, Stefan, along with some of his class colleagues, founded a new business. They have just secured a loan to get their idea off the ground.

In year two of her studies, Geraldine, a social studies student, accepted a part-time internship at a charity providing services to people with mental health issues. While her main role was in administration, she also helped with sports activities that took place at the care centre where she worked. This kindled Geraldine’s interest in the relationship between mental health and sport. From a practical perspective, it also afforded her access to more than 150 people with mental health issues who had participated in sport. When Geraldine finished her third-year exams, she returned to the same charity for paid work over her summer vacation. She used this time to start drafting her thesis research proposal and discuss with her manager how she could pilot-test some group interviews. When Geraldine returned to college in the autumn, she was research proposal ready, avoiding the usual thesis pitfalls, thus relieving some of the stress of final year.

Alfonso studied for a BSc in Culinary Entrepreneurship and worked part-time for an Italian take-away. Speaking with customers, he noticed that a considerable number of them used the services because they were too busy to cook at home. This happened even though they preferred to provide home-cooked meals for their family when they had the time. Alfonso spotted both a commercial opening and a research gap. For his dissertation, he produced a research statement investigating the potential target market segment for offering healthy eating products and bespoke culinary expertise. Arising from this, Alfonso generated two business ideas. He is now developing two business plans. One exploring the eating habits of busy professionals and their families, and the other examining the demand for personal chefs in the home.

After graduating with a mathematics and physics teaching diploma, Albert spent almost two years searching unsuccessfully for a teaching post. He eventually landed a temporary job in a patent office. Whilst there, Albert evaluated patent applications for a variety of devices related to transmission of electric signals and electrical-mechanical synchronisation of time. This led him to develop several research hypotheses about the nature of light and the fundamental connection between space and time, in addition to significant research projects supported by the US Government. His full name? Albert Einstein.

Whether you are an Albert, or an Alfonso, the secret is to be constantly on the look-out for thesis and dissertation possibilities. You often hear that research was inspired by customer service difficulties or because of problems with internal systems. Every organisation has its challenges and issues, and using a structured process to solve them, pays dividends for research students.

Such a systematic and structured approach to your research is particularly helpful at the beginning of your research journey. Thesis Upgrade’s Developing Your Research Proposal may aid your journey. The digital resource contains easy-to-understand information and straightforward explanations to help you develop a robust and realistic research proposal. Buy now for immediate use.

Stuck for a research topic?

Our research indicates that the key concern of students as they begin their thesis process is choosing a topic. One reason for this is that novice researchers tend to put a limitation on how they generate ideas. They are inclined to look internally, rather than externally, searching in their own brain for subjects and areas to focus on.

An alternative option is to draw on the collective brains of others. This technique is known as ‘intellectual pinching’! It involves taking a structured approach to noticing the main, external opportunities and threats that an individual, team, organisation, sector or country, faces. It is best to rely on an established internet browser tool, such as Google Analytics, to ascertain these key contextual issues. Search using broad phrases like ‘business trends in X sector 2022’. Your collated insights can then be converted into a realistic research topic.

Here are two examples of ‘intellectual pinching’ our expert team of leading academics have used when guiding research students in recent months.

Yvonne is in the final year of her BSc in Agricultural Economics degree at a UK university. She is planning to complete her thesis over the summer months. Her initial idea was that she would like to base her research on one of the major external changes affecting managers in the agribusinesses in Scotland. Working with her, using a structured and systematic search on Microsoft Edge, our results directed us to the websites of some the world’s largest consulting firms. We found numerous reports on the environmental challenges that such firms predict will impact organisations over the next decade. Yvonne used one such report from Deloitte about food security to identify a topic related to her studies. The report suggested that the world’s food systems should be inclusive, sustainable, efficient and nutritious. Yvonne felt that in a post-Covid, post-Brexit, trading environment, most Scotland-based agribusiness managers would need to pay more attention to these four policy goals. She is now exploring the extant literature on the topic. Her core research question is to examine the preparedness of the agribusinesses in Scotland to produce nutritious cheese-based food, enabling the consumption of a diverse range of healthy and safe snacks.

A postgraduate, part-time student, Peter, works as a technician at a large, coal-fired power station in Indiana, USA. He is studying for an MBA, which he hopes will give him the opportunity to be promoted into a management position. When helping Peter to search for relevant industry research, we retrieved leading-edge studies about the energy industry. We found that hydro-cracking – the splitting of water (H2O) into hydrogen (H) and oxygen (O) to provide energy – was being mentioned increasingly in trend reports. Reading around the topic, Peter found that historically hydrogen cracking has been technically feasible, but uncommercial. The main reason for this is that hydrogen cracking currently involves using processes that require more energy to be put in than is taken out. Recent scientific breakthroughs with new solar heat collecting materials, however, have the potential to reverse this. Using new materials could mean that less energy goes into hydrogen cracking than comes out. Over the next decade, it is likely that power companies will be able to use these new materials on large-scale solar panels to produce cheaper, cleaner energy. Peter is now investigating the feasibility of basing his consultancy project on whether his employer could retrofit the existing oil-fired power station to make use of this novel source of energy production as it comes on stream.

With thesis or dissertation research, you do not always have to think of an original idea yourself. As Isaac Newton said, ‘if I have seen further, it is by standing on the shoulders of giants’. His statement is often used to symbolise scientific progress. Just as the space race gave us innovative products, such as wireless headsets and insulin pumps, your research can build on the groundwork laid done by others. You could, for instance, explore how best to market an existing product or service to a new customer base in an emerging market, or examine the feasibility of implementing an established policy from Head Office to a regional office. So, if you are still stuck for an idea for your thesis or dissertation, remember to use intellectual pinching to generate a topic.

Alternatively, you could buy our downloadable Developing Your Research Proposal Toolbook for immediate use. This informative handbook contains useful activities and practical checklists that help build your skills to develop a realistic, and feasible, research proposal.

Volunteering as a book or periodical editor or reviewer can help with your thesis

Students writing their thesis, or dissertation, are often criticised for providing a ‘summary’, rather than a ‘review’, of the existing literature for their topic. A common comment by examiners and supervisors is that researchers show a thorough knowledge and understanding of the literature. This is characterised by definition, description and explanation, rather than analysis, application and integration. In other words, students often paraphrase the original source material, rather than use advance reviewing skills such as classifying, comparing, contrasting and critiquing.

One way you gain the skills to help you to become a more critical reader and writer is to become a reviewer of academic books and periodicals. By peer-reviewing other authors’ works, you can learn how to examine, assess and evaluate their aims, methods and outcomes. This, in turn, will equip you with skills to summarise, reframe and critically review the work of others.

All academic publishers strive to publish titles on the most cutting-edge research areas. These publications are often niche in nature, and feature robust research, targeted reflections and summaries of trending topics across academic disciplines. Such a rigorous approach is the product of a two-stage, peer-review. First, authored book and periodical publications are subject to an internal appraisal process. Second, the publisher calls on external reviewers to evaluate these works, thus complementing their internal process. This supports the publishers in determining the merit, marketability and viability of the material.

Every paper, article, book chapter or textbook (as well as other materials) that is published also needs to be reviewed externally by peers with subject matter expertise in that area. For example, the majority of the main academic publishing houses often employ graduates working on a part-time basis. They assess published academic books and articles, write accurate summaries or abstracts for inclusion in their databases, and assist with citation checks, scoring, etc. Equally, many college newspapers, magazines and libraries include book/article review sections in their own periodicals and on their websites. They also need people to act as editors and reviewers of articles from contributors. This is an activity you can do both when researching your dissertation and after your thesis is done.

Most editors and reviewers are active members of the academic and research community, such as lecturers, tutors, doctoral researchers, as well as postgraduate and undergraduate students. They apply to, or are approached by, publishers, database owners, and so on. Appointments as reviewers usually run for a one-year term. A reviewer is expected to complete an agreed number of article/chapter/book evaluations per academic year. The work itself is relatively straightforward once you have the requisite subject matter expertise. Draft or completed manuscripts for publication are sent to you electronically. You are requested to complete a set of prescribed closed and open-ended, evaluation questions, with as much detail as possible. Your response should be fulsome and timely, and framed in a collegial and constructive manner.

Volunteering or applying for these roles can give you invaluable publishing and reviewing exposure. They are an effective way of acquiring skills that will help you to search for and review literature, tasks that you also have to conduct as part of your thesis journey. As you become more experienced, you may be able to earn more tangible rewards. For instance, reviewers typically receive an honorarium, complimentary copies of books or free subscriptions to leading periodicals. Sometimes, database owners and college magazines employ reviewers on a part-time basis and pay a (modest) salary. More importantly, while doing this, you will be developing expertise in the evaluation process for authored books and periodicals. This should make the process of researching and writing your thesis much easier and a less taunting task.

To help you write your thesis, we have produced Doing Your Thesis – A Practical Guide. This most popular digital resource contains 90 pages of easy-to-understand information and straightforward explanations to help you research and write a compelling thesis. Buy now for immediate use.

Piling on CRAAP – Uncover a useful literature review search tool

A student recently emailed us a question: “What should I do if my supervisor has told me I need to take a more critical approach to evaluating the material I have searched for my literature review? I don’t really understand what this means.” This student is not alone, as some of the most frequently asked questions from researchers doing their thesis literature review relate to how to go about finding and evaluating useful information. We recommend one simple literature review search tool that can help you to single out the good stuff from what should be flushed out. It is called the ‘CRAAP’ Test.

‘CRAAP’ is a mnemonic for Currency, Relevance, Authority, Accuracy and Purpose. It is a ‘test’ because you can use it to filter out the useful from the useless when choosing what to include in your literature review. Although it was originally intended to be employed by academic researchers to evaluate internet sites, you can apply the CRAAP Test to any book, paper or article you are considering using.

The literature review search tool test itself is straightforward. It involves giving a mark out of 20 for each of the five letters in the mnemonic and then adding the five scores together to give an overall mark out of 100. You then focus on the materials searched for that have achieved the highest scores and discard those with the lowest. This is a quick and easy way of reducing the amount of reading you do while efficiently and effectively searching the literature.

Consider the following example. Say you are in the early stages of researching the topic of ‘machine learning and its application in the automotive industry’ and you visit your college library. The librarian suggests several academic resources, plus an artificial intelligence (AI) magazine targeted at a technical audience. Browsing through back issues of the magazine, you come across an article about future trends in machine learning from an earlier edition, two years. It looks promising, so you decide to utilise the CRAAP Test.

The first letter (that is, ‘C’) stands for Currency. It refers to the timeliness of the material (that is, how new or recent the information is). Generally, you score material highly (15-20 marks) if it has been written in the past three years, been recently revised or updated, and is obviously current. By contrast, you score material low (1-5 marks) if it is 20+ years old and evidently out-of-date. With machine learning, things are changing rapidly, so even an article published two years ago might be considered somewhat old. You, therefore, might decide to place such material at the lower end of the top (category 15-20 marks) and give it a score of, say, 15.

R for Relevance relates to the importance of the information for your specific needs. High marks should be given where the information relates closely to your topic and helps you provide a direct answer to your research questions. Furthermore, score the material highly if it has been written for an academic audience and is pitched at the right level (that is, not too elementary or advanced for your needs). Again, you give a mark out of 20. At this initial stage of your research, a relatively recent article that discusses future trends seems interesting. You may, however, note that it is not associated with the automotive sector, so you give it a score of 17.

Authority covers the provenance of the source of the information. A useful place to start when evaluating this is with the author. For example, are they well known in the topic area, have they published before, what is their job title, which organisation do they work for, what faculty/school or department are they involved in, what qualifications does they have, and so on? In addition, it can be beneficial to assess the publisher. For instance, are they an acknowledge publisher in the academic field, does their publication use peer review, is the material sponsored by any person or organisation, is there contact information (such as a publisher or email address)? If searching online, check if the URL reveals anything about the author or source. For example, .com is a commercial extension, .edu and .ac are educational sites, and .gov refers to the US government.

A high-scoring source is likely to be a peer-reviewed piece by a well-known doctoral or postgraduate-level educator, from a recognised institution, published in an academic journal. In the case of the AI magazine article previously referred to, the authors are well-known lecturers and researchers. They work in relevant departments at respected universities that have high-profile research centres in the areas of machine learning. The AI magazine is a widely disseminated publication, produced by a long-established not-for-profit organisation, and read by practicing scholars and industry professionals. All articles submitted to it are peer-reviewed and approved by an editorial board. Again, a high score is appropriate here, say 18.

Accuracy refers to the reliability, truthfulness and correctness of the content. You should ask where does the data or information used come from? Is the information supported by evidence? Has the information been reviewed or refereed? Does the language or tone seem biased and free of emotion? Are there spelling, grammar or other typographical errors? It can be difficult to give a score for this, as any marks given are relative. Ask yourself, can you verify any of the information from another source or from personal knowledge? In this case, although the authors and publisher of the article appear to have high-standing, and the article itself is peer-reviewed, it is not based on a specific study. It also seems to contain opinion and projection. Nonetheless, it refers to several empirical and well-researched studies to support its central points. Despite its overall credentials, it will score lower than a paper based on a full research study. The nature and style of the article leads to a lower score for accuracy, of say, 13.

Finally, Purpose is linked to the reason the information exists. You should ask what is the intent of the information (that is, to inform, teach, sell, entertain or persuade)? Do the authors/publishers/sponsors make their intentions, or aim, clear? Is the information ‘fact’, opinion or just propaganda? Does the point of view appear objective and impartial? Are there political, ideological, cultural, religious, institutional or personal biases? In contrast to the accuracy score, the article under review might do well here. This is because the overall purpose of the AI magazine is to popularise the discipline. Thus, the authors would be expected to adopt simpler language and a chatty style, rather than the more formal language of an academic paper. Overall, the article would get a score of 15 for purpose

By scoring each category on a scale from 1 to 20 (1 = worst, 20 = best possible), you can give each piece of literature. For example, a book receives an overall score of 78 (C= 15, R = 17, A = 18, A = 13, P = 15). This can be related to:

  • 90 – 100: Excellent (it is essential that I include this in my review).
  • 80 – 89: Very good (it is likely that I will use this, perhaps in conjunction with other material).
  • 70 – 79: Good (this literature may support or link to the main material above).
  • 60 – 69: Moderate (I may mention this or use it to expand on the above.
  • 50 – 59: Average (I will only use this if I need to lengthen a section or broaden a discussion).
  • 40 – 49: Borderline (I will probably not use this but will keep it in my database just in case).
  • 30 – 39: Unacceptable (I will keep a note of it, in case it comes up in discussions, only).
  • Below 30: Poor (do not go there!).

In this case, the literature review search tool test scored 78 overall (a ‘good’ score). This means that although the book or paper might not be a core source, it would be suitable to cite in support of the main material. It might also highlight themes that are relevant to the topic.

As the article scores highest on Relevance and Authority, it is appropriate to the topic and the contribution comes from a recognised source. The paper scores lower on Currency, Accuracy and Purpose. Although it is only two years old, there are probably more up-to-date articles, papers and books that discuss trends in machine learning. On its own, this is not too much to be concerned about. However, any academic article should ideally score higher on Accuracy. Some allowance should be made, in this instance, for the fact that the Purpose of the publication is to act as a popular AI magazine.

When writing up your literature review, part of your critique may involve stating what is in the article (citing) but adding a comment on issues related to accuracy. Mentioning such limitations indicates that you are reading widely, in addition to assessing the value of each piece of literature. In fact, this article provides a helpful list of references that could be used to extend your search further so you can source extra material based on empirical, scholarly research.

In summary, applying the literature review search tool – the CRAAP Test – helps you to separate the useful from the useless. It is, therefore, a helpful mnemonic to use during your literature review search and critique.

Where’s my supervisor?

It’s May and you are doing your exams, looking forward to the final one next. You plan to take a short break to rest and recharge afterwards, before you ‘really get going’ with your thesis. But come 1st June when you arrive at your supervisor’s office you find a note is pinned to the door: Away for the public holiday weekend, back next Tuesday. You return the following week to find another note: Out Marking Exam Papers – Back next Monday. You return another week later to find: At Exam Board – Back tomorrow. On your fourth attempt, you find a note: On holidays – Back in September. Aghhhhhhh!

While that might sound like an exaggeration, Thesis Upgrade’s research shows that contact with their supervisor is one of the biggest problems students experience while doing their research.

That issue often begins at the research proposal stage: “In my college supervisors were not appointed until after the research proposal was submitted”, said Peter, who is studying for a BA in Social Sciences. “That meant that there was nobody to talk to, when considering your research area, or deciding on your topic”.

Mark Campbell, Director of Thesis Upgrade, advises: “If your supervisor is not known to you at the research proposal stage you should reach out to potential supervisors – talk to lecturers, tutors and subject matter experts who are familiar with the research area, topic and process”.

Equally, during the first month of your research you may find that your supervisor is unavailable: “During the first month of my research my dissertation supervisor was rarely available” said Sabina, an MBA student. “She was always ‘marking exam papers, attending exam boards, away at a conference, on holidays’ … it made no difference whether I tried to contact her in person at her office, or on the phone or by Microsoft Teams”. It is important to recognise that supervisors also have busy lives and that you may be one of many students trying to contact them, particularly at this time of year. “As a supervisor, I always suggest students send me an email and suggest a time when we can meet, or ask when I am available first, so they won’t waste their time chasing me around the college. As I work off-campus during the summer months, it is often easier to communicate virtually, for example, by Zoom, than to meet in person. Provided we have agreed a time and date to have the meeting!”

“You can feel very alone when your supervisor is on holidays”, said one of our survey respondents, Samantha, who graduated last year with a Postgraduate Diploma in Engineering. “At various stages I found it useful to talk to others who could help, for example the library staff during the literature search and review stage. It was also useful to become friendly with other research students who were working the lab at the college. Even if I didn’t know them, I might offer to buy doctoral student a coffee in exchange for some advice”. “Librarians are another great person to get in contact with, particularly during the early stages of your research such as your literature review”, said Mark Campbell. “They can help you with literature, and usually can show you examples of completed thesis and dissertations. They are also familiar with structure, layout, style and so on”. Most college libraries have a database or catalogue of past theses.

“It’s important to recognise that your supervisor is not the only expert you can turn to” said Brian, a Computer Science graduate: “My research involved a rarely used statistical analysis technique and my supervisor was unfamiliar with it. She suggested that I speak with one of the Research Methods lecturers at the college, which I found very useful. I also got in contact, via an online forum, with another researcher in Canada who was using the same technique, so we were able to compare notes”.

“The best way to deal with the absence of a supervisor is to ask: who else has this problem, or who else can help me with this problem?” says Mark Campbell of Thesis Upgrade. “The answer might include people from your support network, such as family, friends, fellow students, work colleagues, lecturers, tutors, librarians, school administrators, laboratory workers, people in the field … the list is endless. Or it may include resources from supportive sources, such as Thesis Upgrade’s practical research guidance.”

Nationaldagen – Five useful things the Swedes have given research students (apart from ABBA and IKEA)

One of the best times of year to visit Sweden is in June. That’s because the Swede’s have some great ideas about work-life balance that result in two very long, public holiday, weekends in June. Ono celebrate Nationaldagen (National Day), which this year falls on Tuesday, 6 June, and the other, Midsommer, which takes place over the weekend beginning 23 June. These weekends typically begin with a de facto half-day on the first day and continue right through until very late on the fourth! They remind us of the positive effect the Swedish work-lifestyle can have while you are studying or researching.

Here are five useful things we have picked up from working with students from Sweden.

Planning: Good planning is an integral part of the Swedish way of living. IKEA’s devotion to precise design is testimony to this. Many Swedish students and researchers will be able to tell you exactly what they will be doing on any given day, three or four months ahead. Planning doesn’t just apply on the work front but also socially, where planning meetings for group activities are a useful way of socialising, making connections, and collaborating with other students and academic staff (lecturers, librarians, school administrators, etc.). LinkedIn, Facebook Messenger, WhatsApp and countless other apps have not matched the Swedish desire to meet face-to-face and plan. Get into the habit of planning socially, and not just for today, but for next week, next month, and even next year, when necessary.

Lagom (pronounced lar-gohm): This is a Swedish concept meaning ‘just the right amount’, ‘in moderation’, ‘in balance’ and ‘perfect-simple’. Like ABBA’s music, the Swede’s prefer most things in life to be not too hot, not too cold, but just right. Lagom is important when studying or researching your thesis. It is important not to waste too much time on the ‘hot stuff’, but also not to spend too little time or waste good time on the ‘cold stuff’. Getting it ‘just right’ – Lagom – is more likely to earn approval from colleagues in the academic world; too much or too little usually gets the thumbs down.

Fika (coffee breaks): These are an important part of Swedish learning culture. In Sweden it is essential that you take regular breaks from your studies or research to socialise with your colleagues, have a coffee, and catch up on college news in an informal way. The only challenges of fika are gauging what is a lagom amount of time to spend at it (so that you get the maximum information in the least time), getting used to drinking copious amounts of coffee (black, as sugar and milk are considered non-essential), developing an appreciation of bulle (Swedish cinnamon buns), developing a liking for easy-listening music, and avoiding getting cornered by another researcher who wants to talk non-stop about their project. Oh, and watch out for the concept of patar (the second cup) – if the conversation is flowing, be prepared to delay for a while over second cup of coffee.

Punctuality: This differs from culture to culture, but for the Swedes coming late to anything is simply bad. If IKEA delivers something to your home late, that is very bad. Coming late to study or research meetings is perceived as disrespectful Let’s just say that when it comes to collaborating and working together on you must always take the Swedish approach – be there on time.

Preciseness: Swedish, unlike English and some other languages, is a very precise language and the Swedes are not generally known for their linguistic padding or waffle. So, whether you are sending e-mails, asking questions in an interview or requesting respondents to fill out a questionnaire be concise and get rid of unnecessary words. Think of the IKEA assembly manuals as the model – use as few words as possible.

Following the Swedish example pays dividends. Whether you are at the planning, doing or writing-up stage of your thesis or dissertation, what better time to adopt this way of working than on Nationaldagen!

Is your research as reliable as your car?

Recently we posted a blog about valid in research. Since then, we received many requests to explain, in simple terms, the related concept of reliability. So, here goes ….

Picture the scene. You leave your house in the morning and step into your brand-new BMW car. That unique, ‘new car’ smell surrounds you – you can almost taste the freshness. The sound system immediately starts-up, playing light, classical music that is a pleasure to your ears. You sink into the Alcantara covered seats, enjoying the support and comfort that brings. Your eyes are drawn to the soft-glow of the media-screen which politely requests your input. It asks “Destination: Work?”. You press “Accept”. It blinks: “Ready to start?”. You press “Yes”.

And nothing happens.

No engine sounds. No lights. No seat-belt warning sound. Nothing.

Welcome to the world of unreliability.

When it comes to things like cars, computers, TVs, microwave ovens in our lives, we have come to expect a very high level of what is called ‘engineering reliability. This is the ability of a system, or component, to perform its required functions under stated conditions for a specified time. Yet, when it comes to research, many of us pay scant attention to reliability.

Research is only credible if the data you collect, collate and analyse is reliable. In research, reliability refers to the consistency of your findings based on how you went about gathering it. In other words, if your study is repeated at various times or by a variety of researchers, using equivalent methods, techniques and procedures, perhaps in a different setting, it should yield the same, or similar results. In a car, when you turn the key, you expect it to start. In research, when you use the same method under the same conditions, you should expect and get the same results.

What, then, is the key to achieving reliability?

If you are carrying out quantitative research, your data collection, collation and analysis approach should be designed so that it will yield the same results on different occasions or in different settings; and other researchers using the same process should reach similar observations. In other words, you must design for reliability by ensuring that no extraneous variables exist or enter the situation.

For qualitative research, your data collection, collation and analysis process should be conducted in such a way that you will be able to verify the accuracy, credibility, dependability and trustworthiness of the raw data. It should also provide transparency regarding the interpretation, analysis and transferability of that data.

While in most cases, it is not possible to ensure the high levels of reliability we have in engineering, we should strive to make our research as reliable as possible. Effective research design can help us to achieve this.