PSY5012 ASSESSMENT BRIEF
Module: Research Methods in Psychology 2
Assessment Title: Quantitative research report
Word Count: The word count for this assessment is 2,500 words. Your report must not exceed 2,500 words. If you write more than 2,500 words, everything that exceeds the word limit will not be marked (e.g. if you write 2,501 words, the tutor will assess your work based on the first 2,500 words). There are no penalties for writing fewer than 2,500 words. The word count does not include the reference list or any appendices.
Referencing: APA style should be used APA 7th Referencing (leedstrinity.ac.uk)
Assessment Details
The report centres around a hypothetical cohort study with data collected by the module co-ordinator focusing on understanding university student satisfaction. You are tasked with selecting from a series of variables from within a larger datafile provided in week 1 and performing appropriate analysis in order to produce a comprehensive research report. Each week we will work at different components of the project including study design, ethics, measurement techniques, data collection and analysis. In addition to this we will also sequentially look at each section of the report as detailed below. This will culminate in the report which is written individually.
Report sections
The report will consist of assessment of 6 key areas of a research report, namely abstract, introduction, method, results, discussion and references. Each section is individually weighted given the magnitude of its importance to the assessment.
Abstract (5%)
Introduction
The rationale provided in Introduction is weighted (10%).
Clarity and precision of the hypotheses (5%)
Method
Design, Materials, Participants, Procedure
Overall quality of the Method section (25%)
Results (25%)
Discussion: Consideration and interpretation of theoretical and practical implications of the findings (20%)
References: Adherence to APA-style referencing (5%)
Starting Point
Before you start writing your report, review the feedback that you received on your level 4 report. If there is feedback that would apply to this report, make sure you note and act upon it. Follow the advice in the Writing Guide, as this applies directly to experimental reports. There are also some notes in the module booklet that you should read and make use of. The remainder of this guide will cover specific points that are not covered elsewhere. If you are not sure about any aspect of the report, including the interpretation and reporting of the analysis, despite having read through this guide and referring to an appropriate statistics book, then please ask for help.
Introduction
This should be straightforward as you have written introductions pieces of work previously. But to summarise you will need to explain the following points to the reader:
- What literature was the experiment based upon?
- How is the rationale developed?
- Why was the work designed?
A frequently-asked question is “how many references do I need to include?”. The answer is that you are explaining the background to the piece of work and what your design and associated decisions were based on. You should include whatever sources you used to inform the design of your experiment, as you cannot work backwards and provide a post-hoc rationalisation for an experiment that has already been designed and conducted. If you try to do this, your rationale becomes weaker and the argument much less convincing.
Method
Think about what you are writing and what the reader needs to know (or doesn’t need to know!). You may have specific feedback about this in your Semester 1 report.
Focus on the details that are necessary for the reader to understand and be able to replicate your experiment (e.g. “data was collected in in MHLT” is meaningless to most people but “The experiment was conducted during a level 5 teaching session” is informative. Focus on clarity and conciseness.
Design
Describing and Explaining Variables
Define and explain the independent and dependent variables. Be clear and precise. Don’t be tempted to try and write in soundbites.
Here’s a bad example:
“One independent variable was the size and the dependent variable was the response time”
The size of what? How was it changed? How many different sizes were there and what were they?
The dependent variable was the response time to do what? Explain what the task involved and what was being measured.
The purpose is to explain, not merely to label.
Participants
Sample size is all-important, so don’t forget to mention this. It’s surprising how often this gets missed out. Include ages and gender split if you recorded them. Otherwise, do not guess.
Participants are very rarely recruited ‘randomly’, and in the case of this experiment, there is a very systematic way of recruiting participants as they will have all been L5 students participating as part of a practical class. Explain, in plain English, how they were recruited. Don’t just throw in a term (e.g. ‘opportunity sampling’) without explaining the details. If participants were allocated to different conditions, explain how this was done. Note: you do not need to describe using G*power to calculate your sample size at this point.
Accurate Description of Materials
Focus on the detail that would be needed to allow the development of ‘equivalent’ materials. Therefore, we want you to correctly make use of appendices, because it can lead you into bad habits if used at the expense of including key information within the report body. For example, “There were thirty words, they are listed in the appendix” is an easy trap to fall into. Having a list is nice, but it doesn’t explain anything. If offered the choice between a full list or an explanation of the materials plus a few examples, you should go for the explanation plus examples every time.
Remember that the reader should not have to guess about any important details. The underlying principle here is that if the reader has to guess, and they guess wrongly, if it would fundamentally change the nature of the experiment, then your report has not been successful.
Note that examples are necessary, but not enough. It must be clear what they are examples of and why they are examples of it.
It is not just the ‘what’, but the ‘why’ that is important
Describing Procedure
This is the procedure for the work conducted (i.e. not for the practical class, so we don’t want “we were split into groups in the practical class and then we considered what we were going to do…. and we took part in each other’s experiments…”)
Explain the experience of a participant from the start of the experiment through to the end.
Tested individually or in groups?
What were they told was going to happen? What were they told they had to do? What did they see? What did they do?
Asking someone to do something is not the same as them doing it.
Describe the sequence of events in each individual trial, including durations of presentation, etc.
What is necessary for replication? What is unnecessary?
Assume that the reader has no prior knowledge of your experiment, so focus on providing clear, accurate explanations of the aspects necessary for replication of the experiment.
You do not need to describe how participants read an information sheet or signed a consent form. These are common to all studies and do not tell the reader anything specific about your experiment and how it was conducted (or how it could be replicated).
Here’s an example of how to describe a type of experiment. This is from: Jones, S. (2015). The mediating effects of facial expression on spatial interference between gaze direction and gaze location. The Journal of General Psychology, 142(2), 106-117. Be mindful that this example refers to an experimental piece of work, however the basic nature of what a procedure is designed to do is common to the work that you are reporting on in the student experience survey.
Procedure
In both of the experiments reported here, participants were seated approximately 60 cm from the computer screen. They were informed that the study was investigating how people process and identify the direction in which other people’s eyes are looking, and that the task involved looking at a series of faces, presented one at a time, and identifying, as quickly and as accurately as possible, whether the face’s eyes were looking to the left or right from the participant’s point of view.
Each trial began with a white fixation cross presented in the centre of a green (to match the background of the stimulus images) screen for 500 ms. This disappeared simultaneously with the presentation of a stimulus face to the left or right of the screen. The distance from the fixation point to the centre of the stimulus was 11 cm. The stimulus remained on-screen until the participant made a response.
Participants were instructed to press the ‘L’ button on the response box if the eyes were directed to their left, and the ‘R’ button if the eyes were directed to the right.
Experiment 1 began with a practice block of 8 trials, with visual feedback provided after each trial informing participants whether the response had been correct. Following the practice block, participants undertook four experimental blocks, each of 32 trials, with a 30-second rest in-between each block. Each block of 32 trials contained each combination of stimulus face and congruency, presented on each side of the screen. The order of stimulus presentation was randomised for each participant.
Reporting the Results
There are several stages involved in reporting the results. Here these sections are numbered, but don’t separate these out in your report, and certainly don’t number them. They are presented in this format to highlight their features.
- Initial Treatment of Data
Were any data points excluded? Explain what these where and why it was important to remove them. You will have made decisions about the individual participants data, so explain these decisions here. Were there any incomplete data sets from individual participants?
- Tests of Normality
Now that you have the summarised data for each participant, you will have checked the shapes of the distributions using a test of normality, coupled with an inspection of the descriptive statistics (e.g. skew and kurtosis). You don’t need to dwell on this too much and a sentence like the following would suffice: “A Shapiro-Wilk test of normality showed that the data were normally distributed, for all of the conditions”. If, however, there are problems with normality, explain what they are and what (if anything) you did about them.
- Trimming & Transforming
If one or more scales/variables was not normally distributed, you might have identified outliers. If so, explain how they were identified (how many, which variables) and removed (why?) or left (why?).
It is unlikely that you will have needed to transform the data, but if so, then explain that (for example): “A logarithmic (log10) transformation was successfully applied”. Bear in mind that in level 6 (for your dissertation) you may be required to perform this additional step in normalising a data set.
- Descriptives Table (and written description)
Note: It does not make sense to report the means or SDs until you have sorted out the data in steps 1-3 above. Quite often, we find that students present a table of means and SDs, and then explain how the scores were calculated, cases removed, and so on. Just present the information in the order it occurred. You cannot calculate the means and SDs for the final analysis until you have decided what the final data set will consist of.
Present a table in APA style (see Writing Guide). A graph can be helpful to illustrate an interaction (or the lack of one) as it is easier to see the patterns in a graph than in a table. Include either a table or a graph, but if you use a graph make sure to report the means and SDs in the text. The nature of the graph will greatly depend upon the type of analysis that you elect to perform on the data. Finally, after visually displaying the findings from the work, describe the pattern of results in words (e.g. say which of your conditions produced the longest response time). Again, the example in the Writing Guide will be useful.
- Analysis Outcomes
Report the analysis you select in line with example given in the Writing Guide/week of study on specific analysis technique. When reporting the analysis, name the test being used say whether the result was significant (e.g. for each main effect and the interaction). Present the test statistic for each effect (e.g. F(2,24)=3.45, p = .032) Note, this is not a ‘formula’, it is merely a representation of the key figures from the statistical test. With just a little thought (and revisiting the handout) you should be able to get the right numbers in the right places.
Include the effect sizes (partial eta squared) as in the Writing Guide.
Post-Hoc Tests (if appropriate)
If your interaction was significant, you need to report the post-hoc analyses used to explore it. Again, name the test being used and explain the outcomes along with the appropriate significance levels. Briefly summarise the results and explain whether they support the hypothesis or hypotheses.
General Advice for the Results
Your results section should resemble the examples in the Writing Guide
Explain the results in a logical order. Means and SDs are not calculated until AFTER tests of normality, outliers removed, etc, so it does not make sense to present the means and SDs before you have described all the other steps.
Do not paste SPSS tables into the results section – Make a table of means and SDs in Word and label it clearly. Do not just mimic whatever format SPSS has provided you with as this might not be the best way of displaying the data.
Discussion
Consider what the results mean in terms of theory. Your experiment will have been developed from previous studies, which all found things and explained what they meant. You should be doing the same by relating your results back to those other studies.
Don’t focus on trivial criticisms and remember that you (hopefully) performed a power analysis beforehand to ensure you had a sufficiently large sample (or, just as importantly, you know what sample size would have been ideal if it was larger than the one you could achieve). Do not, therefore, conclude that a ‘bigger sample’ would be needed, as this is not necessarily true and, in any case, is too vague to be useful.
Focus on what your results mean. Read journal articles and model your discussion sections on theirs.
