What is an Empirical Research (ER)?
At the very beginning of my application for the empirical research project of my FYP, my first impression for a project labelled with 'ER' was like an in-between concept combining the theoretical research and software engineering, which meant that the deliverables should include both a workable system and an insightful report (almost like a paper to be published on xxx conference!).
This was a bit naive.
After I faced issues writing my project scheme for the proposal of an ER project, the supervisor for that project was extremely strict about that, since students who got accepted for his project last year all failed to conduct the empirical research, most of them only ended up with a specific software while did not propose a research problem and conduct experiment to solve it - on the other hand, the module convenor for the FYP did not draw out the topic - WHAT EMPIRICAL RESEARCH IS.
Begin With Determining Your Research Objective
The scope defines the boundaries and implies the focus of the research. So pay attention to it. Moreover, it suggests that the system design should follow the first principles so that each module you designed is valid for the experiment.
Example:
A study on workplace productivity might focus on understanding how flexible work hours influence output within a given population.The Framework: Everything is About Your Research Questions (RQs)
Literally, everything you do should revolve around the research questions you’ve identified based on the project description - I admit this is hard, which means that you'll need to break down and brainstorm every detail from a short, few-line project brief.
Notice that the research questions you proposed should reflect on specific problems that have not been covered by existing research. For instance, you want to integrate GenAI for a certain function in a system in order to provide recommendations for users, because this could reduce trial-and-errors. At the same time, you should also perceive that the delays caused by the inference time might on the other hand affect user's experience. All these factors should be considered and manifested in the form of hypotheses corresponding your RQs.
Hypotheses can reflect your bias toward how to verify the RQ, serving as a bridge from your assumptions to the methodology you designed for scientific validation, but most importantly your assumptions should be based on the findings of existing research - otherwise you'll be pitting yourself against the entire academic field, which is not likely plausible.
Normally, your hypotheses will include certain dependent variables (DVs). DVs can be divided into two subgroups, quantitative ones that can be measured and analysed in a mathematical way, qualitative ones which involves users' subjective opinions toward certain functions of your system and are usually be measured by surveys or interviews.
An example template of RQ and the corresponding hypothesis:
Research Question
RQ: Does [Independent Variable A] significantly affect [Dependent Variable B] in the target task?
Hypotheses
H0 (Null Hypothesis): [Independent Variable A] has no significant effect on [Dependent Variable B].
H1 (Alternative Hypothesis): [Independent Variable A] significantly improves/reduces [Dependent Variable B].Be Careful About the Experimental Design
The first thing you should consider is whether you gonna implement a within-subject or a between-subject study design, which is a trade-off between Individual Differences and Order Effects. Simply to judge, if your experiement involves too many conditions or variables, within-subject pattern might be more suitable, otherwise you need plenty of groups and the costs of recruiting participants and conducting the experiment will increase significantly. Additionally, based on experience, minimalism and first principles can always benefit the design of a user study, so do trying to simplify your experiment conditions (independent variables), tasks and DVs.